We study a general class of nonlinear and shift-varying smoothing filters that operate based on averaging. This important class of filters includes many well-known examples such as the bilateral filter, nonlocal means, general adaptive moving average filters, and more. (Many linear filters such as linear minimum mean-squared error smoothing filters, Savitzky--Golay filters, smoothing splines, and wavelet smoothers can be considered special cases.) They are frequently used in both signal and image processing as they are elegant, computationally simple, and high performing. The operators that implement such filters, however, are not symmetric in general. The main contribution of this paper is to provide a provably stable method for symmetrizing the smoothing operators. Specifically, we propose a novel approximation of smoothing operators by symmetric doubly stochastic matrices and show that this approximation is stable and accurate, even more so in higher dimensions. We demonstrate that there are several important advantages to this symmetrization, particularly in image processing/filtering applications such as denoising. In particular, (1) doubly stochastic filters generally lead to improved performance over the baseline smoothing procedure; (2) when the filters are applied iteratively, the symmetric ones can be guaranteed to lead to stable algorithms; and (3) symmetric smoothers allow an orthonormal eigendecomposition which enables us to peer into the complex behavior of such nonlinear and shift-varying filters in a locally adapted basis using principal components. Finally, a doubly stochastic filter has a simple and intuitive interpretation. Namely, it implies the very natural property that every pixel in the given input image has the same sum total contribution to the output image.


  1. nonparametric regression
  2. data smoothing
  3. filtering
  4. stochastic matrices
  5. applications of Markov chains
  6. positive matrices
  7. Laplacian operator
  8. applications of graph theory

MSC codes

  1. 62G08
  2. 93E14
  3. 93E11
  4. 15B51
  5. 60J20
  6. 15B48
  7. 35J05
  8. 05C90

Get full access to this article

View all available purchase options and get full access to this article.


J. M. Aldaz, Concentration of the ratio between the geometric and arithmetic means, J. Theoret. Probab., 23 (2010), pp. 498--508.
S. P. Awate and R. T. Whitaker, Unsupervised, information-theoretic, adaptive image filtering for image restoration, IEEE Trans. Pattern Anal. Mach. Intell., 28 (2006), pp. 364--376.
M. Belkin and P. Niyogi, Laplacian eigenmaps for dimensionality reduction and data representation, Neural Comput., 15 (2003), pp. 1373--1396.
R. Bhatia, Perturbation Bounds for Matrix Eigenvalues, Classics Appl. Math. 53, SIAM, Philadelphia, 2007.
C. Bordenave, P. Caputo, and D. Chafai, Circular law theorem for random Markov matrices, Probab. Theory Related Fields, 152 (2012), pp. 751--779.
R. A. Brualdi, Matrices of $0$'s and $1$'s with total support, J. Combin. Theory Ser. A, 28 (1980), pp. 249--256.
A. Buades, B. Coll, and J. M. Morel, A review of image denoising algorithms, with a new one, Multiscale Model. Simul. 4 (2005), pp. 490--530.
P. Buhlmann and B. Yu, Boosting with the $L_2$ loss: Regression and classification, J. Amer. Statist. Assoc., 98 (2003), pp. 324--339.
A. Buja, T. Hastie, and R. Tibshirani, Linear smoothers and additive models, Ann. Statist., 17 (1989), pp. 453--510.
D. Chafai, Aspects of large random Markov kernels, Stochastics, 81 (2009), pp. 415--429.
P. Chatterjee and P. Milanfar, Is denoising dead?, IEEE Trans. Image Process., 19 (2010), pp. 895--911.
P. Chatterjee and P. Milanfar, Patch-based near-optimal denoising, IEEE Trans. Image Process., 21 (2012), pp. 1635--1649.
A. Cohen, All admissible linear estimates of the mean vector, Ann. Math. Statist., 37 (1966), pp. 458--463.
R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker, Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps, Proc. Natl. Acad. Sci. USA, 102 (2005), pp. 7426--7431.
I. Csiszar, I-divergence geometry of probability distributions and minimization problems, Ann. Probab., 3 (1975), pp. 146--158.
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse $3$-D transform-domain collaborative filtering, IEEE Trans. Image Process., 16 (2007), pp. 2080--2095.
J. Darroch and D. Ratcliff, Generalized iterative scaling for log-linear models, Ann. Math. Statist., 43 (1972), pp. 1470--1480.
G. Deng and L. Cahill, An adaptive Gaussian filter for noise reduction and edge detection, in Nuclear Science Symposium and Medical Imaging Conference, IEEE Conference Record, Vol. 3, 1993, pp. 1615--1619.
J. Digne, J.-M. Morel, C.-M. Souzani, and C. Lartigue, Scale space meshing of raw data point sets, Comput. Graph. Forum, 30 (2011), pp. 1630--1642.
A. Dimakis, S. Kar, J. Moura, M. Rabbat, and A. Scaglione, Gossip algorithms for distributed signal processing, Proc. IEEE, 98 (2010), pp. 1847--1864.
M. Elad, On the origin of the bilateral filter and ways to improve it, IEEE Trans. Image Process., 11 (2002), pp. 1141--1150.
C. Fowlkes, S. Belongie, F. Chung, and J. Malik, Spectral grouping using the Nyström method, IEEE Trans. Pattern Anal. Mach. Intell., 26 (2004), pp. 214--225.
E. Gluskin and V. Milman, Note on the geometric-arithmetic mean inequality, in Geometric Aspects of Functional Analysis, Lecture Notes in Math. 1807, Springer, Berlin, 2003, pp. 130--135.
G. Goldberg and M. Neumann, Distribution of subdominant eigenvalues of matrices with random rows, SIAM J. Matrix Anal. Appl., 24 (2003), pp. 747--761.
G. Goldberg, P. Okunev, M. Neumann, and H. Schneider, Distribution of subdominant eigenvalues of random matrices, Methodol. Comput. Appl. Probab., 2 (2000), pp. 137--151.
W. Härdle, Applied Nonparametric Regression, Cambridge University Press, Cambridge, UK, 1990.
G. Hardy, J. E. Littlewood, and G. Pólya, Inequalities, 2nd ed., Cambridge University Press, Cambridge, UK, 1988.
T. Hastie and R. Tibshirani, Bayesian backfitting, Statist. Sci., 15 (2000), pp. 196--223.
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed., Springer, New York, 2009.
R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, Cambridge, UK, 1990.
M. Horvat, The ensemble of random Markov matrices, J. Stat. Mech. Theory Exp., No. 7, (2009), P07005.
C. R. Johnson and R. B. Kellogg, An inequality for doubly stochastic matrices, J. Res. Nat. Bur. Standards. Sect. B, 80 (1976), pp. 433--436.
C. Kervrann and J. Boulanger, Optimal spatial adaptation for patch-based image denoising, IEEE Trans. Image Process., 15 (2006), pp. 2866--2878.
R. Khoury, Closest matrices in the space of generalized doubly stochastic matrices, J. Math. Anal. Appl., 222 (1998), pp. 562--568.
S. Kindermann, S. Osher, and P. W. Jones, Deblurring and denoising of images by nonlocal functionals, Multiscale Model. Simul., 4 (2005), pp. 1091--1115.
P. A. Knight, The Sinkhorn--Knopp algorithm: Convergence and applications, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 261--275.
A. Levin and B. Nadler, Natural image denoising: Optimality and inherent bounds, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2011, pp. 2833--2840.
D. Levin, The approximation power of moving least-squares, Math. Comp., 67 (1998), pp. 1517--1531.
P. Milanfar, A tour of modern image filtering, IEEE Signal Processing Mag., 30 (2013), pp. 106--128.
N. Nordstrom, Biased anisotropic diffusion---a unified regularization and diffusion approach to edge detection, Image Vision Comput., 8 (1990), pp. 318--327.
E. J. Nyström, Über die praktische Auflösung von linearen Integralgleichungen mit Anwendungen auf Randwertaufgaben der Potentialtheorie, Comment. Phys.-Math., 4 (1928), pp. 1--52.
S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul., 4 (2005), pp. 460--489.
G. Peyré, Image processing with nonlocal spectral bases, Multiscale Model. Simul., 7 (2008), pp. 703--730.
E. Seneta, Non-Negative Matrices and Markov Chains, Springer Ser. Statist., Springer, NewYork, 1981.
J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 22 (2000), pp. 888--905.
R. Sinkhorn, A relationship between arbitrary positive matrices and doubly stochastic matrices, Ann. Math. Statist., 35 (1964), pp. 876--879.
R. Sinkhorn and P. Knopp, Concerning nonnegative matrices and doubly stochastic matrices, Pacific J. Math., 21 (1967), pp. 343--348.
A. Spira, R. Kimmel, and N. Sochen, A short time Beltrami kernel for smoothing images and manifolds, IEEE Trans. Image Process., 16 (2007), pp. 1628--1636.
W. J. Stewart, Introduction to the Numerical Solution of Markov Chains, Princeton University Press, Princeton, NJ, 1994.
H. Takeda, S. Farsiu, and P. Milanfar, Kernel regression for image processing and reconstruction, IEEE Trans. Image Process., 16 (2007), pp. 349--366.
C. Tomasi and R. Manduchi, Bilateral filtering for gray and color images, in Proceedings of the 1998 IEEE International Conference on Computer Vision, Bombay, India, 1998, pp. 836--846.
J. W. Tukey, Exploratory Data Analysis, Addison-Wesley, Reading, MA, 1977.
M. P. Wand and M. C. Jones, Kernel Smoothing, Monogr. Statist. Appl. Probab., Chapman and Hall, London, 1995.
J. Weickert, Coherence-enhancing diffusion, Int. J. Comput. Vision, 31 (1999), p. 111--127.
L. P. Yaroslavsky, Digital Picture Processing, Springer-Verlag, Berlin, 1985.
R. Zass and A. Shashua, Doubly stochastic normalization for spectral clustering, in Advances in Neural Information Processing Systems (NIPS), MIT Press, Cambridge, MA, 2006, pp. 1569--1576.

Information & Authors


Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 263 - 284
ISSN (online): 1936-4954


Submitted: 3 May 2012
Accepted: 2 October 2012
Published online: 12 February 2013


  1. nonparametric regression
  2. data smoothing
  3. filtering
  4. stochastic matrices
  5. applications of Markov chains
  6. positive matrices
  7. Laplacian operator
  8. applications of graph theory

MSC codes

  1. 62G08
  2. 93E14
  3. 93E11
  4. 15B51
  5. 60J20
  6. 15B48
  7. 35J05
  8. 05C90



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By







Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.