A Symmetry Preserving Algorithm for Matrix Scaling
Abstract
We present an iterative algorithm which asymptotically scales the $\infty$-norm of each row and each column of a matrix to one. This scaling algorithm preserves symmetry of the original matrix and shows fast linear convergence with an asymptotic rate of 1/2. We discuss extensions of the algorithm to the 1-norm, and by inference to other norms. For the 1-norm case, we show again that convergence is linear, with the rate dependent on the spectrum of the scaled matrix. We demonstrate experimentally that the scaling algorithm improves the conditioning of the matrix and that it helps direct solvers by reducing the need for pivoting. In particular, for symmetric matrices the theoretical and experimental results highlight the potential of the proposed algorithm over existing alternatives.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
E. Achilles, Implications of convergence rates in Sinkhorn balancing, Linear Algebra Appl., 187 (1993), pp. 109--112.
2.
P. R. Amestoy, I. S. Duff, J.-Y. L'Excellent, and J. Koster, A fully asynchronous multifrontal solver using distributed dynamic scheduling, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 15--41.
3.
P. R. Amestoy, I. S. Duff, D. Ruiz, and B. Uçar, A parallel matrix scaling algorithm, in 8th International Conference on High Performance Computing for Computational Science - VECPAR 2008, Lecture Notes in Comput. Sci. 5336, Springer-Verlag, Berlin, Heidelberg, 2008, pp. 301--313.
4.
P. R. Amestoy, A. Guermouche, J.-Y. L'Excellent, and S. Pralet, Hybrid scheduling for the parallel solution of linear systems, Parallel Comput., 32 (2006), pp. 136--156.
5.
F. L. Bauer, Optimally scaled matrices, Numer. Math., 5 (1963), pp. 73--87.
6.
F. L. Bauer, Remarks on optimally scaled matrices, Numer. Math., 13 (1969), pp. 1--3.
7.
A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathematical Sciences, SIAM, Philadelphia, 1994.
8.
G. D. Birkhoff, Tres observaciones sobre el algebra lineal, Universidad Nacional de Tucuman Revista, Serie A, 5 (1946), pp. 147--151.
9.
A. Borobia and R. Cantó, Matrix scaling: A geometric proof of Sinkhorn's theorem, Linear Algebra Appl., 268 (1998), pp. 1--8.
10.
R. A. Brualdi, S. V. Parter, and H. Schneider, The diagonal equivalence of a nonnegative matrix to a stochastic matrix, J. Math. Anal. Appl., 16 (1966), pp. 31--50.
11.
R. A. Brualdi and H. J. Ryser, Combinatorial Matrix Theory, Encyclopedia Math. Appl. 39, Cambridge University Press, Cambridge, New York, 1991.
12.
J. R. Bunch, Equilibration of symmetric matrices in the max-norm, J. Assoc. Comput. Mach., 18 (1971), pp. 566--572.
13.
J. R. Bunch and B. N. Parlett, Direct methods for solving symmetric indefinite systems of linear equations, SIAM J. Numer. Anal., 8 (1971), pp. 639--655.
14.
A. R. Curtis and J. K. Reid, On the automatic scaling of matrices for Gaussian elimination, IMA J. Appl. Math., 10 (1972), pp. 118--124.
15.
E. D. Dolan and J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), pp. 201--213.
16.
I. S. Duff, A. M. Erisman, and J. K. Reid, Direct Methods for Sparse Matrices, Oxford University Press, London, 1986.
17.
I. S. Duff and J. Koster, The design and use of algorithms for permuting large entries to the diagonal of sparse matrices, SIAM J. Matrix Anal. Appl., 20 (1999), pp. 889--901.
18.
I. S. Duff and J. Koster, On algorithms for permuting large entries to the diagonal of a sparse matrix, SIAM J. Matrix Anal. Appl., 22 (2001), pp. 973--996.
19.
I. S. Duff and S. Pralet, Strategies for scaling and pivoting for sparse symmetric indefinite problems, SIAM J. Matrix Anal. Appl., 27 (2005), pp. 313--340.
20.
G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., The Johns Hopkins University Press, Baltimore, MD, 1996.
21.
P. A. Knight, The Sinkhorn--Knopp algorithm: Convergence and applications, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 261--275.
22.
P. A. Knight and D. Ruiz, A fast algorithm for matrix balancing, IMA J. Numer. Anal., 33 (2013), pp. 1029--1047.
23.
O. E. Livne and G. H. Golub, Scaling by binormalization, Numer. Algorithms, 35 (2004), pp. 97--120.
24.
B. N. Parlett and T. L. Landis, Methods for scaling to double stochastic form, Linear Algebra Appl., 48 (1982), pp. 53--79.
25.
H. Perfect and L. Mirsky, The distribution of positive elements in doubly-stochastic matrices, J. London Math. Soc., 40 (1965), pp. 689--698.
26.
U. G. Rothblum, H. Schneider, and M. H. Schneider, Scaling matrices to prescribed row and column maxima, SIAM J. Matrix Anal. Appl., 15 (1994), pp. 1--14.
27.
D. Ruiz, A Scaling Algorithm to Equilibrate Both Rows and Columns Norms in Matrices, Technical Report RAL-TR-2001-034 and RT/APO/01/4, Rutherford Appleton Laboratory, Oxon, UK and ENSEEIHT-IRIT, Toulouse, France, 2001.
28.
M. H. Schneider and S. Zenios, A comparative study of algorithms for matrix balancing, Oper. Res., 38 (1990), pp. 439--455.
29.
R. Sinkhorn, A relationship between arbitrary positive matrices and doubly stochastic matrices, Ann. Math. Statist., 35 (1964), pp. 876--879.
30.
R. Sinkhorn, Diagonal equivalence to matrices with prescribed row and column sums, Amer. Math. Monthly, 74 (1967), pp. 402--405.
31.
R. Sinkhorn and P. Knopp, Concerning nonnegative matrices and doubly stochastic matrices, Pacific J. Math., 21 (1967), pp. 343--348.
32.
R. Sinkhorn and P. Knopp, Problems involving diagonal products in nonnegative matrices, Trans. Amer. Math. Soc., 136 (1969), pp. 67--75.
33.
G. W. Soules, The rate of convergence of Sinkhorn balancing, in Proceedings of the First Conference of the International Linear Algebra Society (Provo, UT, 1989), Linear Algebra Appl., 150 (1991), pp. 3--40.
34.
A. van der Sluis, Condition numbers and equilibration of matrices, Numer. Math., 14 (1969), pp. 14--23.
Information & Authors
Information
Published In
SIAM Journal on Matrix Analysis and Applications
Pages: 931 - 955
DOI: 10.1137/110825753
ISSN (online): 1095-7162
Copyright
© 2014, Society for Industrial and Applied Mathematics.
History
Submitted: 25 February 2011
Accepted: 16 April 2014
Published online: 17 July 2014
Keywords
MSC codes
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Bi-stochastically normalized graph Laplacian: convergence to manifold Laplacian and robustness to outlier noiseInformation and Inference: A Journal of the IMA, Vol. 13, No. 4 | 20 September 2024
- Avoiding Breakdown in Incomplete Factorizations in Low Precision ArithmeticACM Transactions on Mathematical Software, Vol. 50, No. 2 | 28 June 2024
- Automatic initial value generation procedure for nonlinear process models by interval arithmetic based cutting and splittingComputers & Chemical Engineering, Vol. 185 | 1 Jun 2024
- Incomplete double-cone factorizations of centrosymmetric matrices arising in spectral methodsNumerical Algorithms, Vol. 95, No. 3 | 3 August 2023
- Scalable block preconditioners for saturated thermo-hydro-mechanics problemsAdvanced Modeling and Simulation in Engineering Sciences, Vol. 10, No. 1 | 26 June 2023
- Robust Inference of Manifold Density and Geometry by Doubly Stochastic ScalingSIAM Journal on Mathematics of Data Science, Vol. 5, No. 3 | 20 July 2023
- Many-objective optimization of real and reactive power dispatch problemsInternational Journal of Electrical Power & Energy Systems, Vol. 146 | 1 Mar 2023
- Unbalanced Optimal Transport, from theory to numericsNumerical Control: Part B | 1 Jan 2023
- BTE-Barna: An extension of almaBTE for thermal simulation of devices based on 2D materialsComputer Physics Communications, Vol. 281 | 1 Dec 2022
- Doubly stochastic scaling unifies community detectionNeurocomputing, Vol. 504 | 1 Sep 2022
- Mixed precision algorithms in numerical linear algebraActa Numerica, Vol. 31 | 9 June 2022
- Scaling matrices and counting the perfect matchings in graphsDiscrete Applied Mathematics, Vol. 308 | 1 Feb 2022
- A survey of numerical linear algebra methods utilizing mixed-precision arithmeticThe International Journal of High Performance Computing Applications, Vol. 35, No. 4 | 19 March 2021
- On the automatic parallelization of subscripted subscript patterns using array property analysisProceedings of the ACM International Conference on Supercomputing | 4 June 2021
- OSQP: an operator splitting solver for quadratic programsMathematical Programming Computation, Vol. 12, No. 4 | 20 February 2020
- GPU acceleration of ADMM for large-scale quadratic programmingJournal of Parallel and Distributed Computing, Vol. 144 | 1 Oct 2020
- Max-Balanced Hungarian ScalingsSIAM Journal on Matrix Analysis and Applications, Vol. 40, No. 1 | 26 February 2019
- Squeezing a Matrix into Half Precision, with an Application to Solving Linear SystemsSIAM Journal on Scientific Computing, Vol. 41, No. 4 | 1 August 2019
- Depth Completion with Edge Optimization for Depth Maps from Lidar2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) | 1 Jul 2018
- Learning Intrinsic Image Decomposition from Watching the World2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition | 1 Jun 2018
- Thermodynamic equilibrium solutions through a modified Newton Raphson methodAIChE Journal, Vol. 63, No. 4 | 2 October 2016
- Preconditioning Techniques Based on the Birkhoff–von Neumann DecompositionComputational Methods in Applied Mathematics, Vol. 17, No. 2 | 14 December 2016
- Comparison of linear solvers for equilibrium geochemistry computationsComputational Geosciences, Vol. 21, No. 1 | 10 December 2016
- On the Complexity of the Block Low-Rank Multifrontal FactorizationSIAM Journal on Scientific Computing, Vol. 39, No. 4 | 30 August 2017
- Notes on Birkhoff–von Neumann decomposition of doubly stochastic matricesLinear Algebra and its Applications, Vol. 497 | 1 May 2016
- Two approximation algorithms for bipartite matching on multicore architecturesJournal of Parallel and Distributed Computing, Vol. 85 | 1 Nov 2015
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].