Abstract.

Error contaminated linear approximation problems appear in a large variety of applications. The presence of redundant or irrelevant data complicates their solution. It was shown that such data can be removed by the core reduction yielding a minimally dimensioned subproblem called the core problem. Direct (SVD or Tucker decomposion-based) reduction has been introduced previously for problems with matrix models and vector, or matrix, or tensor observations; and also for problems with bilinear models. For the cases of vector and matrix observations a Krylov subspace method, the generalized Golub–Kahan bidiagonalization, can be used to extract the core problem. In this paper, we first unify previously studied variants of linear approximation problems under the general framework of a multilinear approximation problem. We show how the direct core reduction can be extended to it. Then we show that the generalized Golub–Kahan bidiagonalization yields the core problem for any multilinear approximation problem. This further allows one to prove various properties of core problems, in particular, we give upper bounds on the multiplicity of singular values of reduced matrices.

Keywords

  1. (multi)linear approximation problems
  2. error-in-variables modeling
  3. total least squares
  4. core problem
  5. orthogonal transformations
  6. Krylov subspace methods

MSC codes

  1. 15A06
  2. 15A18
  3. 15A21
  4. 15A24
  5. 65F20
  6. 65F25

Get full access to this article

View all available purchase options and get full access to this article.

Acknowledgment.

The authors wish to thank the anonymous referees for their useful comments that helped to improve this paper.

References

1.
Å. Björck, A band-Lanczos algorithm for least squares and total least squares problems, in Book of Abstracts of 4th Total Least Squares and Errors-in-Variables Modeling Workshop in Leuven, Katholieke Universiteit Leuven, Leuven, Belgium, 2006, pp. 22–23.
2.
L. De Lathauwer, Signal Processing Based on Multilinear Algebra, Ph.D. thesis, Faculty of Engineering, KU Leuven, Leuven, Belgium, 1997.
3.
D. K. Faddeev, V. N. Kublanovskaya, and V. N. Faddeeva, Solution of linear algebraic systems with rectangular matrices, Proc. Steklov Inst. Math., 96 (1968), pp. 76–92.
4.
G. H. Golub, Some modified matrix eigenvalue problems, SIAM Rev., 15 (1973), pp. 318–334, https://doi.org/10.1137/1015032.
5.
G. Golub and W. Kahan, Calculating the singular values and pseudo-inverse of a matrix, J. Soc. Ind. Appl. Math. Ser. B Numer. Anal., 2 (1965), pp. 205–224, https://doi.org/10.1137/0702016.
6.
G. H. Golub and C. Reinsch, Singular value decomposition and least squares solutions, Numer. Math., 14 (1970), pp. 403–420, https://doi.org/10.1007/BF02163027.
7.
G. H. Golub and R. Underwood, The Block Lanczos Method for Computing Eigenvalues, Mathematical Software III, J. Rice, ed., Academic Press, New York, 1977, pp. 364–377.
8.
G. H. Golub and C. F. van Loan, An analysis of the total least squares problem, SIAM J. Num. Anal., 17 (1980), pp. 883–893, https://doi.org/10.1137/0717073.
9.
F. Han, Y. Wei, and P. Xie, Tensor Regularized Total Least Squares Methods with Applications to Image and Video Deblurring, preprint, 2022, https://doi.org/10.48550/arXiv.2211.00217.
10.
P. C. Hansen, Least Squares Data Fitting, Johns Hopkins University Press, Baltimore, MD, 2013.
11.
R. J. Hanson and C. L. Lawson, Extensions and applications of the Householder algorithm for solving linear least squares problems, Math. Comp., 23 (1969), pp. 787–812.
12.
I. Hnětynková and M. Plešinger, Complex wedge-shaped matrices: A generalization of Jacobi matrices, Linear Algebra Appl., 487 (2015), pp. 203–219, https://doi.org/10.1016/j.laa.2015.09.017.
13.
I. Hnětynková, M. Plešinger, and D. M. Sima, Solvability of the core problem with multiple right-hand sides in the TLS sense, SIAM J. Matrix Anal. Appl., 37 (2016), pp. 861–876, https://doi.org/10.1137/15M1028339.
14.
I. Hnětynková, M. Plešinger, D. M. Sima, Z. Strakoš, and S. Van Huffel, The total least squares problem in \(AX\approx B\): A new classification with the relationship to the classical works, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 748–770, https://doi.org/10.1137/100813348.
15.
I. Hnětynková, M. Plešinger, and Z. Strakoš, The core problem within linear approximation problem \(AX\approx B\) with multiple right-hand sides, SIAM J. Matrix Anal. Appl., 34 (2013), pp. 917–931, https://doi.org/10.1137/120884237.
16.
I. Hnětynková, M. Plešinger, and Z. Strakoš, Band generalization of the Golub–Kahan bidiagonalization, generalized Jacobi matrices, and the core problem, SIAM J. Matrix Anal. Appl., 36 (2015), pp. 417–434, https://doi.org/10.1137/140968914.
17.
I. Hnětynková, M. Plešinger, and J. Žáková, TLS formulation and core reduction for problems with structured right-hand sides, Linear Algebra Appl., 555 (2018), pp. 241–265, https://doi.org/10.1016/j.laa.2018.06.016.
18.
I. Hnětynková, M. Plešinger, and J. Žáková, On TLS formulation and core reduction for data fitting with generalized models, Linear Algebra Appl., 577 (2019), pp. 1–20, https://doi.org/10.1016/j.laa.2019.04.018.
19.
B. N. Khoromskij, Tenor Numerical Methods in Scientific Computing, Radon Series on Comput. Appl. Math. 19, Walter de Gruyter, Berlin, 2018.
20.
H. A. L. Kiers, Towards a standardized notation and terminology in multiway analysis, J. Chemometr., 14 (2000), pp. 105–122, https://doi.org/10.1002/1099-128X(200005/06)14:3%3C105::AID-CEM582%3E3.0.CO;2-I.
21.
T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev., 51 (2009), pp. 455–500, https://doi.org/10.1137/07070111X.
22.
A. Kukush, I. Markovsky, and S. Van Huffel, On consistent estimators in linear and bilinear multivariate errors-in-variables models, in Total Least Squares and Errors-in-Variables Modeling, S. Van Huffel and P. Lemmerling, eds., Kluwer Academic, Dordrecht, Netherlands, 2002, pp. 155–164, https://doi.org/10.1007/978-94-017-3552-0_14.
23.
A. Kukush, I. Markovsky, and S. Van Huffel, Consistent estimation in the bilinear multivariate errors-in-variables model, Metrika, 57 (2003), pp. 254–285, https://doi.org/10.1007/s001840200217.
24.
C.-K. Li, X.-G. Liu, and X.-F. Wang, Extension of the total least square problem using general unitarily invariant norms, Linear Multilinear Algebra, 55 (2007), pp. 71–79, https://doi.org/10.1080/03081080600593428.
25.
B. Ottersten, D. Asztély, M. Kristensson, and S. Parkvall, A statistical approach to subspace based estimation with applications in telecommunications, in Recent Advances in Total Least Squares Techniques and Errors-in-Variables Modeling, S. Van Huffel, ed., SIAM, Philadelphia, 1997, pp. 285–294.
26.
C. C. Paige and Z. Strakos, Core problem in linear algebraic systems, SIAM J. Matrix Anal. Appl., 27 (2005), pp. 861–875, https://doi.org/10.1137/040616991.
27.
C. C. Paige and Z. Strakoš, Scaled total least squares fundamentals, Numer. Math., 91 (2002), pp. 117–146, https://doi.org/10.1007/s002110100314.
28.
C. C. Paige and Z. Strakoš, Unifying least squares, total least squares and data least squares, in Total Least Squares and Errors-in-Variables Modeling, S. Van Huffel and P. Lemmerling, eds., Kluwer Academic, Dordrecht, Netherlands, 2002, pp. 25–34, https://doi.org/10.1007/978-94-017-3552-0_3.
29.
C. C. Paige and M. Wei, Analysis of the generalized total least squares problem \(AX \approx B\) when some columns of \(A\) are free of error, Numer. Math., 65 (1993), pp. 177–202, https://doi.org/10.1007/BF01385747.
30.
J. B. Rosen, H. Park, and J. Glick, Total least norm for linear and nonlinear structured problems, in Recent Advances in Total Least Squares Techniques and Errors-in-Variables Modeling, S. Van Huffel, ed., SIAM, Philadelphia, 1997, pp. 203–214.
31.
T. Söderström, U. Soverini, and K. Mahata, Perspectives on errors-in-variables estimation for dynamic systems, in Total Least Squares and Errors-in-Variables Modeling, S. Van Huffel and P. Lemmerling, eds., Kluwer Academic, Dordrecht, Netherlands, 2002, pp. 271–280, https://doi.org/10.1007/978-94-017-3552-0_24.
32.
G. W. Stewart, Updating a rank-revealing ULV decomposition, SIAM J. Matrix Anal. Appl., 14 (1993), pp. 494–499, https://doi.org/10.1137/0614034.
33.
L. R. Tucker, Implications of factor analysis of three-way matrices for measurement of change, in Problems in Measuring Change, C. W. Harries, ed., University of Wisconsin Press, Madison, WI, 1963, pp. 122–137.
34.
L. R. Tucker, The extension of factor analysis to three-dimensional matrices, in Contributions to Mathematical Psychology, H. Gulliksen and N. HoltFrederiksen, eds., Holt, Rinehardt & Winston, New York, 1964, pp. 110–127.
35.
L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31 (1966), pp. 279–311, https://doi.org/10.1007/BF02289464.
36.
A. van der Sluis, Stability of the solutions of linear least squares problems, Numer. Math., 23 (1974), pp. 241–254, https://doi.org/10.1007/BF01400307.
37.
S. Van Huffel, Documented Fortran 77 Programs of the Extended Classical Total Least Squares Algorithm, the Partial Singular Value Decomposition Algorithm and the Partial Total Least Squares Algorithm, ESAT-KUL 88/1, Katholieke Universiteit Leuven, Leuven, Belgium, 1988.
38.
S. Van Huffel, The extended classical total least squares algorithm, J. Comput. Appl. Math., 25 (1989), pp. 111–119, https://doi.org/10.1016/0377-0427(89)90080-0.
39.
S. Van Huffel, ed., Recent Advances in Total Least Squares Techniques and Error-in-Variables Modeling, SIAM, Philadelphia, 1997.
40.
S. Van Huffel and P. Lemmerling, eds., Total Least Squares and Error-in-Variables Modeling. Analysis, Algorithms and Applications, Kluwer Academic, Dordrecht, Netherlands, 2002.
41.
S. Van Huffel and J. Vandewalle, The Total Least Squares Problem: Computational Aspects and Analysis, SIAM, Philadelphia, 1991.
42.
H. Wang and H. Xiang, Quantum algorithm for total least squares data fitting, Phys. Lett. A, 383 (2019), pp. 2235–2240, https://doi.org/10.1016/j.physleta.2019.04.037.
43.
X.-F. Wang, Total least squares problem with the arbitrary unitarily invariant norms, Linear Multilinear Algebra, 65 (2017), pp. 438–456, https://doi.org/10.1080/03081087.2016.1189493.
44.
M. Wei, The analysis for the total least squares problem with more than one solution, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 746–763, https://doi.org/10.1137/0613047.
45.
M. Wei, Algebraic relations between the total least squares and least squares problems with more than one solution, Numer. Math., 62 (1992), pp. 123–148, https://doi.org/10.1007/BF01396223.
46.
P. Xie, H. Xiang, and Y. Wei, Randomized algorithms for total least squares problems, Numer. Linear Algebra Appl., 26 (2019), e2219, https://doi.org/10.1002/nla.2219.
47.
S. Yan and K. Huang, The original TLS solution sets of the multidimensional TLS problem, Int. J. Comput. Math., 73 (2000), pp. 349–359, https://doi.org/10.1080/00207160008804902.
48.
L. Zhang and Y. Wei, Randomized core reduction for discrete ill-posed problem, J. Comput. Appl. Math., 375 (2020), 112797, https://doi.org/10.1016/j.cam.2020.112797.
49.
Q. Zuo, Y. Wei, and H. Xiang, Quantum-Inspired Algorithm for Truncated Total Least Squares Solution, preprint, 2022, https://doi.org/10.48550/arXiv.2205.00455.

Information & Authors

Information

Published In

cover image SIAM Journal on Matrix Analysis and Applications
SIAM Journal on Matrix Analysis and Applications
Pages: 53 - 79
ISSN (online): 1095-7162

History

Submitted: 29 November 2021
Accepted: 13 September 2022
Published online: 2 February 2023

Keywords

  1. (multi)linear approximation problems
  2. error-in-variables modeling
  3. total least squares
  4. core problem
  5. orthogonal transformations
  6. Krylov subspace methods

MSC codes

  1. 15A06
  2. 15A18
  3. 15A21
  4. 15A24
  5. 65F20
  6. 65F25

Authors

Affiliations

Iveta Hnětynková
Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic.
Martin Plešinger
Department of Mathematics, Technical University of Liberec, Czech Republic.
Jana Žáková Contact the author
Department of Mathematics, Technical University of Liberec, Czech Republic.

Funding Information

Funding: The work of the second and third authors was partially supported by the Technical University of Liberec (TUL) projects SGS-2021-4039, SGS-2022-4025.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

There are no citations for this item

View Options

View options

PDF

View PDF

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.