Frames and Numerical Approximation
Abstract
Functions of one or more variables are usually approximated with a basis: a complete, linearly independent system of functions that spans a suitable function space. The topic of this paper is the numerical approximation of functions using the more general notion of frames: that is, complete systems that are generally redundant but provide infinite representations with bounded coefficients. While frames are well known in image and signal processing, coding theory, and other areas of applied mathematics, their use in numerical analysis is far less widespread. Yet, as we show via a series of examples, frames are more flexible than bases and can be constructed easily in a range of problems where finding orthonormal bases with desirable properties (rapid convergence, high-resolution power, etc.) is difficult or impossible. For instance, we exhibit a frame which yields simple, high-order approximations of smooth, multivariate functions in arbitrary geometries.
A key concern when using frames is that computing a best approximation requires solving an ill-conditioned linear system. Nonetheless, we construct a frame approximation via regularization with bounded condition number (with respect to perturbations in the data), which approximates any function up to an error of order $\sqrt{\epsilon}$, or even of order $\epsilon$ with suitable modifications. Here, $\epsilon$ is a threshold value that can be chosen by the user. Crucially, rate of decay of the error down to this level is determined by the existence of approximate representations of $f$ in the frame possessing small-norm coefficients. We demonstrate the existence of such representations in all of our examples. Overall, our analysis suggests that frames are a natural generalization of bases in which to develop numerical approximations. In particular, even in the presence of severely ill-conditioned linear systems, the frame condition imposes sufficient mathematical structure in order to give rise to accurate, well-conditioned approximations.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
Supplementary Material
PLEASE NOTE: These supplementary files have not been peer-reviewed.
Index of Supplementary Materials
Title of paper: Frames and Numerical approximation
Authors: Daan Huybrechs and Ben Adcock
File: M111469SupMat.pdf
Type: PDF
Contents: Additional material for the paper
References
1.
B. Adcock, Modified Fourier Expansions: Theory, Construction and Applications, Ph.D. thesis, University of Cambridge, 2010.
2.
B. Adcock, Convergence acceleration of modified Fourier series in one or more dimensions, Math. Comp., 80 (2011), pp. 225--261.
3.
B. Adcock and A. C. Hansen, A generalized sampling theorem for stable reconstructions in arbitrary bases, J. Fourier Anal. Appl., 18 (2012), pp. 685--716.
4.
B. Adcock and A. C. Hansen, Generalized sampling and infinite-dimensional compressed sensing, Found. Comput. Math., 16 (2016), pp. 1263--1323.
5.
B. Adcock, A. C. Hansen, and C. Poon, Beyond consistent reconstructions: Optimality and sharp bounds for generalized sampling, and application to the uniform resampling problem, SIAM J. Math. Anal., 45 (2013), pp. 3132--3167, https://doi.org/10.1137/120895846.
6.
B. Adcock and D. Huybrechs, On the resolution power of Fourier extensions for oscillatory functions, J. Comput. Appl. Math., 260 (2014), pp. 312--336.
7.
B. Adcock and D. Huybrechs, Frames and Numerical Approximation II: Generalized Sampling, preprint, https://arxiv.org/abs/1802.01950, 2018.
8.
B. Adcock, D. Huybrechs, and J. Martín-Vaquero, On the numerical stability of Fourier extensions, Found. Comput. Math., 14 (2014), pp. 635--687.
9.
B. Adcock and J. Ruan, Parameter selection and numerical approximation properties of Fourier extensions from fixed data, J. Comput. Phys., 273 (2014), pp. 453--471.
10.
N. Albin and O. P. Bruno, A spectral FC solver for the compressible Navier--Stokes equations in general domains I: Explicit time-stepping, J. Comput. Phys., 230 (2011), pp. 6248--6270.
11.
P. Balazs and K. Gröchenig, A guide to localized frames and applications to Galerkin-like representations of operators, in Novel Methods in Harmonic Analysis with Applications to Numerical Analysis and Data Processing, I. Pesenson, Q. Le Gia, A. Mayeli, H. Mhaskar, and D.-X. Zhou, eds., Applied and Numerical Harmonic Analysis, Birkhäuser, Cham, 2017, pp. 47--79.
12.
J. J. Benedetto, Irregular sampling and frames, in Wavelets: A Tutorial in Theory and Applications, C. K. Chui, ed., CRC Press, Boca Raton, FL, 1994, pp. 445--508.
13.
D. Boffi, N. Cavallini, and L. Gastaldi, The finite element immersed boundary method with distributed Lagrange multiplier, SIAM J. Numer. Anal., 53 (2015), pp. 2584--2604, https://doi.org/10.1137/140978399.
14.
A. Böttcher, Infinite matrices and projection methods, in Lectures on Operator Theory and Its Applications (Waterloo, ON, 1994), Fields Inst. Monogr. 3, AMS, Providence, RI, 1996, pp. 1--72.
15.
J. Boyd, Fourier embedded domain methods: Extending a function defined on an irregular region to a rectangle so that the extension is spatially periodic and ${C}^{\infty}$, Appl. Math. Comput., 161 (2005), pp. 591--597.
16.
J. P. Boyd, A comparison of numerical algorithms for Fourier extension of the first, second, and third kinds, J. Comput. Phys., 178 (2002), pp. 118--160.
17.
O. Bruno and M. Lyon, High-order unconditionally stable FC-AD solvers for general smooth domains I. Basic elements, J. Comput. Phys., 229 (2010), pp. 2009--2033.
18.
O. P. Bruno, Y. Han, and M. M. Pohlman, Accurate, high-order representation of complex three-dimensional surfaces via Fourier continuation analysis, J. Comput. Phys., 227 (2007), pp. 1094--1125.
19.
E. J. Candès and D. Donoho, Ridgelets: A key to higher-dimensional intermittency?, R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci., 357 (1999), pp. 2495--2509.
20.
E. J. Candès and D. Donoho, New tight frames of curvelets and optimal representations of objects with piecewise ${C}^2$ singularities, Comm. Pure Appl. Math., 57 (2004), pp. 219--266.
21.
E. J. Candès, Y. C. Eldar, D. Needell, and P. Randall, Compressed sensing with coherent and redundant dictionaries, Appl. Comput. Harmon. Anal., 31 (2010), pp. 59--73.
22.
P. G. Casazza, The art of frame theory, Taiwanese J. Math., 4 (2000), pp. 129--202.
23.
P. G. Casazza and O. Christensen, Approximation of the frame coefficients using finite-dimensional methods, J. Electronic Imaging, 6 (1997), pp. 479--483.
24.
P. G. Casazza and O. Christensen, Riesz frames and approximation of the frame coefficients, Approx. Theory Appl., 14 (1998), pp. 1--11.
25.
P. G. Casazza and O. Christensen, Approximation of the inverse frame operator and applications to Gabor frames, J. Approx. Theory, 103 (2000), pp. 338--356.
26.
P. G. Casazza and G. Kutyniok, eds., Finite Frames: Theory and Applications, Birkhäuser, 2013.
27.
Z. Chen and C.-W. Shu, Recovering exponential accuracy from collocation point values of smooth functions with end-point singularities, J. Comput. Appl. Math., 265 (2014), pp. 83--95.
28.
Z. Chen and C.-W. Shu, Recovering exponential accuracy in Fourier spectral methods involving piecewise smooth functions with unbounded derivative singularities, J. Sci. Comput., 65 (2015), pp. 1145--1165.
29.
O. Christensen, Frames and the projection method, Appl. Comput. Harmon. Anal., 1 (1993), pp. 50--53.
30.
O. Christensen, Frames containing a Riesz basis and approximation of the frame coefficients using finite dimensional methods, J. Math. Anal. Appl., 199 (1996), pp. 256--270.
31.
O. Christensen, Finite-dimensional approximation of the inverse frame operator and applications to Weyl--Heisenberg frames and wavelet frames, J. Fourier Anal. Appl., 6 (2001), pp. 79--91.
32.
O. Christensen, An Introduction to Frames and Riesz Bases, 2nd ed., Applied and Numerical Harmonic Analysis, Birkhäuser, 2016.
33.
O. Christensen and A. Lindner, Frames containing a Riesz basis and approximation of the inverse frame operator, Internat. Ser. Numer. Math., 137 (2001), pp. 89--100.
34.
O. Christensen and T. Strohmer, The finite section method and problems in frame theory, J. Approx. Theory, 133 (2005), pp. 221--237.
35.
S. Dahlke, M. Fornasier, and T. Raasch, Adaptive frame methods for elliptic operator equations, Adv. Comput. Math., 27 (2007), pp. 27--63.
36.
S. Datta, S. Howard, and D. Cochran, Geometry of the Welch bounds, Linear Algebra Appl., 437 (2012), pp. 2455--2470.
37.
I. Daubechies, Ten Lectures on Wavelets, SIAM, Philadelphia, 1992, https://doi.org/10.1137/1.9781611970104.
38.
I. Daubechies, A. Grossmann, and Y. Meyer, Painless nonorthogonal expansions, J. Math. Phys., 27 (1986), pp. 1271--1283.
39.
E. B. Davies and M. Plum, Spectral pollution, IMA J. Numer. Anal., 23 (2004), pp. 417--438.
40.
M. Dörfler and G. A. Velasco, Adaptive Gabor frames by projection onto time-frequency subspaces, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp. 3097--3101.
41.
R. J. Duffin and A. C. Schaeffer, A class of nonharmonic Fourier series, Trans. Amer. Math. Soc., 72 (1952), pp. 341--366.
42.
K. S. Eckhoff, Accurate and efficient reconstruction of discontinuous functions from truncated series expansions, Math. Comp., 61 (1993), pp. 745--763.
43.
H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic, Dordrecht, The Netherlands, 1996.
44.
M. Fickus, D. G. Mixon, and J. C. Tremain, Steiner equiangular tight frames, Linear Algebra Appl., 436 (2012), pp. 1014--1027.
45.
M. Fornasier and K. Gröchenig, Intrinsic localization of frames, Constr. Approx., 22 (2005), pp. 395--415.
46.
K. Gröchenig, Localization of frames, Banach frames, and the invertibility of the frame operator, J. Fourier Anal. Appl., 10 (2004), pp. 105--132.
47.
K. Gröchenig, Z. Rzeszotnik, and T. Strohmer, Quantitative estimates for the finite section method and Banach algebras of matrices, Integral Equations Oper. Theory, 67 (2011), pp. 183--202.
48.
A. C. Hansen, On the approximation of spectra of linear operators on Hilbert spaces, J. Funct. Anal., 254 (2008), pp. 2092--2126.
49.
A. C. Hansen, On the solvability complexity index, the n-pseudospectrum and approximations of spectra of operators, J. Amer. Math. Soc., 24 (2011), pp. 81--124.
50.
P. C. Hansen, V. Pereyra, and G. Scherer, Least Squares Data Fitting with Applications, Johns Hopkins University Press, Baltimore, MD, 2012.
51.
M. L. Harrison, Frames and Irregular Sampling from a Computational Perspective, Ph.D. thesis, University of Maryland, College Park, MD, 1998.
52.
D. P. Hewett, S. N. Chandler-Wilde, S. Langdon, and A. Twigger, A high frequency boundary element method for scattering by a class of nonconvex obstacles, Numer. Math., 129 (2015), pp. 647--689.
53.
J. A. Hogan and J. D. Lakey, Duration and Bandwidth Limiting, Applied and Numerical Harmonic Analysis, Birkhäuser, 2012.
54.
D. Huybrechs, On the Fourier extension of nonperiodic functions, SIAM J. Numer. Anal., 47 (2010), pp. 4326--4355, https://doi.org/10.1137/090752456.
55.
M. Javed and L. N. Trefethen, Euler-Maclaurin and Gregory interpolants, Numer Math, 132 (2016), pp. 201--216.
56.
F. Kasolis, E. Wadbro, and M. Berggren, Analysis of fictitious domain approximations of hard scatterers, SIAM J. Numer. Anal., 53 (2015), pp. 2347--2362, https://doi.org/10.1137/140981630.
57.
J. Kovacevic and A. Chebira, Life beyond bases: The advent of frames (Part II), IEEE Signal Process. Mag., 24 (2007), pp. 115--125.
58.
J. Kovacevic and A. Chebira, Life beyond bases: The advent of frames (Part I), IEEE Signal Process. Mag., 24 (2007), pp. 86--104.
59.
A. Krylov, On Approximate Calculations, Lectures delivered in 1906, St. Petersburg, 1907 (in Russian).
60.
G. Kutyniok and D. Labate, eds., Shearlets: Multiscale Analysis for Multivariate Data, Springer, 2012.
61.
H. J. Landau and H. O. Pollak, Prolate spheroidal wave functions, Fourier analysis and uncertainty---III: The dimension of the space of essentially time-and band-limited signals, Bell System Tech J., 41 (1962), pp. 1295--1336.
62.
M. Lewin and É. Séré, Spectral pollution and how to avoid it, Proc. London Math. Soc., 100 (2009), pp. 864--900.
63.
M. Lindner, Infinite Matrices and Their Finite Sections, Frontiers in Mathematics, Birkhäuser Verlag, Basel, 2006.
64.
S. H. Lui, Spectral domain embedding for elliptic PDEs in complex domains, J. Comput. Appl. Math., 225 (2009), pp. 541--557.
65.
M. Lyon, Approximation error in regularized SVD-based Fourier continuations, Appl. Numer. Math., 62 (2012), pp. 1790--1803.
66.
M. Lyon, A fast algorithm for Fourier continuation, SIAM J. Sci. Comput., 33 (2011), pp. 3241--3260, https://doi.org/10.1137/11082436X.
67.
M. Lyon and O. Bruno, High-order unconditionally stable FC-AD solvers for general smooth domains II. Elliptic, parabolic and hyperbolic PDEs; theoretical considerations, J. Comput. Phys., 229 (2010), pp. 3358--3381.
68.
S. G. Mallat, A Wavelet Tour of Signal Processing: The Sparse Way, 3rd ed., Academic Press, 2009.
69.
R. Matthysen and D. Huybrechs, Fast algorithms for the computation of Fourier extensions of arbitrary length, SIAM J. Sci. Comput., 38 (2016), pp. A899--A922, https://doi.org/10.1137/15M1030923.
70.
R. Matthysen and D. Huybrechs, Function approximation on arbitrary domains using Fourier extension frames, SIAM J. Numer. Anal., 56 (2018), pp. 1360--1385, http://doi.org/10.1137/17M1134809.
71.
A. Neumaier, Solving ill-conditioned and singular linear systems: A tutorial on regularization, SIAM Rev., 40 (1998), pp. 636--666, https://doi.org/10.1137/S0036144597321909.
72.
R. Pasquetti and M. Elghaoui, A spectral embedding method applied to the advection--diffusion equation, J. Comput. Phys., 125 (1996), pp. 464--476.
73.
R. Platte, A. J. Gutierrez, and A. Gelb, Fourier reconstruction of univariate piecewise-smooth functions from non-uniform spectral data with exponential convergence rates, Appl. Comput. Harmon. Anal., 39 (2015), pp. 427--449.
74.
J. M. Renes, Equiangular tight frames from Paley tournaments, Linear Algebra Appl., 426 (2007), pp. 497--501.
75.
D. Shirokoff and J.-C. Nave, A sharp-interface active penalty method for the incompressible Navier--Stokes equations, J. Sci. Comput., 62 (2015), pp. 53--77.
76.
D. Slepian, Prolate spheroidal wave functions. Fourier analysis, and uncertainty V: The discrete case, Bell System Tech J., 57 (1978), pp. 1371--1430.
77.
G. Song, J. Davis, and A. Gelb, A high-dimensional inverse frame operator approximation technique, SIAM J. Numer. Anal., 54 (2016), pp. 2282--2301, https://doi.org/10.1137/15M1047593.
78.
G. Song and A. Gelb, Approximating the inverse frame operator from localized frames, Appl. Comput. Harmon. Anal., 35 (2013), pp. 94--110.
79.
D. B. Stein, R. D. Guy, and B. Thomases, Immersed boundary smooth extension: A high-order method for solving PDE on arbitrary smooth domains using Fourier spectral methods, J. Comput. Phys., 304 (2016), pp. 252--274.
80.
R. Stevenson, Adaptive solution of operator equations using wavelet frames, SIAM J. Numer. Anal., 41 (2003), pp. 1074--1100, https://doi.org/10.1137/S0036142902407988.
81.
T. Strohmer, Numerical analysis of the nonuniform sampling problem, J. Comput. Appl. Math., 122 (2000), pp. 297--316.
82.
T. Strohmer, A note on equiangular tight frames, Linear Algebra Appl., 429 (2008), pp. 326--330.
83.
M. A. Sustik, J. A. Tropp, I. S. Dhillon, and R. W. Heath, On the existence of equiangular tight frames, Linear Algebra Appl., 426 (2007), pp. 619--635.
84.
A. Teolis and J. J. Benedetto, Local frames and noise reduction, Signal Process., 45 (1995), pp. 369--387.
85.
L. N. Trefethen, Approximation Theory and Approximation Practice, SIAM, Philadelphia, 2013.
Information & Authors
Information
Published In
Copyright
© 2019, Society for Industrial and Applied Mathematics.
History
Submitted: 2 February 2017
Accepted: 29 October 2018
Published online: 7 August 2019
Keywords
MSC codes
Authors
Funding Information
Natural Sciences and Engineering Research Council of Canada https://doi.org/10.13039/501100000038 : 611675
Funding Information
Alfred P. Sloan Foundation https://doi.org/10.13039/100000879
Funding Information
Fonds Wetenschappelijk Onderzoek https://doi.org/10.13039/501100003130 : G.0641.11, G.A004.14
Funding Information
KU Leuven https://doi.org/10.13039/501100004040 : C14/15/055
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Resolution of Singularities by Rational FunctionsSIAM Journal on Numerical Analysis, Vol. 61, No. 6 | 15 November 2023
- On the Computation of Gaussian Quadrature Rules for Chebyshev Sets of Linearly Independent FunctionsSIAM Journal on Numerical Analysis, Vol. 60, No. 3 | 26 May 2022
- Vandermonde with ArnoldiSIAM Review, Vol. 63, No. 2 | 6 May 2021
- Near-Optimal Sampling Strategies for Multivariate Function Approximation on General DomainsSIAM Journal on Mathematics of Data Science, Vol. 2, No. 3 | 16 July 2020
- The AZ Algorithm for Least Squares Systems with a Known Incomplete Generalized InverseSIAM Journal on Matrix Analysis and Applications, Vol. 41, No. 3 | 27 August 2020