# Frames and Numerical Approximation

## Abstract

Functions of one or more variables are usually approximated with a basis: a complete, linearly independent system of functions that spans a suitable function space. The topic of this paper is the numerical approximation of functions using the more general notion of frames: that is, complete systems that are generally redundant but provide infinite representations with bounded coefficients. While frames are well known in image and signal processing, coding theory, and other areas of applied mathematics, their use in numerical analysis is far less widespread. Yet, as we show via a series of examples, frames are more flexible than bases and can be constructed easily in a range of problems where finding orthonormal bases with desirable properties (rapid convergence, high-resolution power, etc.) is difficult or impossible. For instance, we exhibit a frame which yields simple, high-order approximations of smooth, multivariate functions in arbitrary geometries.
A key concern when using frames is that computing a best approximation requires solving an ill-conditioned linear system. Nonetheless, we construct a frame approximation via regularization with bounded condition number (with respect to perturbations in the data), which approximates any function up to an error of order $\sqrt{\epsilon}$, or even of order $\epsilon$ with suitable modifications. Here, $\epsilon$ is a threshold value that can be chosen by the user. Crucially, rate of decay of the error down to this level is determined by the existence of approximate representations of $f$ in the frame possessing small-norm coefficients. We demonstrate the existence of such representations in all of our examples. Overall, our analysis suggests that frames are a natural generalization of bases in which to develop numerical approximations. In particular, even in the presence of severely ill-conditioned linear systems, the frame condition imposes sufficient mathematical structure in order to give rise to accurate, well-conditioned approximations.

## Supplementary Material

PLEASE NOTE: These supplementary files have not been peer-reviewed.

Index of Supplementary Materials

Title of paper: Frames and Numerical approximation

Authors: Daan Huybrechs and Ben Adcock

File: M111469SupMat.pdf

Type: PDF

Contents: Additional material for the paper

## References

1.
B. Adcock, Modified Fourier Expansions: Theory, Construction and Applications, Ph.D. thesis, University of Cambridge, 2010.
2.
B. Adcock, Convergence acceleration of modified Fourier series in one or more dimensions, Math. Comp., 80 (2011), pp. 225--261.
3.
B. Adcock and A. C. Hansen, A generalized sampling theorem for stable reconstructions in arbitrary bases, J. Fourier Anal. Appl., 18 (2012), pp. 685--716.
4.
B. Adcock and A. C. Hansen, Generalized sampling and infinite-dimensional compressed sensing, Found. Comput. Math., 16 (2016), pp. 1263--1323.
5.
B. Adcock, A. C. Hansen, and C. Poon, Beyond consistent reconstructions: Optimality and sharp bounds for generalized sampling, and application to the uniform resampling problem, SIAM J. Math. Anal., 45 (2013), pp. 3132--3167, https://doi.org/10.1137/120895846.
6.
B. Adcock and D. Huybrechs, On the resolution power of Fourier extensions for oscillatory functions, J. Comput. Appl. Math., 260 (2014), pp. 312--336.
7.
B. Adcock and D. Huybrechs, Frames and Numerical Approximation II: Generalized Sampling, preprint, https://arxiv.org/abs/1802.01950, 2018.
8.
B. Adcock, D. Huybrechs, and J. Martín-Vaquero, On the numerical stability of Fourier extensions, Found. Comput. Math., 14 (2014), pp. 635--687.
9.
B. Adcock and J. Ruan, Parameter selection and numerical approximation properties of Fourier extensions from fixed data, J. Comput. Phys., 273 (2014), pp. 453--471.
10.
N. Albin and O. P. Bruno, A spectral FC solver for the compressible Navier--Stokes equations in general domains I: Explicit time-stepping, J. Comput. Phys., 230 (2011), pp. 6248--6270.
11.
P. Balazs and K. Gröchenig, A guide to localized frames and applications to Galerkin-like representations of operators, in Novel Methods in Harmonic Analysis with Applications to Numerical Analysis and Data Processing, I. Pesenson, Q. Le Gia, A. Mayeli, H. Mhaskar, and D.-X. Zhou, eds., Applied and Numerical Harmonic Analysis, Birkhäuser, Cham, 2017, pp. 47--79.
12.
J. J. Benedetto, Irregular sampling and frames, in Wavelets: A Tutorial in Theory and Applications, C. K. Chui, ed., CRC Press, Boca Raton, FL, 1994, pp. 445--508.
13.
D. Boffi, N. Cavallini, and L. Gastaldi, The finite element immersed boundary method with distributed Lagrange multiplier, SIAM J. Numer. Anal., 53 (2015), pp. 2584--2604, https://doi.org/10.1137/140978399.
14.
A. Böttcher, Infinite matrices and projection methods, in Lectures on Operator Theory and Its Applications (Waterloo, ON, 1994), Fields Inst. Monogr. 3, AMS, Providence, RI, 1996, pp. 1--72.
15.
J. Boyd, Fourier embedded domain methods: Extending a function defined on an irregular region to a rectangle so that the extension is spatially periodic and ${C}^{\infty}$, Appl. Math. Comput., 161 (2005), pp. 591--597.
16.
J. P. Boyd, A comparison of numerical algorithms for Fourier extension of the first, second, and third kinds, J. Comput. Phys., 178 (2002), pp. 118--160.
17.
O. Bruno and M. Lyon, High-order unconditionally stable FC-AD solvers for general smooth domains I. Basic elements, J. Comput. Phys., 229 (2010), pp. 2009--2033.
18.
O. P. Bruno, Y. Han, and M. M. Pohlman, Accurate, high-order representation of complex three-dimensional surfaces via Fourier continuation analysis, J. Comput. Phys., 227 (2007), pp. 1094--1125.
19.
E. J. Candès and D. Donoho, Ridgelets: A key to higher-dimensional intermittency?, R. Soc. Lond. Philos. Trans. Ser. A Math. Phys. Eng. Sci., 357 (1999), pp. 2495--2509.
20.
E. J. Candès and D. Donoho, New tight frames of curvelets and optimal representations of objects with piecewise ${C}^2$ singularities, Comm. Pure Appl. Math., 57 (2004), pp. 219--266.
21.
E. J. Candès, Y. C. Eldar, D. Needell, and P. Randall, Compressed sensing with coherent and redundant dictionaries, Appl. Comput. Harmon. Anal., 31 (2010), pp. 59--73.
22.
P. G. Casazza, The art of frame theory, Taiwanese J. Math., 4 (2000), pp. 129--202.
23.
P. G. Casazza and O. Christensen, Approximation of the frame coefficients using finite-dimensional methods, J. Electronic Imaging, 6 (1997), pp. 479--483.
24.
P. G. Casazza and O. Christensen, Riesz frames and approximation of the frame coefficients, Approx. Theory Appl., 14 (1998), pp. 1--11.
25.
P. G. Casazza and O. Christensen, Approximation of the inverse frame operator and applications to Gabor frames, J. Approx. Theory, 103 (2000), pp. 338--356.
26.
P. G. Casazza and G. Kutyniok, eds., Finite Frames: Theory and Applications, Birkhäuser, 2013.
27.
Z. Chen and C.-W. Shu, Recovering exponential accuracy from collocation point values of smooth functions with end-point singularities, J. Comput. Appl. Math., 265 (2014), pp. 83--95.
28.
Z. Chen and C.-W. Shu, Recovering exponential accuracy in Fourier spectral methods involving piecewise smooth functions with unbounded derivative singularities, J. Sci. Comput., 65 (2015), pp. 1145--1165.
29.
O. Christensen, Frames and the projection method, Appl. Comput. Harmon. Anal., 1 (1993), pp. 50--53.
30.
O. Christensen, Frames containing a Riesz basis and approximation of the frame coefficients using finite dimensional methods, J. Math. Anal. Appl., 199 (1996), pp. 256--270.
31.
O. Christensen, Finite-dimensional approximation of the inverse frame operator and applications to Weyl--Heisenberg frames and wavelet frames, J. Fourier Anal. Appl., 6 (2001), pp. 79--91.
32.
O. Christensen, An Introduction to Frames and Riesz Bases, 2nd ed., Applied and Numerical Harmonic Analysis, Birkhäuser, 2016.
33.
O. Christensen and A. Lindner, Frames containing a Riesz basis and approximation of the inverse frame operator, Internat. Ser. Numer. Math., 137 (2001), pp. 89--100.
34.
O. Christensen and T. Strohmer, The finite section method and problems in frame theory, J. Approx. Theory, 133 (2005), pp. 221--237.
35.
S. Dahlke, M. Fornasier, and T. Raasch, Adaptive frame methods for elliptic operator equations, Adv. Comput. Math., 27 (2007), pp. 27--63.
36.
S. Datta, S. Howard, and D. Cochran, Geometry of the Welch bounds, Linear Algebra Appl., 437 (2012), pp. 2455--2470.
37.
I. Daubechies, Ten Lectures on Wavelets, SIAM, Philadelphia, 1992, https://doi.org/10.1137/1.9781611970104.
38.
I. Daubechies, A. Grossmann, and Y. Meyer, Painless nonorthogonal expansions, J. Math. Phys., 27 (1986), pp. 1271--1283.
39.
E. B. Davies and M. Plum, Spectral pollution, IMA J. Numer. Anal., 23 (2004), pp. 417--438.
40.
M. Dörfler and G. A. Velasco, Adaptive Gabor frames by projection onto time-frequency subspaces, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp. 3097--3101.
41.
R. J. Duffin and A. C. Schaeffer, A class of nonharmonic Fourier series, Trans. Amer. Math. Soc., 72 (1952), pp. 341--366.
42.
K. S. Eckhoff, Accurate and efficient reconstruction of discontinuous functions from truncated series expansions, Math. Comp., 61 (1993), pp. 745--763.
43.
H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic, Dordrecht, The Netherlands, 1996.
44.
M. Fickus, D. G. Mixon, and J. C. Tremain, Steiner equiangular tight frames, Linear Algebra Appl., 436 (2012), pp. 1014--1027.
45.
M. Fornasier and K. Gröchenig, Intrinsic localization of frames, Constr. Approx., 22 (2005), pp. 395--415.
46.
K. Gröchenig, Localization of frames, Banach frames, and the invertibility of the frame operator, J. Fourier Anal. Appl., 10 (2004), pp. 105--132.
47.
K. Gröchenig, Z. Rzeszotnik, and T. Strohmer, Quantitative estimates for the finite section method and Banach algebras of matrices, Integral Equations Oper. Theory, 67 (2011), pp. 183--202.
48.
A. C. Hansen, On the approximation of spectra of linear operators on Hilbert spaces, J. Funct. Anal., 254 (2008), pp. 2092--2126.
49.
A. C. Hansen, On the solvability complexity index, the n-pseudospectrum and approximations of spectra of operators, J. Amer. Math. Soc., 24 (2011), pp. 81--124.
50.
P. C. Hansen, V. Pereyra, and G. Scherer, Least Squares Data Fitting with Applications, Johns Hopkins University Press, Baltimore, MD, 2012.
51.
M. L. Harrison, Frames and Irregular Sampling from a Computational Perspective, Ph.D. thesis, University of Maryland, College Park, MD, 1998.
52.
D. P. Hewett, S. N. Chandler-Wilde, S. Langdon, and A. Twigger, A high frequency boundary element method for scattering by a class of nonconvex obstacles, Numer. Math., 129 (2015), pp. 647--689.
53.
J. A. Hogan and J. D. Lakey, Duration and Bandwidth Limiting, Applied and Numerical Harmonic Analysis, Birkhäuser, 2012.
54.
D. Huybrechs, On the Fourier extension of nonperiodic functions, SIAM J. Numer. Anal., 47 (2010), pp. 4326--4355, https://doi.org/10.1137/090752456.
55.
M. Javed and L. N. Trefethen, Euler-Maclaurin and Gregory interpolants, Numer Math, 132 (2016), pp. 201--216.
56.
F. Kasolis, E. Wadbro, and M. Berggren, Analysis of fictitious domain approximations of hard scatterers, SIAM J. Numer. Anal., 53 (2015), pp. 2347--2362, https://doi.org/10.1137/140981630.
57.
J. Kovacevic and A. Chebira, Life beyond bases: The advent of frames (Part II), IEEE Signal Process. Mag., 24 (2007), pp. 115--125.
58.
J. Kovacevic and A. Chebira, Life beyond bases: The advent of frames (Part I), IEEE Signal Process. Mag., 24 (2007), pp. 86--104.
59.
A. Krylov, On Approximate Calculations, Lectures delivered in 1906, St. Petersburg, 1907 (in Russian).
60.
G. Kutyniok and D. Labate, eds., Shearlets: Multiscale Analysis for Multivariate Data, Springer, 2012.
61.
H. J. Landau and H. O. Pollak, Prolate spheroidal wave functions, Fourier analysis and uncertainty---III: The dimension of the space of essentially time-and band-limited signals, Bell System Tech J., 41 (1962), pp. 1295--1336.
62.
M. Lewin and É. Séré, Spectral pollution and how to avoid it, Proc. London Math. Soc., 100 (2009), pp. 864--900.
63.
M. Lindner, Infinite Matrices and Their Finite Sections, Frontiers in Mathematics, Birkhäuser Verlag, Basel, 2006.
64.
S. H. Lui, Spectral domain embedding for elliptic PDEs in complex domains, J. Comput. Appl. Math., 225 (2009), pp. 541--557.
65.
M. Lyon, Approximation error in regularized SVD-based Fourier continuations, Appl. Numer. Math., 62 (2012), pp. 1790--1803.
66.
M. Lyon, A fast algorithm for Fourier continuation, SIAM J. Sci. Comput., 33 (2011), pp. 3241--3260, https://doi.org/10.1137/11082436X.
67.
M. Lyon and O. Bruno, High-order unconditionally stable FC-AD solvers for general smooth domains II. Elliptic, parabolic and hyperbolic PDEs; theoretical considerations, J. Comput. Phys., 229 (2010), pp. 3358--3381.
68.
S. G. Mallat, A Wavelet Tour of Signal Processing: The Sparse Way, 3rd ed., Academic Press, 2009.
69.
R. Matthysen and D. Huybrechs, Fast algorithms for the computation of Fourier extensions of arbitrary length, SIAM J. Sci. Comput., 38 (2016), pp. A899--A922, https://doi.org/10.1137/15M1030923.
70.
R. Matthysen and D. Huybrechs, Function approximation on arbitrary domains using Fourier extension frames, SIAM J. Numer. Anal., 56 (2018), pp. 1360--1385, http://doi.org/10.1137/17M1134809.
71.
A. Neumaier, Solving ill-conditioned and singular linear systems: A tutorial on regularization, SIAM Rev., 40 (1998), pp. 636--666, https://doi.org/10.1137/S0036144597321909.
72.
R. Pasquetti and M. Elghaoui, A spectral embedding method applied to the advection--diffusion equation, J. Comput. Phys., 125 (1996), pp. 464--476.
73.
R. Platte, A. J. Gutierrez, and A. Gelb, Fourier reconstruction of univariate piecewise-smooth functions from non-uniform spectral data with exponential convergence rates, Appl. Comput. Harmon. Anal., 39 (2015), pp. 427--449.
74.
J. M. Renes, Equiangular tight frames from Paley tournaments, Linear Algebra Appl., 426 (2007), pp. 497--501.
75.
D. Shirokoff and J.-C. Nave, A sharp-interface active penalty method for the incompressible Navier--Stokes equations, J. Sci. Comput., 62 (2015), pp. 53--77.
76.
D. Slepian, Prolate spheroidal wave functions. Fourier analysis, and uncertainty V: The discrete case, Bell System Tech J., 57 (1978), pp. 1371--1430.
77.
G. Song, J. Davis, and A. Gelb, A high-dimensional inverse frame operator approximation technique, SIAM J. Numer. Anal., 54 (2016), pp. 2282--2301, https://doi.org/10.1137/15M1047593.
78.
G. Song and A. Gelb, Approximating the inverse frame operator from localized frames, Appl. Comput. Harmon. Anal., 35 (2013), pp. 94--110.
79.
D. B. Stein, R. D. Guy, and B. Thomases, Immersed boundary smooth extension: A high-order method for solving PDE on arbitrary smooth domains using Fourier spectral methods, J. Comput. Phys., 304 (2016), pp. 252--274.
80.
R. Stevenson, Adaptive solution of operator equations using wavelet frames, SIAM J. Numer. Anal., 41 (2003), pp. 1074--1100, https://doi.org/10.1137/S0036142902407988.
81.
T. Strohmer, Numerical analysis of the nonuniform sampling problem, J. Comput. Appl. Math., 122 (2000), pp. 297--316.
82.
T. Strohmer, A note on equiangular tight frames, Linear Algebra Appl., 429 (2008), pp. 326--330.
83.
M. A. Sustik, J. A. Tropp, I. S. Dhillon, and R. W. Heath, On the existence of equiangular tight frames, Linear Algebra Appl., 426 (2007), pp. 619--635.
84.
A. Teolis and J. J. Benedetto, Local frames and noise reduction, Signal Process., 45 (1995), pp. 369--387.
85.
L. N. Trefethen, Approximation Theory and Approximation Practice, SIAM, Philadelphia, 2013.

## Information & Authors

### Information

#### Published In SIAM Review
Pages: 443 - 473
ISSN (online): 1095-7200

#### History

Submitted: 2 February 2017
Accepted: 29 October 2018
Published online: 7 August 2019

### Authors

#### Funding Information

Natural Sciences and Engineering Research Council of Canada https://doi.org/10.13039/501100000038 : 611675

#### Funding Information

Alfred P. Sloan Foundation https://doi.org/10.13039/100000879

#### Funding Information

Fonds Wetenschappelijk Onderzoek https://doi.org/10.13039/501100003130 : G.0641.11, G.A004.14

#### Funding Information

KU Leuven https://doi.org/10.13039/501100004040 : C14/15/055