Compressive sensing is a powerful technique for recovering sparse solutions of underdetermined linear systems, which is often encountered in uncertainty quantification analysis of expensive and high-dimensional physical models. We perform numerical investigations employing several compressive sensing solvers that target the unconstrained LASSO formulation, with a focus on linear systems that arise in the construction of polynomial chaos expansions. With core solvers l1_ls, SpaRSA, CGIST, FPC_AS, and ADMM, we develop techniques to mitigate overfitting through an automated selection of regularization constant based on cross-validation, and a heuristic strategy to guide the stop-sampling decision. Practical recommendations on parameter settings for these techniques are provided and discussed. The overall method is applied to a series of numerical examples of increasing complexity, including large eddy simulations of supersonic turbulent jet-in-crossflow involving a 24-dimensional input. Through empirical phase-transition diagrams and convergence plots, we illustrate sparse recovery performance under structures induced by polynomial chaos, accuracy, and computational trade-offs between polynomial bases of different degrees, and practicability of conducting compressive sensing for a realistic, high-dimensional physical application. Across test cases studied in this paper, we find ADMM to have demonstrated empirical advantages through consistent lower errors and faster computational times.


  1. uncertainty quantification
  2. LASSO
  3. compressed sensing
  4. sparse regression
  5. sparse reconstruction
  6. $\ell_1$-regularization
  7. sequential compressive sensing

MSC codes

  1. 62J05
  2. 94A12
  3. 65Z05
  4. 62P35

Get full access to this article

View all available purchase options and get full access to this article.


H. Akaike, A new look at the statistical model identification, IEEE Trans. Automat. Contr., 19 (1974), pp. 716--723, https://doi.org/10.1109/TAC.1974.1100705.
S. Babacan, R. Molina, and A. Katsaggelos, Bayesian compressive sensing using Laplace priors, IEEE Trans. Image Process., 19 (2010), pp. 53--63, https://doi.org/10.1109/TIP.2009.2032894.
V. Barthelmann, E. Novak, and K. Ritter, High dimensional polynomial interpolation on sparse grids, Adv. Comput. Math., 12 (2000), pp. 273--288, https://doi.org/10.1023/A:1018977404843.
G. Blatman and B. Sudret, Adaptive sparse polynomial chaos expansion based on least angle regression, J. Comput. Phys., 230 (2011), pp. 2345--2367, https://doi.org/10.1016/j.jcp.2010.12.021.
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (2010), pp. 1--122, https://doi.org/10.1561/2200000016.
S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Solve LASSO via ADMM, 2011, https://web.stanford.edu/~boyd/papers/admm/lasso/lasso.html (accessed 2017-07-02).
E. J. Candès, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489--509, https://doi.org/10.1109/TIT.2005.862083.
S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM Rev., 43 (2001), pp. 129--159, https://doi.org/10.1137/S003614450037906X.
G. Davis, S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Constr. Approx., 13 (1997), pp. 57--98, https://doi.org/10.1007/BF02678430.
D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289--1306, https://doi.org/10.1109/Tit.2006.871582.
D. L. Donoho, For most large underdetermined systems of linear equations the minimal $\ell_1$-norm solution is also the sparsest solution, Comm. Pure Appl. Math., 59 (2006), pp. 797--829, https://doi.org/10.1002/cpa.20132.
D. L. Donoho and J. Tanner, Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing, Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 367 (2009), pp. 4273--4293, https://doi.org/10.1098/rsta.2009.0152.
D. L. Donoho and J. Tanner, Precise undersampling theorems, Proc. IEEE, 98 (2010), pp. 913--924, https://doi.org/10.1109/JPROC.2010.2045630.
B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, Least angle regression, Ann. Statist., 32 (2004), pp. 407--499, https://doi.org/10.1214/009053604000000067.
M. S. Eldred, L. W. T. Ng, M. F. Barone, and S. P. Domino, Multifidelity uncertainty quantification using spectral stochastic discrepancy models, in Handbook of Uncertainty Quantification, Springer, Cham, 2015, pp. 1--45, https://doi.org/10.1007/978-3-319-11259-6_25-1.
O. G. Ernst, A. Mugler, H.-J. Starkloff, and E. Ullmann, On the convergence of generalized polynomial chaos expansions, ESAIM Math. Model. Numer. Anal., 46 (2012), pp. 317--339, https://doi.org/10.1051/m2an/2011045.
N. Fajraoui, S. Marelli, and B. Sudret, On Optimal Experimental Designs for Sparse Polynomial Chaos Expansions, preprint, https://arxiv.org/abs/1703.05312, 2017.
T. Gerstner and M. Griebel, Numerical integration using sparse grids, Numer. Algorithms, 18 (1998), pp. 209--232, https://doi.org/10.1023/A:1019129717644.
T. Gerstner and M. Griebel, Dimension-adaptive tensor-product quadrature, Computing, 71 (2003), pp. 65--87, https://doi.org/10.1007/s00607-003-0015-5.
R. G. Ghanem and P. D. Spanos, Stochastic Finite Elements: A Spectral Approach, 1st ed., Springer-Verlag, New York, 1991.
T. Goldstein and S. Setzer, High-order Methods for Basis Pursuit, Tech. Report CAM Report 10-41, University of California, Los Angeles, 2010.
T. Goldstein and S. Setzer, CGIST: A Fast and Exact Solver for L1 Minimization, 2011, http://tag7.web.rice.edu/CGIST.html (accessed 2017-07-02).
J. H. Halton, On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals, Numer. Math., 2 (1960), pp. 84--90, https://doi.org/10.1007/BF01386213.
J. Hampton and A. Doostan, Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies, J. Comput. Phys., 280 (2015), pp. 363--386, https://doi.org/10.1016/j.jcp.2014.09.019.
N. E. Hass, K. F. Cabell, and A. M. Storch, HIFiRE Direct-Connect Rig (HDCR) Phase I Ground Test Results from the NASA Langley Arc-Heated Scramjet Test Facility, Tech. Report CR-2010-002215, NASA, 2010.
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed., Springer, New York, 2009.
X. Huan, C. Safta, K. Sargsyan, G. Geraci, M. S. Eldred, Z. P. Vane, G. Lacaze, J. C. Oefelein, and H. N. Najm, Global sensitivity analysis and estimation of model error, toward uncertainty quantification in scramjet computations, AIAA J., 56 (2018), pp. 1170--1184, https://doi.org/10.2514/1.J056278.
J. D. Jakeman, M. S. Eldred, and K. Sargsyan, Enhancing $\ell_1$-minimization estimates of polynomial chaos expansions using basis selection, J. Comput. Phys., 289 (2015), pp. 18--34, https://doi.org/10.1016/j.jcp.2015.02.025.
J. D. Jakeman, A. Narayan, and T. Zhou, A generalized sampling and preconditioning scheme for sparse approximation of polynomial chaos expansions, SIAM J. Sci. Comput., 39 (2017), pp. A1114--A1144, https://doi.org/10.1137/16M1063885.
S. Ji, Y. Xue, and L. Carin, Bayesian compressive sensing, IEEE Trans. Signal Process., 56 (2008), pp. 2346--2356, https://doi.org/10.1109/TSP.2007.914345.
R. E. Kass and A. E. Raftery, Bayes factor, J. Amer. Statist. Assoc., 90 (1995), pp. 773--795, https://doi.org/10.2307/2291091.
S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, An interior point method for large-scale $\ell_1$-regularized least squares, IEEE J. Sel. Topics Signal Process., 1 (2007), pp. 606--617, https://doi.org/10.1109/JSTSP.2007.910971.
K. Koh, S.-J. Kim, and S. Boyd, Simple Matlab Solver for l$1$-regularized Least Squares Problems, beta version, 2008, https://stanford.edu/~boyd/l1_ls/ (accessed 2017-07-02).
O. P. Le Maître and O. M. Knio, Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics, Springer Netherlands, Houten, Netherlands, 2010.
K. Mahesh, The interaction of jets with crossflow, Annu. Rev. Fluid Mech., 45 (2013), pp. 379--407, https://doi.org/10.1146/annurev-fluid-120710-101115.
D. M. Malioutov, S. R. Sanghavi, and A. S. Willsky, Sequential compressed sensing, IEEE J. Sel. Topics Signal Process., 4 (2010), pp. 435--444, https://doi.org/10.1109/JSTSP.2009.2038211.
H. N. Najm, Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics, Annu. Rev. Fluid Mech., 41 (2009), pp. 35--52, https://doi.org/10.1146/annurev.fluid.010908.165248.
J. C. Oefelein, Simulation and Analysis of Turbulent Multiphase Combustion Processes at High Pressures, Ph.D. thesis, The Pennsylvania State University, State College, PA 1997.
J. C. Oefelein, Large eddy simulation of turbulent combustion processes in propulsion and power systems, Prog. Aerosp. Sci., 42 (2006), pp. 2--37, https://doi.org/10.1016/j.paerosci.2006.02.001.
J. Peng, J. Hampton, and A. Doostan, A weighted $\ell_1$-minimization approach for sparse polynomial chaos expansions, J. Comput. Phys., 267 (2014), pp. 92--111, https://doi.org/10.1016/j.jcp.2014.02.024.
H. Rauhut and R. Ward, Sparse Legendre expansions via $\ell_1$-minimization, J. Approx. Theory, 164 (2012), pp. 517--533, https://doi.org/10.1016/j.jat.2012.01.008.
K. Sargsyan, C. Safta, H. N. Najm, B. J. Debusschere, D. Ricciuto, and P. Thornton, Dimensionality reduction for complex models via Bayesian compressive sensing, Int. J. Uncertain. Quantif., 4 (2014), pp. 63--93, https://doi.org/10.1615/Int.J.UncertaintyQuantification.2013006821.
G. Schwarz, Estimating the dimension of a model, Ann. Statist., 6 (1978), pp. 461--464, https://doi.org/10.1214/aos/1176344136.
J. Tanner, Phase Transitions of the Regular Polytopes and Cone, 2012, https://people.maths.ox.ac.uk/tanner/polytopes.shtml (accessed 2017-07-03).
E. van den Berg and M. P. Friedlander, Probing the Pareto frontier for basis pursuit solutions, SIAM J. Sci. Comput., 31 (2008), pp. 890--912, https://doi.org/10.1137/080714488.
L. Wasserman, Bayesian model selection and model averaging, J. Math. Psych., 44 (2000), pp. 92--107, https://doi.org/10.1006/jmps.1999.1278.
Z. Wen, W. Yin, D. Goldfarb, and Y. Zhang, A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation, SIAM J. Sci. Comput., 32 (2010), pp. 1832--1857, https://doi.org/10.1137/090747695.
S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, SpaRSA: Sparse Reconstruction by Separable Approximation, version 2.0, 2009, http://www.lx.it.pt/~mtf/SpaRSA/ (accessed 2017-07-02).
S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, Sparse reconstruction by separable approximation, IEEE Trans. Signal Process., 57 (2009), pp. 2479--2493, https://doi.org/10.1109/TSP.2009.2016892.
D. Xiu, Fast numerical methods for stochastic computations: A review, Commun. Comput. Phys., 5 (2009), pp. 242--272.
D. Xiu and G. E. Karniadakis, The Wiener--Askey polynomial chaos for stochastic differential equations, SIAM J. Sci. Comput., 24 (2002), pp. 619--644, https://doi.org/10.1137/S1064827501387826.
W. Yin and Z. Wen, FPC_AS: A MATLAB Solver for L1-Regularization Problems, version 1.21, 2010, http://www.caam.rice.edu/~optimization/L1/FPC_AS/ (accessed 2017-07-02).

Information & Authors


Published In

cover image SIAM/ASA Journal on Uncertainty Quantification
SIAM/ASA Journal on Uncertainty Quantification
Pages: 907 - 936
ISSN (online): 2166-2525


Submitted: 28 July 2017
Accepted: 27 March 2018
Published online: 19 June 2018


  1. uncertainty quantification
  2. LASSO
  3. compressed sensing
  4. sparse regression
  5. sparse reconstruction
  6. $\ell_1$-regularization
  7. sequential compressive sensing

MSC codes

  1. 62J05
  2. 94A12
  3. 65Z05
  4. 62P35



Funding Information

Defense Advanced Research Projects Agency https://doi.org/10.13039/100000185

Funding Information

National Nuclear Security Administration https://doi.org/10.13039/100006168 : DE-NA-0003525

Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By







Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.