Local Improvement Results for Anderson Acceleration with Inaccurate Function Evaluations
Abstract
We analyze the convergence of Anderson acceleration when the fixed point map is corrupted with errors. We consider uniformly bounded errors and stochastic errors with infinite tails. We prove local improvement results which describe the performance of the iteration up to the point where the accuracy of the function evaluation causes the iteration to stagnate. We illustrate the results with examples from neutronics.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
Supplementary Material
PLEASE NOTE: These supplementary files have not been peer-reviewed.
Index of Supplementary Materials
Title of paper: Local Improvement Results for Anderson Acceleration with Inaccurate Function Evaluations
Authors: Alex Toth, J. Austin Ellis Tom Evans, Steven Hamilton, C. T. Kelley, Roger Pawlowski, Stuart Slattery
File: Matlab_Kelley_SISC.zip
Type: Compressed Matlab code files
Contents: Matlab files for example: Section 3.1
"heq_anderson_kelley.m" is the driver to produce figure 1 in the paper. The driver uses anderson_optset to configure the solve and anderson_kelley to solve the equations.
"anderson_optset.m" is the options control for Anderson Acceleration code.
"anderson_kelley.m" is the main Anderson Acceleration code.
These three files enable a reader to reproduce the results in section 3.1. The other two examples use large codes, some of which are proprietary, and are not reproducable by others at the present time.
References
1.
E. L. Allgower and K. Böhmer, Application of the mesh independence principle to mesh refinement strategies, SIAM J. Numer. Anal., 24 (1987), pp. 1335--1351, https://doi.org/10.1137/0724086.
2.
E. L. Allgower, K. Böhmer, F. A. Potra, and W. C. Rhienboldt, A mesh-independence principle for operator equations and their discretizations, SIAM J. Numer. Anal., 23 (1986), pp. 160--169, https://doi.org/10.1137/0723011.
3.
D. G. Anderson, Iterative procedures for nonlinear integral equations, J. ACM, 12 (1965), pp. 547--560.
4.
I. W. Busbridge, The Mathematics of Radiative Transfer, Cambridge Tracts 50, Cambridge University Press, Cambridge, UK, 1960.
5.
N. N. Carlson and K. Miller, Design and application of a gradient-weighted moving finite element code I: In one dimension, SIAM J. Sci. Comput., 19 (1998), pp. 728--765, https://doi.org/10.1137/S106482759426955X.
6.
S. Chandrasekhar, Radiative Transfer, Dover, New York, 1960.
7.
A. M. Collier, A. C. Hindmarsh, R. Serban, and C. S. Woodward, User Documentation for KINSOL v2.8.0, Tech. Report UCRL-SM-208116, Lawrence Livermore National Laboratory, Livermore, CA, 2015.
8.
B. Collins, T. Downar, J. Gehin, A. Godfrey, A. Graham, D. Jabaay, B. Kelley, K. Clarno, K. Kim, B. Kochunas, E. Larsen, Y. Liu, Z. Liu, W. Martin, S. Palmtag, M. Rose, T. Saller, S. Stimpson, T. Trahan, J. Wang, W. Wieselquist, M. Young, and A. Zhu, MPACT Theory Manual, Tech. Report CASL-U-2015-0078-000, Consortium for Advanced Simulation of LWRs, 2015.
9.
J. E. Dennis and H. F. Walker, Inaccuracy in quasi-Newton methods: Local improvement theorems, in Mathematical Programming at Oberwolfach II, Mathematical Programming Study 22, North--Holland, Amsterdam, 1984, pp. 70--85.
10.
T. Evans, S. Hamilton, and S. Slattery, ORNL-CEES/Profugus, Oak Ridge National Laboratory, 2016, https://github.com/ORNL-CEES/Profugus.
11.
W. R. Ferng and C. T. Kelley, Mesh independence of matrix-free methods for path following, SIAM J. Sci. Comput., 21 (2000), pp. 1835--1850, https://doi.org/10.1137/S1064827598339360.
12.
D. Gaston, C. Newman, and G. Hansen, MOOSE: A parallel computational framework for coupled systems of nonlinear equations, Nuclear Engineering and Design, 239 (2009), pp. 1768--1778, https://doi.org/10.1016/j.nucengdes.2009.05.021.
13.
J. Hales, S. Novascone, G. Pastore, D. Perez, B. Spencer, and R. Williamson, BISON Theory Manual, Tech. Report October, Fuels Modeling and Simulation Department, Idaho National Laboratory, 2013, https://neup.inl.gov/SiteAssets/FY2014%20Documents/BISON_theory_manual.pdf.
14.
S. Hamilton, M. Berrill, K. Clarno, R. Pawlowski, A. Toth, C. T. Kelley, T. Evans, and B. Philip, An assessment of coupling algorithms for nuclear reactor core physics simulations, J. Comput. Phys., 311 (2016), pp. 241--257.
15.
C. T. Kelley, Iterative Methods for Linear and Nonlinear Equations, Frontiers Appl. Math. 16, SIAM, Philadelphia, 1995.
16.
C. T. Kelley and E. W. Sachs, Mesh independence of Newton-like methods for infinite dimensional problems, J. Int. Eq. Appl., 3 (1991), pp. 549--573.
17.
D. A. Knoll, H. Park, and K. Smith, Application of the Jacobian-free Newton-Krylov method to nonlinear acceleration of transport source iteration in slab geometry, Nuclear Sci. Eng., 167 (2011), pp. 122--132.
18.
K. N. Kudin, G. E. Scuseria, and E. Cancès, A black-box self-consistent field convergence algorithm: One step closer, J. Chem. Phys., 116 (2002), pp. 8255--8261, https://doi.org/10.1063/1.1470195.
19.
E. Lewis and W. Miller, Computational Methods of Neutron Transport, American Nuclear Society, LaGrange Park, IL, 1993.
20.
L. Lin and C. Yang, Elliptic preconditioner for accelerating the self-consistent field iteration in Kohn--Sham density functional theory, SIAM J. Sci. Comput., 35 (2013), pp. S277--S298, https://doi.org/10.1137/120880604.
21.
P. A. Lott, H. F. Walker, C. S. Woodward, and U. M. Yang, An accelerated Picard method for nonlinear systems related to variably saturated flow, Adv. Water Res., 38 (2012), pp. 92--101.
22.
K. Miller, Nonlinear Krylov and moving nodes in the method of lines, J. Comput. Appl. Math., 183 (2005), pp. 275--287.
23.
L. Monti and T. Schulenberg, Coupled ERANOS/TRACE system for HPLWR \textup3 pass core analyses, in International Conference on Mathematics, Computational Methods and Reactor Physics, Saratoga Springs, NY, 2009, pp. 1--14.
24.
C. W. Oosterlee and T. Washio, Krylov subspace acceleration for nonlinear multigrid with application to recirculating flows, SIAM J. Sci. Comput., 21 (2000), pp. 1670--1690, https://doi.org/10.1137/S1064827598338093.
25.
R. Pawlowski, K. Clarno, R. Montgomery, R. Salko, T. Evans, J. Turner, and D. Gaston, Design of a high fidelity core simulator for analysis of pellet clad interaction, in Proceedings of the ANS MC2015 -- Joint International Conference on Mathematics and Computation (M&C), Supercomputing in Nuclear Applications (SNA) and the Monte Carlo (MC) Method, vol. ANS MC2015 CD, Nashville, TN, 2015.
26.
P. Pulay, Convergence acceleration of iterative sequences. The case of SCF iteration, Chem. Phys. Lett., 73 (1980), pp. 393--398.
27.
P. Pulay, Improved SCF convergence acceleration, J. Comput. Chem., 3 (1982), pp. 556--560.
28.
T. Rohwedder and R. Schneider, An analysis for the DIIS acceleration method used in quantum chemistry calculations, J. Math. Chem., 49 (2011), pp. 1889--1914.
29.
R. Salko and M. Avramova, CTF Theory Manual, Tech. Report CASL-U-2015-0054-000, Consortium for Advanced Simulation of LWRs, 2015.
30.
R. Schneider, T. Rohwedder, A. Neelov, and J. Blauert, Direct minimization for calculating invariant subspaces in density functional computations of the electronic structure, J. Comput. Math., 27 (2008), pp. 360--387.
31.
V. Simoncini and D. B. Szyld, Flexible inner-outer Krylov subspace methods, SIAM J. Numer. Anal., 40 (2003), pp. 2219--2239, https://doi.org/10.1137/S0036142902401074.
32.
V. Simoncini and D. B. Szyld, Theory of inexact Krylov subspace methods and applications to scientific computing, SIAM J. Sci. Comput., 25 (2003), pp. 454--477, https://doi.org/10.1137/S1064827502406415.
33.
V. Simoncini and D. B. Szyld, Recent computational developments in Krylov subspace methods for linear systems, Numer. Linear Algebra Appl., 14 (2007), pp. 1--59.
34.
SUNDIALS (SUite of Nonlinear and DIfferential/ALgebraic Solvers), http://www.llnl.gov/casc/sundials.
35.
A. Toth and C. T. Kelley, Convergence analysis for Anderson acceleration, SIAM J. Numer. Anal., 53 (2015), pp. 805--819, https://doi.org/10.1137/130919398.
36.
A. Toth, C. T. Kelley, S. Slattery, S. Hamilton, K. Clarno, and R. Pawlowski, Analysis of Anderson acceleration on a simplified neutronics/thermal hydraulics system, in Proceedings of the ANS MC2015 -- Joint International Conference on Mathematics and Computation (M&C), Supercomputing in Nuclear Applications (SNA) and the Monte Carlo (MC) Method, vol. ANS MC2015 CD, Nashville, TN, 2015.
37.
A. Toth and R. Pawlowski, NOX::Solver::AndersonAcceleration Class Reference, https://trilinos.org/docs/dev/packages/nox/doc/html/classNOX_1_1Solver_1_1AndersonAcceleration.html (2015).
38.
H. W. Walker and P. Ni, Anderson acceleration for fixed-point iterations, SIAM J. Numer. Anal., 49 (2011), pp. 1715--1735, https://doi.org/10.1137/10078356X.
39.
T. Washio and C. Oosterlee, Krylov subspace acceleration for nonlinear multigrid schemes, Electron. Trans. Numer. Anal., 6 (1997), pp. 271--290.
40.
J. Willert, Hybrid Deterministic/Monte Carlo Methods for Solving the Neutron Transport Equation and $k$-Eigenvalue Problem, Ph.D. thesis, North Carolina State University, Raleigh, NC, 2013.
41.
J. Willert, X. Chen, and C. T. Kelley, Newton's method for Monte Carlo--based residuals, SIAM J. Numer. Anal., 53 (2015), pp. 1738--1757, https://doi.org/10.1137/130905691.
42.
J. Willert, C. T. Kelley, D. A. Knoll, and H. K. Park, Hybrid deterministic/Monte Carlo neutronics, SIAM J. Sci. Comput., 35 (2013), pp. S62--S83, https://doi.org/10.1137/120880021.
43.
J. Yan, B. Kochunas, M. Hursin, T. Downar, Z. Karoutas, and E. Baglietto, Coupled computational fluid dynamics and MOC neutronic simulations of Westinghouse PWR fuel assemblies with grid spacers, in 14th International Topical Meeting on Nuclear Reactor Thermalhydraulics, Toronto, Ontario, 2011.
Information & Authors
Information
Published In
Copyright
© 2017, Society for Industrial and Applied Mathematics.
History
Submitted: 11 July 2016
Accepted: 26 October 2016
Published online: 26 October 2017
Keywords
MSC codes
Authors
Funding Information
National Science Foundation https://doi.org/10.13039/100000001 : DMS-1406349, SI2-SSE-1339844
U.S. Department of Energy https://doi.org/10.13039/100000015 : DE-AC05-00OR22725
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Non-stationary Anderson acceleration with optimized dampingJournal of Computational and Applied Mathematics, Vol. 451 | 1 Dec 2024
- Anderson acceleration with approximate calculations: Applications to scientific computingNumerical Linear Algebra with Applications, Vol. 31, No. 5 | 9 May 2024
- Stability and Performance of the X-CMFD Method for Multiphysics Reactor CalculationsNuclear Science and Engineering, Vol. 198, No. 9 | 27 November 2023
- Anderson acceleration for nonlinear PDEs discretized by space–time spectral methodsComputers & Mathematics with Applications, Vol. 167 | 1 Aug 2024
- Anderson Acceleration of Gradient Methods with Energy for Optimization ProblemsCommunications on Applied Mathematics and Computation, Vol. 6, No. 2 | 28 December 2023
- Generalized modulus-based matrix splitting algorithm with Anderson acceleration strategy for vertical linear complementarity problemsJournal of Computational and Applied Mathematics, Vol. 443 | 1 Jun 2024
- Efficient spin-up of Earth System Models using sequence accelerationScience Advances, Vol. 10, No. 18 | 3 May 2024
- SARAH-M: A fast stochastic recursive gradient descent algorithm via momentumExpert Systems with Applications, Vol. 238 | 1 Mar 2024
- Asymptotic convergence analysis and influence of initial guesses on composite Anderson accelerationAdvances in Computational Mathematics, Vol. 49, No. 6 | 13 December 2023
- Adaptive Anderson mixing for electronic structure calculationsComputer Physics Communications, Vol. 292 | 1 Nov 2023
- Nonmonotone Globalization for Anderson Acceleration via Adaptive RegularizationJournal of Scientific Computing, Vol. 96, No. 1 | 18 May 2023
- Stochastic algorithms for self-consistent calculations of electronic structuresMathematics of Computation, Vol. 92, No. 342 | 28 February 2023
- On an improved PDE-based elliptic parameterization method for isogeometric analysis using preconditioned Anderson accelerationComputer Aided Geometric Design, Vol. 102 | 1 May 2023
- Fast Spin‐Up of Geochemical Tracers in Ocean Circulation and Climate ModelsJournal of Advances in Modeling Earth Systems, Vol. 15, No. 2 | 30 January 2023
- Composite Anderson acceleration method with two window sizes and optimized dampingInternational Journal for Numerical Methods in Engineering, Vol. 123, No. 23 | 26 August 2022
- Anderson Acceleration for Distributed Training of Deep Learning ModelsSoutheastCon 2022 | 26 Mar 2022
- Convergence analysis of adaptive DIIS algorithms with application to electronic ground state calculationsESAIM: Mathematical Modelling and Numerical Analysis, Vol. 55, No. 6 | 25 November 2021
- Predictive Mixing for Density Functional Theory (and Other Fixed-Point Problems)Journal of Chemical Theory and Computation, Vol. 17, No. 9 | 16 August 2021
- Numerical optimization of a multiphysics calculation scheme based on partial convergenceAnnals of Nuclear Energy, Vol. 151 | 1 Feb 2021
- Anderson Acceleration for a Class of Nonsmooth Fixed-Point ProblemsSIAM Journal on Scientific Computing, Vol. 43, No. 5 | 20 January 2021
- Anderson Acceleration for Nonconvex ADMM Based on Douglas‐Rachford SplittingComputer Graphics Forum, Vol. 39, No. 5 | 12 August 2020
- A Brief Introduction to Manifold OptimizationJournal of the Operations Research Society of China, Vol. 8, No. 2 | 4 April 2020
- Globally Convergent Type-I Anderson Acceleration for Nonsmooth Fixed-Point IterationsSIAM Journal on Optimization, Vol. 30, No. 4 | 18 November 2020
- Anderson Accelerated Douglas--Rachford SplittingSIAM Journal on Scientific Computing, Vol. 42, No. 6 | 9 November 2020
- Accelerating ADMM for efficient simulation and optimizationACM Transactions on Graphics, Vol. 38, No. 6 | 8 November 2019
- Convergence of the EDIIS Algorithm for Nonlinear EquationsSIAM Journal on Scientific Computing, Vol. 41, No. 1 | 17 January 2019
- Anderson acceleration for geometry optimization and physics simulationACM Transactions on Graphics, Vol. 37, No. 4 | 30 July 2018
- Numerical methods for nonlinear equationsActa Numerica, Vol. 27 | 4 May 2018
- A Smoothing Direct Search Method for Monte Carlo-Based Bound Constrained Composite Nonsmooth OptimizationSIAM Journal on Scientific Computing, Vol. 40, No. 4 | 12 July 2018
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].