Methods and Algorithms for Scientific Computing

Deflation Techniques for Finding Distinct Solutions of Nonlinear Partial Differential Equations


Nonlinear systems of partial differential equations (PDEs) may permit several distinct solutions. The typical current approach to finding distinct solutions is to start Newton's method with many different initial guesses, hoping to find starting points that lie in different basins of attraction. In this paper, we present an infinite-dimensional deflation algorithm for systematically modifying the residual of a nonlinear PDE problem to eliminate known solutions from consideration. This enables the Newton--Kantorovitch iteration to converge to several different solutions, even starting from the same initial guess. The deflated Jacobian is dense, but an efficient preconditioning strategy is devised, and the number of Krylov iterations is observed not to grow as solutions are deflated. The power of the approach is demonstrated on several problems from special functions, phase separation, differential geometry, and fluid mechanics that permit distinct solutions.


  1. deflation
  2. Newton's method
  3. distinct solutions
  4. continuation

MSC codes

  1. 65N30
  2. 65N35
  3. 65H99
  4. 35B32

Get full access to this article

View all available purchase options and get full access to this article.


M. F. Adams, H. H. Bayraktar, T. M. Keaveny, and P. Papadopoulos, Ultrascalable implicit finite element analyses in solid mechanics with over a half a billion degrees of freedom, in ACM/IEEE Proceedings SC2004, IEEE, Piscataway, NJ, 2004.
S. M. Allen and J. W. Cahn, A microscopic theory for antiphase boundary motion and its application to antiphase domain coarsening, Acta Metall., 27 (1979), pp. 1085--1095.
E. L. Allgower and K. Georg, Continuation and path following, Acta Numer., 2 (1993), pp. 1--64.
E. L. Allgower and K. Georg, Introduction to Numerical Continuation Methods, Classics Appl. Math. 45, SIAM, Philadelphia, 2003.
P. R. Amestoy, I. S. Duff, J. Koster, and J.-Y. L'Excellent, A fully asynchronous multifrontal solver using distributed dynamic scheduling, SIAM J. Matrix Anal. Appl., 23 (2001), pp. 15--41.
S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith, Efficient management of parallelism in object oriented numerical software libraries, in Modern Software Tools in Scientific Computing, E. Arge, A. M. Bruaset, and H. P. Langtangen, eds., Birkhäuser, Boston, 1997, pp. 163--202.
M. S. Bartlett, An inverse matrix adjustment arising in discriminant analysis, Annals Math. Statist., 22 (1951), pp. 107--111.
D. J. Bates, J. D. Hauenstein, A. J. Sommese, and C. W. Wampler, Numerically Solving Polynomial Systems with Bertini, Software Environ. Tools 25, SIAM, Philadelphia, 2013.
T. B. Benjamin, Bifurcation phenomena in steady flows of a viscous fluid. I. Theory, R. Soc. Lond. Proc. Ser. A. Math. Phys. Eng. Sci., 359 (1978), pp. 1--26.
Á. Birkisson, Numerical Solution of Nonlinear Boundary Value Problems for Ordinary Differential Equations in the Continuous Framework, Ph.D. thesis, University of Oxford, Oxford, UK,2014.
Á. Birkisson and T. A. Driscoll, Automatic Fréchet differentiation for the numerical solution of boundary-value problems, ACM Trans. Math. Software, 38 (2012), p. 26.
A. Borzì and V. Schulz, Computational Optimization of Systems Governed by Partial Differential Equations, SIAM, Philadelphia, 2011.
F. H. Branin, Widely convergent method for finding multiple solutions of simultaneous nonlinear equations, IBM J. Res. Develop., 16 (1972), pp. 504--522.
H. Brezis and L. Nirenberg, Positive solutions of nonlinear elliptic equations involving critical Sobolev exponents, Comm. Pure Appl. Math., 36 (1983), pp. 437--477.
J. Brown and P. Brune, Low-rank quasi-Newton updates for robust Jacobian lagging in Newton methods, in Proceedings of the 2013 International Conference on Mathematics and Computational Methods Applied to Nuclear Science and Engineering, Sun Valley, ID, 2013, American Nuclear Society, La Grange Park, IL, 2013, pp. 2554--2565.
K. M. Brown and W. B. Gearhart, Deflation techniques for the calculation of further solutions of a nonlinear system, Numer. Math., 16 (1971), pp. 334--342.
P. Brune, M. G. Knepley, B. Smith, and X. Tu, Composing Scalable Nonlinear Algebraic Solvers, Technical report ANL/MCS-P2010-0112, Argonne National Laboratory, Argonne, IL, 2013.
K.-S. Chao, D.-K. Liu, and C.-T. Pan, A systematic search method for obtaining multiple solutions of simultaneous nonlinear equations, IEEE Trans. Circuits Systems, 22 (1975), pp. 748--753.
M.-J. Chien, Searching for multiple solutions of nonlinear systems, IEEE Trans. Circuits Systems, 26 (1979), pp. 817--827.
K. A. Cliffe, E. J. C. Hall, and P. Houston, $hp$-adaptive discontinuous Galerkin methods for bifurcation phenomena in open flows, Comput. Math. Appl., 67 (2014), pp. 796--806.
D. F. Davidenko, On a new method of numerical solution of systems of nonlinear equations, Dokl. Akad. Nauk SSSR, 88 (1953), pp. 601--602 (in Russian).
H. T. Davis, Introduction to Nonlinear Differential and Integral Equations, Dover, New York, 1962.
P. Deuflhard, Newton Methods for Nonlinear Problems, Springer Ser. Comput. Math. 35, Springer-Verlag, Berlin, 2011.
D. Drikakis, Bifurcation phenomena in incompressible sudden expansion flows, Phys. Fluids, 9 (1997), pp. 76--87.
T. A. Driscoll, N. Hale, and L. N. Trefethen, Chebfun Guide, Pafnuty, Oxford, 2014.
W. E, W. Ren, and E. Vanden-Eijnden, String method for the study of rare events, Phys. Rev. B (3), 66 (2002), 052301.
W. E, W. Ren, and E. Vanden-Eijnden, Minimum action method for the study of rare events, Comm. Pure Appl. Math., 57 (2004), pp. 637--656.
W. E, W. Ren, and E. Vanden-Eijnden, Simplified and improved string method for computing the minimum energy paths in barrier-crossing events, J. Chem. Phys., 126 (2007), 164103.
J. Erway and M. Holst, Barrier Methods for Critical Exponent Problems in Geometric Analysis and Mathematical Physics, preprint, arXiv:1107.0360 [math.PH], 2011.
R. M. Fearn, T. Mullin, and K. A. Cliffe, Nonlinear flow phenomena in a symmetric sudden expansion, J. Fluid Mech., 211 (1990), pp. 595--608.
B. Fornberg and J. A. C. Weideman, A numerical methodology for the Painlevé equations, J. Comput. Phys., 230 (2011), pp. 5957--5973.
C. Geuzaine and J.-F. Remacle, GMSH: A $3$-D finite element mesh generator with built-in pre- and post-processing facilities, Internat. J. Numer. Methods Engrg, 79 (2009), pp. 1309--1331.
A. Griewank, On solving nonlinear equations with simple singularities or nearly singular solutions, SIAM Rev., 27 (1985), pp. 537--563.
W. H. Hager, Updating the inverse of a matrix, SIAM Rev., 31 (1989), pp. 221--239.
W. Hao, J. D. Hauenstein, B. Hu, and A. J. Sommese, A bootstrapping approach for computing multiple solutions of differential equations, J. Comput. Appl. Math., 258 (2014), pp. 181--190.
S. P. C. Hastings and W. P. C. Troy, On some conjectures of Turcotte, Spence, Bau, and Holmes, SIAM J. Math. Anal., 20 (1989), pp. 634--642.
M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich, Optimization with PDE Constraints, Math. Model. Theory Appl. 23, Springer, Dordrecht, 2009.
P. Holmes and D. Spence, On a Painlevé-type boundary-value problem, Quart. J. Mech. Appl. Math., 37 (1984), pp. 525--538.
J. K. Hunter and B. Nachtergaele, Applied Analysis, World Scientific, Singapore, 2001.
H. B. Keller, Numerical solution of bifurcation and nonlinear eigenvalue problems, in Applications of Bifurcation Theory, P. H. Rabinowitz, ed., Academic, New York, 1977, pp. 359--384.
J. M. Lee and T. H. Parker, The Yamabe problem, Bull. Amer. Math. Soc., 17 (1987), pp. 37--91.
A. Leykin, J. Verschelde, and A. Zhao, Newton's method with deflation for isolated singularities of polynomial systems, Theoret. Comput. Sci., 359 (2006), pp. 111--122.
A. Logg, K. A. Mardal, G. N. Wells, et al., Automated Solution of Differential Equations by the Finite Element Method, Springer, Heidelberg, 2011.
V. A. Noonburg, A separating surface for the Painlevé differential equation $x''=x^2-t$, J. Math. Analy. Appl., 193 (1995), pp. 817--831.
T. Ojika, S. Watanabe, and T. Mitsui, Deflation algorithm for the multiple roots of a system of nonlinear equations, J. Math. Anal. Appl., 96 (1983), pp. 463--479.
J. Ortega and W. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Classics Appl. Math. 30, SIAM, Philadelphia, 2000.
G. Peters and J. H. Wilkinson, Practical problems arising in the solution of polynomial equations, IMA J. Appl. Math., 8 (1971), pp. 16--35.
Y. Saad and M. H. Schultz, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856--869.
R. Seydel, Practical Bifurcation and Stability Analysis, 3rd ed., Interdiscip. Appl. Math. 5, Springer, New York, 2010.
I. J. Sobey and T. Mullin, Calculation of multiple solutions for the two--dimensional Navier-Stokes equations, in Proceedings of Numerical Methods for Fluid Dynamics IV, K. W. Morton and M. Baines, eds., Clarendon Press, Oxford, 1993, pp. 417--424.
C. Taylor and P. Hood, A numerical solution of the Navier-Stokes equations using the finite element technique, Comput. & Fluids, 1 (1973), pp. 73--100.
L. N. Trefethen, Spectral Methods in Matlab, Software, Environ. Tools 10, SIAM, Philadelphia, 2000.
J. H. Wilkinson, Rounding Errors in Algebraic Processes, Natl. Phys. Lab. Notes Appl. Sci. 32, H.M.S.O., London, 1963.
H. Yamabe, On a deformation of Riemannian structures on compact manifolds, Osaka Math. J., 12 (1960), pp. 21--37.

Information & Authors


Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: A2026 - A2045
ISSN (online): 1095-7197


Submitted: 2 September 2014
Accepted: 27 May 2015
Published online: 13 August 2015


  1. deflation
  2. Newton's method
  3. distinct solutions
  4. continuation

MSC codes

  1. 65N30
  2. 65N35
  3. 65H99
  4. 35B32



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By







Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account