An Inexact Uzawa Algorithmic Framework for Nonlinear Saddle Point Problems with Applications to Elliptic Optimal Control Problem
Abstract
We consider a class of nonlinear saddle point problems with various applications in PDEs and optimal control problems and propose an algorithmic framework based on some inexact Uzawa methods in the literature. Under mild conditions, the convergence of this algorithmic framework is uniformly proved and the linear convergence rate is estimated. We take an elliptic optimal control problem with control constraints as an example to illustrate how to choose application-tailored preconditioners to generate specific and efficient algorithms by the algorithmic framework. The resulting algorithm does not need to solve any optimization subproblems or systems of linear equations in its iteration; each of its iterations only requires the projection onto a simple admissible set, four algebraic multigrid V-cycles, and a few matrix-vector multiplications. Its numerical efficiency is then demonstrated by some preliminary numerical results.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
K. J. Arrow, L. Hurwicz, and H. Uzawa, Studies in Linear and Non-linear Programming, Stanford Math. Stud. Social Sci., Standford University Press, Stanford, CA, 1958.
2.
C. Bacuta, A unified approach for Uzawa algorithms, SIAM J. Numer. Anal., 44 (2006), pp. 2633--2649.
3.
R. E. Bank, B. D. Welfert, and H. Yserentant, A class of iterative methods for solving saddle point problems, Numer. Math., 56 (1989), pp. 645--666.
4.
M. Benzi, G. H. Golub, and J. Liesen, Numerical solution of saddle point problems, Acta Numer., 14 (2005), pp. 1--137.
5.
M. Bergounioux, K. Ito, and K. Kunisch, Primal-dual strategy for constrained optimal control problems, SIAM J. Control Optim., 37 (1999), pp. 1176--1194.
6.
D. Boffi, F. Brezzi, and M. Fortin, Mixed Finite Element Methods and Applications, Springer Ser. Comput. Math. 44, Springer, New York, 2013.
7.
A. Borz\`\i and V. Schulz, Computational Optimization of Systems Governed by Partial Differential Equations, Comput. Sci. Eng. 8, SIAM, Philadelphia, 2011.
8.
R. Becker and B. Vexler, Optimal control of the convection-diffusion equation using stabilized finite element methods, Numer. Math., 106 (2007), pp. 349--367.
9.
J. H. Bramble and J. E. Pasciak, A preconditioning technique for indefinite systems resulting from mixed approximations of elliptic problems, Math. Comp., 50 (1988), pp. 1--17.
10.
J. H. Bramble, J. E. Pasciak, and A. T. Vassilev, Analysis of the inexact Uzawa algorithm for saddle point problems, SIAM J. Numer. Anal., 34 (1997), pp. 1072--1092.
11.
F. Brezzi and M. Fortin, Mixed and Hybrid Finite Element Methods, Springer Ser. Comput. Math. 15, Springer, New York, 2012.
12.
X. L. Briggs, V. E. Henson, and S. F. McCormick, A Multigrid Tutorial, SIAM, Philadelphia, 2000.
13.
C. Carthel, R. Glowinski, and J. L. Lions, On exact and approximate boundary controllabilities for the heat equation: a numerical approach, J. Optim. Theory Appl., 82 (1994), pp. 429--484.
14.
E. Casas, Using piecewise linear functions in the numerical approximation of semilinear elliptic control problems, Adv. Comput. Math., 26 (2007), pp. 137--153.
15.
L. Chen, iFEM: An Innovative Finite Element Methods Package in MATLAB, Technical report, UC Irvine, 2008.
16.
X. Chen, Global and superlinear convergence of inexact Uzawa methods for saddle point problems with nondifferentiable mappings, SIAM J. Numer. Anal., 35 (1998), pp. 1130--1148.
17.
Z. Chen, Q. Du, and J. Zou, Finite element methods with matching and nonmatching meshes for Maxwell equations with discontinuous coefficients, SIAM J. Numer. Anal., 37 (2000), pp. 1542--1570.
18.
Z. Chen and J. Zou, An augmented Lagrangian method for identifying discontinuous parameters in elliptic systems, SIAM J. Control Optim., 37 (1999), pp. 892--910.
19.
X. Cheng and W. Han, Inexact Uzawa algorithms for variational inequalities of the second kind, Comput. Methods Appl. Mech. Engrg., 192 (2003), pp. 1451--1462.
20.
F. H. Clarke, Optimization and Nonsmooth Analysis, Classics in Appl. Math. 5, SIAM, Philadelphia, 1990.
21.
J. C. De los Reyes, Numerical PDE-Constrained Optimization, Springer, New York, 2015.
22.
P. Destuynder and T. Nevers, Some numerical aspects of mixed finite elements for bending plates, Comput. Methods Appl. Mech. and Engrg., 78 (1990), pp. 73--87.
23.
H. C. Elman and G. H. Golub, Inexact and preconditioned Uzawa algorithms for saddle point problems, SIAM J. Numer. Anal., 31 (1994), pp. 1645--1661.
24.
H. C. Elman, D. J. Silvester, and A. J. Wathen, Finite Elements and Fast Iterative Solvers: With Applications in Incompressible Fluid Dynamics, Oxford University Press, Oxford, UK, 2014.
25.
R. E. Ewing and M. F. Wheeler, Computational aspects of mixed finite element methods, in Numerical Methods for Scientific Computing, North-Holland, New York, 1983, pp. 163--172.
26.
F. Facchinei and J. S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems, Springer, New York, 2007.
27.
R. D. Falgout, An introduction to algebraic multigrid, Comput. Sci. Eng., 8 (2006), pp. 24--33.
28.
R. S. Falk, Approximation of a class of optimal control problems with order of convergence estimates, J. Math. Anal. Appl., 44 (1973), pp. 28--47.
29.
R. S. Falk, An analysis of the finite element method using Lagrange multipliers for the stationary stokes equation, Math. Comp., 30 (1976), pp. 241--249.
30.
M. Fortin and R. Glowinski, Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, Stud. Math. Appl. 15, Elsevier, Amsterdam, 2000.
31.
R. Glowinski, Numerical Methods for Nonlinear Variational Problems, Springer, New York, 1984.
32.
R. Glowinski, Variational Methods for the Numerical Solution of Nonlinear Elliptic Problems, CBMS-NSF Regional Conf. Ser. in Appl. Math. 86, SIAM, Philadelphia, 2015.
33.
R. Glowinski, J. L. Lions, and J. He, Exact and Approximate Controllability for Distributed Parameter Systems: A Numerical Approach, Encyclopedia Math. Appl., Cambridge University Press, Cambridge, UK, 2008.
34.
R. Glowinski and A. Marroco, Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de dirichlet non linéaires, ESAIM Math. Model. Number. Anal., 9 (1975), pp. 41--76.
35.
C. Gräser and R. Kornhuber, On preconditioned Uzawa-type iterations for a saddle point problem with inequality constraints, in Domain Decomposition Methods in Science and Engineering XVI, Lect. Notes Comput. Sci. Eng. 55, Springer, New York, 2007, pp. 91--102.
36.
R. Herzog and E. Sachs, Preconditioned conjugate gradient method for optimal control problems with control and state constraints, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2291--2317.
37.
M. R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303--320.
38.
M. Hintermüller, K. Ito, and K. Kunisch, The primal-dual active set strategy as a semismooth newton method, SIAM J. Optim., 13 (2002), pp. 865--888.
39.
M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich, Optimization with PDE Constraints, Math. Model. Theory Appl. 23, Springer, New York, 2008.
40.
M. Hinze, and M. Vierling,The semi-smooth Newton method for variationally discretized control constrained elliptic optimal control problems; implementation, convergence and globalization, Optim. Methods Softw., 27 (2012), pp. 933--950.
41.
Q. Hu and J. Zou, Two new variants of nonlinear inexact Uzawa algorithms for saddle-point problems, Numer. Math., 93 (2002), pp. 333--359.
42.
Q. Hu and J. Zou, Nonlinear inexact Uzawa algorithms for linear and nonlinear saddle-point problems, SIAM J. Optim., 16 (2006), pp. 798--825.
43.
E. Laitinen, A. Lapin, and S. Lapin, Iterative solution methods for variational inequalities with nonlinear main operator and constraints to gradient of solution, Lobachevskii J. Math., 33 (2012), pp. 341--352.
44.
A. Lapin, Preconditioned Uzawa-type methods for finite-dimensional constrained saddle point problems, Lobachevskii J. Math., 31 (2010), pp. 309--322.
45.
J. L. Lions, Optimal Control of Systems Governed by Partial Differential Equations, Grundlehren Math. Wiss. 170, Springer, Berlin, 1971.
46.
Y. Liu, X. Yuan, S. Zeng, and J. Zhang, Partial error bound conditions and the linear convergence rate of the alternating direction method of multipliers, SIAM J. Numer. Anal., 56 (2018), pp. 2095--2123.
47.
B. Nour-Omid and P. Wriggers, A two-level iteration method for solution of contact problems, Comput. Methods Appl. Mech. Engrg., 54 (1986), pp. 131--144.
48.
C. C. Paige and M. A. Saunders, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12 (1975), pp. 617--629.
49.
J. W. Pearson and A. J. Wathen, A new approximation of the schur complement in preconditioners for PDE-constrained optimization, Numer. Linear Algebra Appl., 19 (2012), pp. 816--829.
50.
J. W. Pearson and A. J. Wathen, Fast iterative solvers for convection-diffusion control problems, Electron. Trans. Numer. Anal., 40 (2013), pp. 294--310.
51.
M. J. Powell, A method for nonlinear constraints in minimization problems, in Optimization, Academic Press, London, 1969, pp. 283--298.
52.
M. Porcelli, V. Simoncini, and M. Tani, Preconditioning of active-set Newton methods for PDE-constrained optimal control problems, SIAM J. Sci. Comput., 37 (2015), pp. S472--S502.
53.
W. Queck, The convergence factor of preconditioned algorithms of the Arrow-Hurwicz type, SIAM J. Numer. Anal., 26 (1989), pp. 1016--1030.
54.
M. P. Robichaud, P. A. Tanguy, and M. Fortin, An iterative implementation of the Uzawa algorithm for 3-d fluid flow problems, Internat. J. Numer. Methods Fluids, 10 (1990), pp. 429--442.
55.
A. Rösch and B. Vexler, Optimal control of the Stokes equations: A priori error analysis for finite element discretization with postprocessing, SIAM J. Numer. Anal., 44 (2006), pp. 1903--1920.
56.
R. T. Rockafellar and R. Wets, Variational Analysis, Springer, New York, 2009.
57.
T. Rusten and R. Winther, A preconditioned iterative method for saddlepoint problems, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 887--904.
58.
B. Rynne, and M. A. Youngson, Linear Functional Analysis, Springer, New York, 2013.
59.
Y. Saad, Iterative Methods for Sparse Linear Systems, SIAM, Philadelphia, 2003.
60.
X. Song and B. Yu, A two-phase strategy for control constrained elliptic optimal control problems, Numer. Linear Algebra Appl., (2018), e2138.
61.
G. Stadler, Elliptic optimal control problems with $L^1$-control cost and applications for the placement of control devices, Comput. Optim. Appl., 44 (2009), pp. 159--181.
62.
F. Tröltzsch, Optimal Control of Partial Differential Equations: Theory, Methods, and Applications, Grad. Stud. Math. 112, AMS, Providence, RI, 2010.
63.
U. Trottenberg, C. Oosterlee, and A. Schüller, Multigrid, Elsevier, Amsterdam, 1987.
64.
P. Tseng, Applications of a splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim., 29 (1991), pp. 119--138.
65.
M. Ulbrich, Semismooth Newton methods for operator equations in function spaces, SIAM J. Optim., 13 (2002), pp. 805--841.
66.
H. A. Van der Vorst, Iterative Krylov Methods for Large Linear Systems, Cambridge Monogr. Appl. Comput. Math. 13, Cambridge University Press, Cambridge, UK, 2003.
67.
C. Wagner, Introduction to Algebraic Multigrid, Course notes, University of Heidelberg, 1999.
68.
X. Wang, J. J. Ye, X. Yuan, S. Zeng, and J. Zhang, Perturbation Techniques for Convergence Analysis of Proximal Gradient Method and Other First-Order Algorithms via Variational Analysis, preprint, arXiv:1810.10051, 2018.
69.
X. Wang and X. Yuan, The linearized alternating direction method of multipliers for Dantzig selector, SIAM J. Sci. Comput., 34 (2012), pp. A2792--A2811.
70.
A. J. Wathen, Realistic eigenvalue bounds for the Galerkin mass matrix, IMA J. Numer. Anal., 7 (1987), pp. 449--457.
71.
J. Xu and L. Zikatanov, Algebraic multigrid methods, Acta Numeri., 26 (2017), pp. 591--721.
72.
J. Yang and X. Yuan, Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization, Math. Comput., 82 (2013), pp. 301--329.
73.
J. J. Ye and X. Y. Ye, Necessary optimality conditions for optimization problems with variational inequality constraints, Math. Oper. Res., 22 (1997), pp. 977--997.
74.
J. J. Ye, X. Yuan, S. Zeng, and J. Zhang, Variational analysis perspective on linear convergence of some first order methods for nonsmooth convex optimization problems, Optimization Online, 2018, Optimization-online.org/DB_HTML/2018/10/6881.html.
75.
X. Zhang, M. Burger, X. Bresson, and S. Osher, Bregmanized nonlocal regularization for deconvolution and sparse reconstruction, SIAM J. Imaging Sci., 3 (2010), pp. 253--276.
Information & Authors
Information
Published In
Copyright
© 2019, Society for Industrial and Applied Mathematics.
History
Submitted: 22 February 2019
Accepted: 25 September 2019
Published online: 12 November 2019
Keywords
MSC codes
Authors
Funding Information
Hong Kong Research Grants Council : 12302318
Funding Information
University of Hong Kong https://doi.org/10.13039/501100003803 : 201807159005
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Transformed primal-dual methods for nonlinear saddle point systemsJournal of Numerical Mathematics, Vol. 0, No. 0 | 15 January 2023
- A least squares approach for saddle point problemsJapan Journal of Industrial and Applied Mathematics, Vol. 40, No. 1 | 10 April 2022
- On Convergence of the Arrow–Hurwicz Method for Saddle Point ProblemsJournal of Mathematical Imaging and Vision, Vol. 64, No. 6 | 29 April 2022
- A conjugate gradient method for distributed optimal control problems with nonhomogeneous Helmholtz equationApplied Mathematics and Computation, Vol. 402 | 1 Aug 2021