Abstract.

Solving the linear system \(({\mathcal{K}}{\mathcal{K}}^\top )^{-1}{\boldsymbol b}\) is often the major computational burden when a forward-backward evolutionary equation must be solved in a problem, where \(\mathcal{K}\) is the so-called all-at-once matrix of the forward subproblem after space-time discretization. An efficient solver requires a good preconditioner for \({\mathcal{K}}{\mathcal{K}}^\top\). Inspired by the structure of \(\mathcal{K}\), we precondition \({\mathcal{K}}{\mathcal{K}}^\top\) by \({\mathcal{P}}_\alpha{\mathcal{P}}_\alpha^\top\) with \({\mathcal{P}}_\alpha\) being a block \(\alpha\)-circulant matrix constructed by replacing the Toeplitz matrices in \(\mathcal{K}\) by the \(\alpha\)-circulant matrices. By a block Fourier diagonalization of \({\mathcal{P}}_\alpha\), the computation of the preconditioning step \(({\mathcal{P}}_\alpha{\mathcal{P}}_\alpha^\top )^{-1}{\boldsymbol r}\) is parallelizable for all the time steps. We give a spectral analysis for the preconditioned matrix \(({\mathcal{P}}_\alpha{\mathcal{P}}_\alpha^\top )^{-1}({\mathcal{K}}{\mathcal{K}}^\top )\) and prove that for any one-step stable time-integrator the eigenvalues of \(({\mathcal{P}}_\alpha{\mathcal{P}}_\alpha^\top )^{-1}({\mathcal{K}}{\mathcal{K}}^\top )\) spread in a mesh-independent interval \([(1+\sqrt{2}\delta )^{-1}, (1-\sqrt{2}\delta )^{-1}]\) if the parameter \(\alpha\) weakly scales in terms of the number of time steps \(N_t\) as \(\alpha =\delta/\sqrt{N_t}\), where \(\delta \in (0, 1/\sqrt{2})\) is a free constant. Two applications of the proposed preconditioner are illustrated: PDE-constrained optimal control problems and parabolic source identification problems. Numerical results for both problems indicate that spectral analysis predicts the convergence rate of the preconditioned conjugate gradient method very well.

Keywords

  1. forward-backward equations
  2. parallel-in-time (PinT)
  3. all-at-once system
  4. preconditioner
  5. optimal control problems
  6. parabolic source identification
  7. spectral analysis

MSC codes

  1. 65M55
  2. 65M12
  3. 65M15
  4. 65Y05

Get full access to this article

View all available purchase options and get full access to this article.

Acknowledgments.

The authors are very grateful to the anonymous referees for their careful reading of a preliminary version of the manuscript and their valuable suggestions, which greatly improved the quality of this paper.

References

1.
M. Andrle and A. El Badia, On an inverse source problem for the heat equation. Application to a pollution detection problem, II, Inverse Probl. Sci. Eng., 23 (2015), pp. 389–412.
2.
M. Andrle and T. H. Duong, On an inverse source problem for the heat equation. Application to a pollution detection problem, J. Inverse Ill-Posed Probl., 10 (2002), pp. 585–599.
3.
L. C. Brown and T. O. Barnwell, The Enhanced Stream Water Quality Models QUAL2E and QUAL2E-UNCAS: Documentation and User Manual, U.S. Environmental Protection Agency, Athens, GA, 1987.
4.
L. Banjai and D. Peterseim, Parallel multistep methods for linear evolution problems, IMA J. Numer. Anal., 32 (2012), pp. 1217–1240.
5.
A. T. Barker, T. Rees, and M. Stoll, A fast solver for an \(\mathcal{H}_1\) regularized PDE-constrained optimization problem, Commun. Comput. Phys., 19 (2016), pp. 143–167.
6.
L. T. Biegler, O. Ghattas, M. Heinkenschloss, D. Keyes, and B. van Bloemen Waanders, Real-Time PDE-Constrained Optimization, SIAM, Philadelphia, PA, 2007.
7.
D. Bini, G. Latouche, and B. Meini, Numerical Methods for Structured Markov Chains, Oxford University Press, New York, 2005.
8.
A. Borzì and K. Kunisch, A Multigrid Method for Optimal Control of Time-Dependent Reaction Diffusion Processes, Fast Solution of Discretized Optimization Problems, ISNM Internat. Ser. Numer. Math. 138, Birkhäuser, Basel, 2001.
9.
A. Borzì and V. Schulz, Computational Optimization of Systems Governed by Partial Differential Equations, SIAM, Philadelphia, PA, 2012.
10.
D. Braess and P. Peisker, On the numerical solution of the biharmonic equation and the role of squaring matrices for preconditioning, IMA J. Numer. Anal., 6 (1986), pp. 393–404.
11.
F. Danieli and A. J. Wathen, All-at-once solution of linear wave equations, Numer. Linear Algebra Appl., 28 (2021), e2386.
12.
F. Danieli, B. Southworth, and A. J. Wathen, Space-time block preconditioning for incompressible flow, SIAM J. Sci. Comput., 44 (2022), pp. A337–A363.
13.
A. Farcas and D. Lesnic, The boundary-element method for the determination of a heat source dependent on one variable, J. Engrg. Math., 54 (2006), pp. 375–388.
14.
A. Goddard and A. Wathen, A note on parallel preconditioning for all-at-once evolutionary PDEs, Electron. Trans. Numer. Anal., 51 (2019), pp. 135–150.
15.
M. J. Gander and S. L. Wu, Convergence analysis of a periodic-like waveform relaxation method for initial-value problems via the diagonalization technique, Numer. Math., 143 (2019), pp. 489–527.
16.
G. Golub and C. Van Loan, Matrix Computations, Johns Hopkins University Press, Baltimore, MD, 2012.
17.
I. Graham, E. Spence, and E. Vainikko, Domain decomposition preconditioning for high frequency Helmholtz problems with absorption, Math. Comp., 86 (2017), pp. 2089–2127.
18.
A. H. Hasano\(\check{\text{g}}\)lu and V. G. Romanov, Introduction to Inverse Problems for Differential Equations, 2nd ed., Springer, Berlin, 2017.
19.
E. Hairer and G. Wanner, Solving Ordinary Differential Equations II: Stiff and Differential-algebraic Problems, Springer, Berlin, 2000.
20.
R. Herzog, J. W. Pearson, and M. Stoll, Fast iterative solvers for an optimal transport problem, Adv. Comput. Math., 45 (2019), pp. 495–517.
21.
M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich, Optimization with PDE constraints, in Mathematical Modelling: Theory and Applications, Springer, Berlin, 2009.
22.
S. P. Hirshman, K. S. Perumalla, V. E. Lynch, and R. Sanchez, BCYCLIC: A parallel block tridiagonal matrix cyclic solver, J. Comput. Phys., 229 (2010), pp. 6392–6404.
23.
R. Hocking and C. Greif, Optimal complex relaxation parameters in multigrid for complex-shifted linear systems, SIAM J. Matrix Anal. Appl., 42 (2021), pp. 475–502.
24.
K. Kunisch and J. Zou, Iterative choices of regularization parameters in linear inverse problems, Inverse Problems, 14 (1998), pp. 1247–1264.
25.
X.-L. Lin and M. Ng, An all-at-once preconditioner for evolutionary partial differential equations, SIAM J. Sci. Comput., 43 (2021), pp. A2766–A2784.
26.
X. Lin, M. Ng, and H. Sun, Efficient preconditioner of one-sided space fractional diffusion equation, BIT Numer. Math., 58 (2018), pp. 729–748.
27.
X. L. Lin and Z. Zhang, A Parallel-in-Time Preconditioner for the Schur Complement of Parabolic Optimal Control Problems, preprint, arXiv:2109.12524 [math.NA], 2021.
28.
S. Leveque and J. W. Pearson, Fast iterative solver for the optimal control of time-dependent PDEs with Crank–Nicolson discretization in time, Numer. Linear Algebra Appl., 29 (2022), e2419.
29.
J.-L. Lions, Optimal Control of Systems Governed by Partial Differential Equations, Springer, Berlin, 1971.
30.
J. Liu and S. L. Wu, A fast block \(\alpha\)-circulant preconditioner for all-at-once systems from wave equations, SIAM J. Matrix Anal. Appl., 41 (2020), pp. 1912–1943.
31.
J. Liu and J. W. Pearson, Parameter-robust preconditioning for the optimal control of the wave equation, Numer. Algorithms, 83 (2019), pp. 1171–1203.
32.
V. A. Morozov, Methods for Solving Incorrectly Posed Problems, Springer, Berlin, 1984.
33.
E. McDonald, J. Pestana, and A. Wathen, Preconditioning and iterative solution of all-at-once systems for evolutionary partial differential equations, SIAM J. Sci. Comput., 40 (2018), pp. A1012–A1033.
34.
Y. J. Ma, C. Fu, and Y. X. Zhang, Identification of an unknown source depending on both time and space variables by a variational method, Appl. Math. Model., 36 (2012), pp. 5080–5090.
35.
M. Nili Ahmadabadi, M. Arab, and F. M. Maalek Ghaini, The method of fundamental solutions for the inverse space-dependent heat source problem, Eng. Anal. Bound. Elem., 33 (2009), pp. 1231–1235.
36.
J. W. Pearson and A. J. Wathen, A new approximation of the Schur complement in preconditioners for PDE constrained optimization, Numer. Linear Algebra Appl., 19 (2012), pp. 816–829.
37.
J. W. Pearson, M. Stoll, and A. J. Wathen, Regularization-robust preconditioners for time-dependent PDE-constrained optimization problems, SIAM J. Matrix Anal. Appl., 33 (2012), pp. 1126–1152.
38.
J. W. Pearson, Fast iterative solvers for large matrix systems arising from time-dependent Stokes control problems, Appl. Numer. Math., 108 (2016), pp. 87–101.
39.
D. E. Reevet and M. Spivack, Determination of a source term in the linear diffusion equation, Inverse Problems, 10 (1994), pp. 1335–3344.
40.
S. Saitoh, V. Tuan, and M. Yamamoto, Reverse convolution inequalities and applications to inverse heat source problems, J. Inequal. Pure Appl. Math., 3 (2003), 80.
41.
E. G. Savateev, On problems of determining the source function in a parabolic equation, J. Inverse Ill-Posed Probl., 3 (1995), pp. 83–102.
42.
M. Stoll, One-shot solution of a time-dependent time-periodic PDE-constrained optimization problem, IMA J. Numer. Anal., 34 (2014), pp. 1554–1577.
43.
M. Stoll and A. Wathen, All-at-once solution of time-dependent Stokes control, J. Comput. Phys., 232 (2013), pp. 498–515.
44.
A. Wathen, Some comments on preconditioning for normal equations and least squares, SIAM Rev., 64 (2022), pp. 640–649.
45.
S. L. Wu and T. Zhou, Parallel implementation for the two-stage SDIRK methods via diagonalization, J. Comput. Phys., 428 (2021), 110076.
46.
S.-L. Wu, H. Zhang, and T. Zhou, Solving time-periodic fractional diffusion equations via diagonalization technique and multigrid, Numer. Linear Algebra Appl., 25 (2018), e2178.
47.
S.-L. Wu, T. Zhou, and Z. Zhou, A uniform spectral analysis for preconditioned all-at-once system from first-order and second-order evolutionary problems, SIAM J. Matrix Anal. Appl., 43 (2022), pp. 1331–1353.
48.
S. N. Wu and Z. Zhou, A parallel-in-time algorithm for high-order BDF methods for diffusion and subdiffusion equations, SIAM J. Sci. Comput., 43 (2021), pp. A3627–A3656.
49.
J. Xie and J. Zou, An improved model function method for choosing regularization parameters in linear inverse problems, Inverse Problems, 18 (2002), pp. 631–643.

Information & Authors

Information

Published In

cover image SIAM Journal on Matrix Analysis and Applications
SIAM Journal on Matrix Analysis and Applications
Pages: 1771 - 1798
ISSN (online): 1095-7162

History

Submitted: 17 August 2022
Accepted: 15 August 2023
Published online: 30 November 2023

Keywords

  1. forward-backward equations
  2. parallel-in-time (PinT)
  3. all-at-once system
  4. preconditioner
  5. optimal control problems
  6. parabolic source identification
  7. spectral analysis

MSC codes

  1. 65M55
  2. 65M12
  3. 65M15
  4. 65Y05

Authors

Affiliations

Shu-Lin Wu
School of Mathematics and Statistics, Northeast Normal University, Changchun, 130024, China.
Zhiyong Wang
School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, 611731, China.
LSEC, Institute of Computational Mathematics and Scientific/Engineering Computing, AMSS, Chinese Academy of Sciences, Beijing, 100190, China.

Funding Information

Funding: The work of the first author was supported by the NSFC (12171080, 12292982) and by the Natural Science Foundation of Jilin Province (JC010284408). The work of the third author was partially supported by the NSF of China (under grant 12288201) and the Youth Innovation Promotion Association of CAS.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Full Text

View Full Text

Figures

Tables

Media

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media