The Splitting Algorithms by Ryu, by Malitsky–Tam, and by Campoy Applied to Normal Cones of Linear Subspaces Converge Strongly to the Projection onto the Intersection

Abstract.

Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that the resolvents of the operators are available, this problem can be tackled with the Douglas–Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. A similar reduction was achieved recently by Campoy through a clever reformulation originally proposed by Kruger. All three splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. To illustrate our results, we also perform numerical experiments.

Keywords

  1. best approximation
  2. Campoy splitting
  3. Hilbert space
  4. intersection of subspaces
  5. linear convergence
  6. Malitsky–Tam splitting
  7. maximally monotone operator
  8. nonexpansive mapping
  9. resolvent
  10. Ryu splitting

MSC codes

  1. 41A50
  2. 49M27
  3. 65K05
  4. 47H05
  5. 15A10
  6. 47H09
  7. 49M37
  8. 90C25

Get full access to this article

View all available purchase options and get full access to this article.

Acknowledgments.

The authors thank Alex Kruger for making us aware of his [22] and [23], which are highly relevant for Campoy splitting. We also thank the editor and the anonymous reviewers for their pertinent and constructive comments.

References

1.
W. N. Anderson and R. J. Duffin, Series and parallel addition of matrices, J. Math. Anal. Appl., 26 (1969), pp. 576–594, https://doi.org/10.1016/0022-247X(69)90200-5.
2.
F. J. Aragón-Artacho, R. Campoy, and M. K. Tam, Strengthened splitting methods for computing resolvents, Comput. Optim. Appl., 80 (2021), pp. 549–585, https://doi.org/10.1007/s10589-021-00291-6.
3.
C. Badea, S. Grivaux, and V. Müller, The rate of convergence in the method of alternating projections, St. Petersburg Math. J., 23 (2011), pp. 413–434, https://doi.org/10.1090/S1061-0022-2012-01202-1.
4.
J. B. Baillon, Quelques propriétés de convergence asymptotique pour les contractions impaires, C. R. Acad. Sci., 238 (1976), pp. A587–A590.
5.
J. B. Baillon, R. E. Bruck, and S. Reich, On the asymptotic behavior of nonexpansive mappings and semigroups in Banach spaces, Houston J. Math., 4 (1978), pp. 1–9.
6.
H. H. Bauschke, J. Y. Bello Cruz, T. T. A. Nghia, H. M. Phan, and X. Wang, The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle, J. Approx. Theory, 185 (2014), pp. 63–79, https://doi.org/10.1016/j.jat.2014.06.002.
7.
H. H. Bauschke, J. Y. Bello Cruz, T. T. A. Nghia, H. M. Phan, and X. Wang, Optimal rates of linear convergence of relaxed alternating projections and generalized Douglas-Rachford methods for two subspaces, Numer. Algorithms, 73 (2016), pp. 33–76, https://doi.org/10.1007/s11075-015-0085-4.
8.
H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2nd ed., Springer, New York, 2017, https://doi.org/10.1007/978-3-319-48311-5.
9.
H. H. Bauschke, F. Deutsch, H. Hundal, and S. H. Park, Accelerating the convergence of the method of alternating projections, Trans. Amer. Math. Soc., 355 (2003), pp. 3433–3461, https://doi.org/10.1090/S0002-9947-03-03136-2.
10.
H. H. Bauschke, B. Lukens, and W. M. Moursi, Affine nonexpansive operators, Attouch-Théra duality, and the Douglas-Rachford algorithm, Set-Valued Var. Anal., 25 (2017), pp. 481–505, https://doi.org/10.1007/s11228-016-0399-y.
11.
H. H. Bauschke and W. M. Moursi, The Douglas-Rachford algorithm for two (not necessarily intersecting) affine subspaces, SIAM J. Optim., 26 (2016), pp. 968–985, https://doi.org/10.1137/15M1016989.
12.
J. Bezanson, A. Edelman, S. Karpinski, and V. B. Shah, Julia: A fresh approach to numerical computing, SIAM Rev., 59 (2017), pp. 65–98, https://doi.org/10.1137/141000671.
13.
L. M. Briceño-Arias, Resolvent Splitting with Minimal Lifting for Composite Monotone Inclusions, https://arxiv.org/abs/2111.09757v2, 2021.
14.
R. E. Bruck and S. Reich, Nonexpansive projections and resolvents of accretive operators in Banach spaces, Houston J. Math., 3 (1977), pp. 459–470.
15.
R. Campoy, A product space reformulation with reduced dimension for splitting algorithms, Comput. Optim. Appl., 83 (2022), pp. 319–348, https://doi.org/10.1007/s10589-022-00395-7.
16.
P. L. Combettes, Iterative construction of the resolvent of a sum of maximal monotone operators, J. Convex Anal., 16 (2009), pp. 727–748, https://www.heldermann.de/JCA/JCA16/JCA163/jca16044.htm.
17.
F. Deutsch, Best Approximation in Inner Product Spaces, Springer, New York, 2001, https://doi.org/10.1007/978-1-4684-9298-9.
18.
J. Douglas and H. H. Rachford, On the numerical solution of heat conduction problems in two and three space variables, Trans. Amer. Math. Soc., 82 (1956), pp. 421–439, https://doi.org/10.1090/S0002-9947-1956-0084194-4.
19.
M. Goldberg and G. Zwas, On matrices having equal spectral radius and spectral norm, Linear Algebra Appl., 8 (1974), pp. 427–434, https://doi.org/10.1016/0024-3795(74)90076-7.
20.
C. W. Groetsch, Generalized Inverses of Linear Operators, Marcel Dekker, New York, 1977.
21.
S. Kayalar and H. L. Weinert, Error bounds for the method of alternating projections, Math. Control Signals Systems, 1 (1988), pp. 43–59, https://doi.org/10.1007/BF02551235.
22.
A. Y. Kruger, Generalized Differentials of Nonsmooth Functions, 1981, https://asterius.federation.edu.au/akruger/research/papers/1332-81.pdf.
23.
A. Y. Kruger, Generalized differentials of nonsmooth functions, and necessary conditions for an extremum, Siberian Math. J., 26 (1985), pp. 370–379, https://doi.org/10.1007/BF00968624.
24.
P.-L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), pp. 964–979, https://doi.org/10.1137/0716071.
25.
Y. Malitsky and M. K. Tam, Resolvent Splitting for Sums of Monotone Operators with Minimal Lifting, https://arxiv.org/abs/2108.02897v2, 2021.
26.
C. D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, Philadelphia, 2000.
27.
E. K. Ryu, Uniqueness of DRS as the 2 operator resolvent-splitting and impossibility of 3 operator resolvent-splitting, Math. Program. Ser. A, 182 (2020), pp. 233–273, https://doi.org/10.1007/s10107-019-01403-1.

Information & Authors

Information

Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization
Pages: 739 - 765
ISSN (online): 1095-7189

History

Submitted: 8 March 2022
Accepted: 28 November 2022
Published online: 12 June 2023

Keywords

  1. best approximation
  2. Campoy splitting
  3. Hilbert space
  4. intersection of subspaces
  5. linear convergence
  6. Malitsky–Tam splitting
  7. maximally monotone operator
  8. nonexpansive mapping
  9. resolvent
  10. Ryu splitting

MSC codes

  1. 41A50
  2. 49M27
  3. 65K05
  4. 47H05
  5. 15A10
  6. 47H09
  7. 49M37
  8. 90C25

Authors

Affiliations

Heinz H. Bauschke
Department of Mathematics, University of British Columbia, Kelowna V1V 1V7, BC, Canada.
Shambhavi Singh
Department of Mathematics, University of British Columbia, Kelowna V1V 1V7, BC, Canada.
Department of Mathematics, University of British Columbia, Kelowna V1V 1V7, BC, Canada.

Funding Information

Funding: The first and third authors were supported by NSERC Discovery Grants.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

There are no citations for this item

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.