In this work, we propose a simple modification of the forward-backward splitting method for finding a zero in the sum of two monotone operators. Our method converges under the same assumptions as Tseng's forward-backward-forward method, namely, it does not require cocoercivity of the single-valued operator. Moreover, each iteration only uses one forward evaluation rather than two as is the case for Tseng's method. Variants of the method incorporating a linesearch, relaxation and inertia, or a structured three operator inclusion are also discussed.


  1. forward-backward algorithm
  2. Tseng's method
  3. operator splitting

MSC codes

  1. 49M29
  2. 90C25
  3. 47H05
  4. 47J20
  5. 65K15

Get full access to this article

View all available purchase options and get full access to this article.


F. Alvarez and H. Attouch, An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping, Set-Valued Anal., 9 (2001), pp. 3--11.
A. Antipin, On a method for convex programs using a symmetrical modification of the Lagrange function, Ekon. Mat. Metody, 12 (1976), pp. 1164--1173.
K. J. Arrow, L. Hurwicz, and H. Uzawa, Studies in Linear and Non-Linear Programming, Stanford Mathematical Studies in the Social Sciences, Stanford University Press, Stanford, CA, 1958.
H. Attouch and A. Cabot, Convergence of a relaxed inertial forward-backward algorithm for structured monotone inclusions, Appl. Math. Optim., (2019), pp. 1--52.
H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer, New York, 2011.
J. Bello Cruz and R. Díaz Millán, A variant of forward-backward splitting method for the sum of two monotone operators with a new search strategy, Optimization, 64 (2015), pp. 1471--1486.
R. I. Boţ and E. R. Csetnek, An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems, Numer. Algorithms, 71 (2016), pp. 519--540.
R. I. Boţ, E. R. Csetnek, and A. Heinrich, A primal-dual splitting algorithm for finding zeros of sums of maximal monotone operators, SIAM J. Optim., 23 (2013), pp. 2011--2036.
L. M. Bricen͂o-Arias and P. L. Combettes, A monotone+ skew splitting model for composite monotone inclusions in duality, SIAM J. Optim., 21 (2011), pp. 1230--1250.
L. M. Bricen͂o-Arias and D. Davis, Forward-backward-half forward algorithm for solving monotone inclusions, SIAM J. Optim., 28 (2018), pp. 2839--2871.
A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis., 40 (2011), pp. 120--145.
G. H. Chen and R. T. Rockafellar, Convergence rates in forward--backward splitting, SIAM J. Optim., 7 (1997), pp. 421--444.
P. L. Combettes, Quasi-Fejérian analysis of some optimization algorithms, in Inherently Parallel Algortihms in Feasibility and Optimization and Their Applications, Stud. Comput. Math. 8, Elsevier, Amsterdam, 2001, pp. 115--152.
P. L. Combettes and J.-C. Pesquet, Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping, SIAM J. Optim., 25 (2015), pp. 1221--1248.
E. R. Csetnek, Y. Malitsky, and M. K. Tam, Shadow Douglas--Rachford splitting for monotone inclusions, Appl. Math. Optim., 80 (2019), pp. 665--678.
C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng, Training GANs with optimism, in Proceedings of ICLR, 2018.
D. Davis and W. Yin, A three-operator splitting scheme and its optimization applications, Set-Valued Var. Anal., 25 (2017), pp. 829--858.
G. Gidel, H. Berard, G. Vignoud, P. Vincent, and S. Lacoste-Julien, A variational inequality perspective on generative adversarial networks, in Proceedings of ICLR, 2019.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, Generative adversarial nets, in Advances in Neural Information Processing Systems, 2014, pp. 2672--2680.
E. Y. Hamedani and N. S. Aybat, A Primal-Dual Algorithm for General Convex-Concave Saddle Point Problems, arXiv:1803.01401, 2018.
Y.-G. Hsieh, F., J. Malick, and P. Mertikopoulos, On the convergence of single-call stochastic extra-gradient methods, in Proceedings of NeurIPS, 2019, pp. 6936--6946.
P. R. Johnstone and J. Eckstein, Projective Splitting with Forward Steps: Asynchronous and Block-Iterative Operator Splitting, arXiv:1803.07043, 2018.
G. M. Korpelevich, The extragradient method for finding saddle points and other problems, Ekon. Mat. Metody, 12 (1976), pp. 747--756.
P. L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), pp. 964--979.
D. Lorenz and T. Pock, An inertial forward-backward algorithm for monotone inclusions, J. Math. Imaging Vis., 51 (2015), pp. 311--325.
Y. Malitsky, Reflected projected gradient method for solving monotone variational inequalities, SIAM J. Optim., 25 (2015), pp. 502--520.
Y. Malitsky, Golden ratio algorithms for variational inequalities, Math. Program., (2019), https://doi.org/10.1007/s10107-019-01416-w.
P. Mertikopoulos, B. Lecouat, H. Zenati, C.-S. Foo, V. Chandrasekhar, and G. Piliouras, Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile, in Proceedings of ICLR, 2019.
K. Mishchenko, D. Kovalev, E. Shulgin, P. Richtárik, and Y. Malitsky, Revisiting Stochastic Extragradient, arXiv:1905.11373, 2019.
A. Moudafi and M. Oliny, Convergence of a splitting inertial proximal method for monotone operators, J. Comput. Appl. Math., 155 (2003), pp. 447--454.
Y. Nesterov, Gradient methods for minimizing composite functions, Math. Program., 140 (2013), pp. 125--161.
Y. E. Nesterov, A method for solving the convex programming problem with convergence rate ${O}(1/k^2)$, Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543--547.
L. D. Popov, A modification of the Arrow--Hurwicz method for finding saddle points, Math. Notes, 28 (1980), pp. 845--848.
J. Rieger and M. K. Tam, Backward-forward-reflected-backward splitting for three operator monotone inclusions, Appl. Math. Comput., 381 (2020), 125248, https://doi.org/10.1016/j.amc.2020.125248.
H. Robbins and D. Siegmund, A convergence theorem for non negative almost supermartingales and some applications, in Optimizing Methods in Statistics, Academic Press, New York, 1971, pp. 233--257.
R. T. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), pp. 877--898.
E. K. Ryu and B. C. Vu, Finding the forward-Douglas--Rachford-forward method, J. Optim. Theory. Appl., 184 (2000), pp. 858--876.
E. K. Ryu, K. Yuan, and W. Yin, ODE Analysis of Stochastic Gradient Methods with Optimism and Anchoring for Minimax Problems and GANs, arXiv:1905.10899, 2019.
B. F. Svaiter, On weak convergence of the Douglas--Rachford method, SIAM J. Control Optim., 49 (2011), pp. 280--287.
P. Tseng, A modified forward-backward splitting method for maximal monotone mappings, SIAM J. Control Optim., 38 (2000), pp. 431--446.

Information & Authors


Published In

cover image SIAM Journal on Optimization
SIAM Journal on Optimization
Pages: 1451 - 1472
ISSN (online): 1095-7189


Submitted: 13 August 2018
Accepted: 17 March 2020
Published online: 21 May 2020


  1. forward-backward algorithm
  2. Tseng's method
  3. operator splitting

MSC codes

  1. 49M29
  2. 90C25
  3. 47H05
  4. 47J20
  5. 65K15



Funding Information

Alexander von Humboldt-Stiftung https://doi.org/10.13039/100005156

Funding Information

Australian Research Council https://doi.org/10.13039/501100000923

Funding Information

Deutsche Forschungsgemeinschaft https://doi.org/10.13039/501100001659 : SFB755-A4

Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By







Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.