SIAM Journal on Control and Optimization


Acceleration of Stochastic Approximation by Averaging

Related Databases

Web of Science

You must be logged in with an active subscription to view this.

Article Data

History

Submitted: 30 July 1990
Accepted: 24 June 1991
Published online: 14 July 2006

Publication Data

ISSN (print): 0363-0129
ISSN (online): 1095-7138
CODEN: sjcodc

A new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated. Convergence with probability one is proved for a variety of classical optimization and identification problems. It is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence.

Copyright © 1992 Society for Industrial and Applied Mathematics

Cited by

(2021) A Bayesian stochastic approximation method. Journal of Statistical Planning and Inference 211, 391-401. Crossref
(2020) Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recognition 108, 107482. Crossref
(2020) Semisupervised Multilabel Deep Learning Based Nonintrusive Load Monitoring in Smart Grids. IEEE Transactions on Industrial Informatics 16:11, 6892-6902. Crossref
(2020) Guaranteed in-control control chart performance with cautious parameter learning. Journal of Quality Technology 52:4, 385-403. Crossref
(2020) Deep Reinforcement Learning for Long Term Hydropower Production Scheduling. 2020 International Conference on Smart Energy Systems and Technologies (SEST), 1-6. Crossref
(2020) Estimation of discrete choice models with hybrid stochastic adaptive batch size algorithms. Journal of Choice Modelling, 100226. Crossref
(2020) Mini-batch learning of exponential family finite mixture models. Statistics and Computing 30:4, 731-748. Crossref
(2020) Robust Training of Vector Quantized Bottleneck Models. 2020 International Joint Conference on Neural Networks (IJCNN), 1-7. Crossref
(2020) On a multistage discrete stochastic optimization problem with stochastic constraints and nested sampling. Mathematical Programming 127. Crossref
(2020) Shallow neural networks for fluid flow reconstruction with limited sensors. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 476:2238, 20200097. Crossref
(2020) Algorithm 1007. ACM Transactions on Mathematical Software 46:2, 1-20. Crossref
(2020) On the rates of convergence of parallelized averaged stochastic gradient algorithms. Statistics 54:3, 618-635. Crossref
(2020) A new convergent hybrid learning algorithm for two-stage stochastic programs. European Journal of Operational Research 283:1, 33-46. Crossref
(2020) Topology Optimization With Many Right-Hand Sides Using Mirror Descent Stochastic Approximation—Reduction From Many to a Single Sample. Journal of Applied Mechanics 87:5. Crossref
(2020) Nesterov-aided stochastic gradient methods using Laplace approximation for Bayesian design optimization. Computer Methods in Applied Mechanics and Engineering 363, 112909. Crossref
(2020) Multidimensional Gains for Stochastic Approximation. IEEE Transactions on Neural Networks and Learning Systems 31:5, 1602-1615. Crossref
(2020) Optimize TSK Fuzzy Systems for Regression Problems: Minibatch Gradient Descent With Regularization, DropRule, and AdaBound (MBGD-RDA). IEEE Transactions on Fuzzy Systems 28:5, 1003-1015. Crossref
(2020) A Streaming On-Device End-To-End Model Surpassing Server-Side Conventional Model Quality and Latency. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6059-6063. Crossref
(2020) Active Learning with Unsupervised Ensembles of Classifiers. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 3967-3971. Crossref
(2020) Mixup-breakdown: A Consistency Training Method for Improving Generalization of Speech Separation Models. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6374-6378. Crossref
(2020) Inexact stochastic mirror descent for two-stage nonlinear stochastic programs. Mathematical Programming 22. Crossref
(2020) Medi-Care AI: Predicting medications from billing codes via robust recurrent neural networks. Neural Networks 124, 109-116. Crossref
(2020) Advising reinforcement learning toward scaling agents in continuous control environments with sparse rewards. Engineering Applications of Artificial Intelligence 90, 103515. Crossref
(2020) Pixelated Semantic Colorization. International Journal of Computer Vision 128:4, 818-834. Crossref
(2020) Dynamic stochastic approximation for multi-stage stochastic optimization. Mathematical Programming 40. Crossref
(2020) Adaptive importance sampling and control variates. Journal of Mathematical Analysis and Applications 483:1, 123608. Crossref
(2020) Prediction and analysis of net ecosystem carbon exchange based on gradient boosting regression and random forest. Applied Energy 262, 114566. Crossref
(2020) Fungi Recognition: A Practical Use Case. 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), 2305-2313. Crossref
(2020) Algorithms for stochastic optimization with function or expectation constraints. Computational Optimization and Applications 16. Crossref
(2020) Modeling the dynamics of PDE systems with physics-constrained deep auto-regressive networks. Journal of Computational Physics 403, 109056. Crossref
(2020) fMRI reveals language-specific predictive coding during naturalistic sentence comprehension. Neuropsychologia 138, 107307. Crossref
(2020) BRANCHING PARTICLE PRICERS WITH HESTON EXAMPLES. International Journal of Theoretical and Applied Finance 23:01, 2050003. Crossref
2020. Stochastic Approximation with Applications. Encyclopedia of Systems and Control, 1-7. Crossref
2020. Optimization Methods. Deep Learning in Mining of Visual Content, 21-33. Crossref
, , and . (2020) An Efficient Stochastic Newton Algorithm for Parameter Estimation in Logistic Regressions. SIAM Journal on Control and Optimization 58:1, 348-367. Abstract | PDF (489 KB) 
, , , and . (2020) Robust Accelerated Gradient Methods for Smooth Strongly Convex Functions. SIAM Journal on Optimization 30:1, 717-751. Abstract | PDF (892 KB) 
(2020) Assessment of model risk due to the use of an inappropriate parameter estimator. Cogent Economics & Finance 8:1. Crossref
(2020) The Asynchronous Training Algorithm Based on Sampling and Mean Fusion for Distributed RNN. IEEE Access 8, 62439-62447. Crossref
2020. Advanced Optimization Solutions. Linear Algebra and Optimization for Machine Learning, 205-253. Crossref
2020. Stochastic Convex Optimization. First-order and Stochastic Optimization Methods for Machine Learning, 113-220. Crossref
(2020) An Efficient v-Minimum Absolute Deviation Distribution Regression Machine. IEEE Access 8, 85533-85551. Crossref
and . (2020) On the Adaptivity of Stochastic Gradient-Based Optimization. SIAM Journal on Optimization 30:2, 1473-1500. Abstract | PDF (516 KB) 
, , , and . (2020) Uncertainty Quantification for Stochastic Approximation Limits Using Chaos Expansion. SIAM/ASA Journal on Uncertainty Quantification 8:3, 1061-1089. Abstract | PDF (975 KB) 
(2020) A Continuous-Time Analysis of Distributed Stochastic Gradient. Neural Computation 32:1, 36-96. Crossref
2020. A Committee of Convolutional Neural Networks for Image Classification in the Concurrent Presence of Feature and Label Noise. Parallel Problem Solving from Nature – PPSN XVI, 498-511. Crossref
2020. DLchain: Blockchain with Deep Learning as Proof-of-Useful-Work. Services – SERVICES 2020, 43-60. Crossref
2020. Parameter Tuning Using Adaptive Moment Estimation in Deep Learning Neural Networks. Computational Science and Its Applications – ICCSA 2020, 261-272. Crossref
(2019) Unsupervised Speech Representation Learning Using WaveNet Autoencoders. IEEE/ACM Transactions on Audio, Speech, and Language Processing 27:12, 2041-2053. Crossref
(2019) Entropy-SGD: biasing gradient descent into wide valleys. Journal of Statistical Mechanics: Theory and Experiment 2019:12, 124018. Crossref
(2019) Scalable statistical inference for averaged implicit stochastic gradient descent. Scandinavian Journal of Statistics 46:4, 987-1002. Crossref
(2019) Online estimation of the asymptotic variance for averaged stochastic gradient algorithms. Journal of Statistical Planning and Inference 203, 1-19. Crossref
(2019) An Adaptive and Incremental Approach to Quantile Estimation. 2019 IEEE 58th Conference on Decision and Control (CDC), 6025-6031. Crossref
(2019) Quasi-Stochastic Approximation and Off-Policy Reinforcement Learning. 2019 IEEE 58th Conference on Decision and Control (CDC), 5244-5251. Crossref
(2019) Massively Scalable Parallel KMeans on the HPCC Systems Platform. 2019 4th International Conference on Computational Systems and Information Technology for Sustainable Solution (CSITSS), 1-8. Crossref
(2019) Solving stochastic frequency-based assignment to transit networks with pre-trip/en-route path choice. EURO Journal on Transportation and Logistics 8:5, 661-681. Crossref
(2019) Optimal stochastic extragradient schemes for pseudomonotone stochastic variational inequality problems and their variants. Computational Optimization and Applications 74:3, 779-820. Crossref
(2019) Random Minibatch Subgradient Algorithms for Convex Problems with Functional Constraints. Applied Mathematics & Optimization 80:3, 801-833. Crossref
(2019) The importance of better models in stochastic optimization. Proceedings of the National Academy of Sciences 116:46, 22924-22930. Crossref
(2019) Unlexicalized Transition-based Discontinuous Constituency Parsing. Transactions of the Association for Computational Linguistics 7, 73-89. Crossref
(2019) Why random reshuffling beats stochastic gradient descent. Mathematical Programming 58. Crossref
(2019) Convergence of Stochastic Proximal Gradient Algorithm. Applied Mathematics & Optimization 18. Crossref
(2019) THz field detection in graphene using deep neural networks. Applied Physics Letters 115:16, 161106. Crossref
(2019) Self-Tuned Mirror Descent Schemes for Smooth and Nonsmooth High-Dimensional Stochastic Optimization. IEEE Transactions on Automatic Control 64:10, 4377-4384. Crossref
(2019) Autonomous Car Racing in Simulation Environment Using Deep Reinforcement Learning. 2019 Innovations in Intelligent Systems and Applications Conference (ASYU), 1-6. Crossref
(2019) Dual Student: Breaking the Limits of the Teacher in Semi-Supervised Learning. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 6727-6735. Crossref
(2019) Demystifying Parallel and Distributed Deep Learning. ACM Computing Surveys 52:4, 1-43. Crossref
(2019) On Matrix Momentum Stochastic Approximation and Applications to Q-learning. 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 749-756. Crossref
(2019) Density Stochastic Approximation of Initial Buffer Decision Problem in Continuous Production Lines. 2019 International Conference on Industrial Engineering and Systems Management (IESM), 1-6. Crossref
(2019) Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization. IEEE Transactions on Signal Processing 67:16, 4189-4203. Crossref
(2019) Multi-proportion channel ensemble model for retinal vessel segmentation. Computers in Biology and Medicine 111, 103352. Crossref
(2019) A Time Averaging Algorithm for Load Balancing in Distributed Power Dispatch. 2019 IEEE Conference on Control Technology and Applications (CCTA), 643-648. Crossref
(2019) Taming Hyper-parameters in Deep Learning Systems. ACM SIGOPS Operating Systems Review 53:1, 52-58. Crossref
(2019) Unsupervised domain adaptation for medical imaging segmentation with self-ensembling. NeuroImage 194, 1-11. Crossref
(2019) Asymptotic Efficiency of Distributed Random Sampling Algorithm. 2019 Chinese Control Conference (CCC), 1625-1630. Crossref
(2019) Crossbow. Proceedings of the VLDB Endowment 12:11, 1399-1412. Crossref
(2019) Long-term temporal averaging for stochastic optimization of deep neural networks. Neural Computing and Applications 31:6, 1733-1745. Crossref
(2019) Parallel Simultaneous Perturbation Optimization. Asia-Pacific Journal of Operational Research 36:03, 1950009. Crossref
(2019) Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9118-9127. Crossref
(2019) Stochastic Gradient Descent and Its Variants in Machine Learning. Journal of the Indian Institute of Science 99:2, 201-213. Crossref
(2019) Urban flood mapping with an active self-learning convolutional neural network based on TerraSAR-X intensity and interferometric coherence. ISPRS Journal of Photogrammetry and Remote Sensing 152, 178-191. Crossref
(2019) A stochastic gradient approach for the reliability maximization of passively controlled structures. Engineering Structures 186, 1-12. Crossref
(2019) Investigations of Real-time Gaussian Fftnet and Parallel Wavenet Neural Vocoders with Simple Acoustic Features. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7020-7024. Crossref
(2019) Data Efficient Voice Cloning for Neural Singing Synthesis. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6840-6844. Crossref
(2019) Context-aware Neural-based Dialog Act Classification on Automatically Generated Transcriptions. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 7265-7269. Crossref
(2019) Probabilistic Bisection Converges Almost as Quickly as Stochastic Approximation. Mathematics of Operations Research 44:2, 651-667. Crossref
(2019) Variance Suppression: Balanced Training Process in Deep Learning. Journal of Physics: Conference Series 1207, 012013. Crossref
(2019) Statistical inference for the population landscape via moment-adjusted stochastic gradients. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 81:2, 431-456. Crossref
(2019) Estimation bias and bias correction in reduced rank autoregressions. Econometric Reviews 38:3, 332-349. Crossref
(2019) On variance reduction for stochastic smooth convex optimization with multiplicative noise. Mathematical Programming 174:1-2, 253-292. Crossref
(2019) A Concentration Bound for Stochastic Approximation via Alekseev’s Formula. Stochastic Systems 9:1, 1-26. Crossref
(2019) Deep Learning–Based Multimedia Analytics. ACM Transactions on Multimedia Computing, Communications, and Applications 15:1s, 1-26. Crossref
(2019) Deep Learning at Scale. 2019 27th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), 124-131. Crossref
(2019) An Asymptotically Optimal Set Approach for Simulation Optimization. INFORMS Journal on Computing 31:1, 21-39. Crossref
(2019) A randomized method for handling a difficult function in a convex optimization problem, motivated by probabilistic programming. Annals of Operations Research 13. Crossref
(2019) Noisy Hamiltonian Monte Carlo for Doubly Intractable Distributions. Journal of Computational and Graphical Statistics 28:1, 220-232. Crossref
, , , and . (2019) Variance-Based Extragradient Methods with Line Search for Stochastic Variational Inequalities. SIAM Journal on Optimization 29:1, 175-206. Abstract | PDF (797 KB) 
, , and . (2019) Convergence Rate of Incremental Gradient and Incremental Newton Methods. SIAM Journal on Optimization 29:4, 2542-2565. Abstract | PDF (508 KB) 
and . (2019) An Uncertainty-Weighted Asynchronous ADMM Method for Parallel PDE Parameter Estimation. SIAM Journal on Scientific Computing 41:5, S129-S148. Abstract | PDF (1400 KB) 
and . (2019) Projected Stochastic Gradients for Convex Constrained Problems in Hilbert Spaces. SIAM Journal on Optimization 29:3, 2079-2099. Abstract | PDF (2090 KB) 
and . (2019) Stochastic (Approximate) Proximal Point Methods: Convergence, Optimality, and Adaptivity. SIAM Journal on Optimization 29:3, 2257-2290. Abstract | PDF (866 KB) 
2019. Scalable Nonlinear AUC Maximization Methods. Machine Learning and Knowledge Discovery in Databases, 292-307. Crossref
2019. MASAGA: A Linearly-Convergent Stochastic First-Order Method for Optimization on Manifolds. Machine Learning and Knowledge Discovery in Databases, 344-359. Crossref
(2019) Learning Algorithm for Fractional Dynamical Systems with Autocorrelated Errors-in-Variables. Procedia Computer Science 154, 311-318. Crossref
(2019) Utility-Optimal Wireless Routing in the Presence of Heavy Tails. IEEE Transactions on Vehicular Technology 68:1, 780-796. Crossref
(2019) On the regularizing property of stochastic gradient descent. Inverse Problems 35:1, 015004. Crossref
(2019) L p and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective. ESAIM: Probability and Statistics 23, 841-873. Crossref
(2019) Efficient Adaptive Speech Reception Threshold Measurements Using Stochastic Approximation Algorithms. Trends in Hearing 23, 233121652091919. Crossref
(2019) Estimation and Inference in Adaptive Learning Models with Slowly Decreasing Gains. SSRN Electronic Journal. Crossref
(2019) Zap Q-Learning - A User's Guide. 2019 Fifth Indian Control Conference (ICC), 10-15. Crossref
(2018) On Stochastic Mirror-prox Algorithms for Stochastic Cartesian Variational Inequalities: Randomized Block Coordinate and Optimal Averaging Schemes. Set-Valued and Variational Analysis 26:4, 789-819. Crossref
(2018) Fast Maximum Likelihood Estimation via Equilibrium Expectation for Large Network Data. Scientific Reports 8:1. Crossref
(2018) Online Successive Convex Approximation for Two-Stage Stochastic Nonconvex Optimization. IEEE Transactions on Signal Processing 66:22, 5941-5955. Crossref
(2018) Cross Entropy of Neural Language Models at Infinity—A New Bound of the Entropy Rate. Entropy 20:11, 839. Crossref
(2018) Two-stage linear decision rules for multi-stage stochastic programming. Mathematical Programming 99. Crossref
2018. Stochastic Gradient Descent: Recent Trends. Recent Advances in Optimization and Modeling of Contemporary Problems, 193-220. Crossref
(2018) Adaptive Sampling of Large Deviations. Journal of Statistical Physics 172:6, 1525-1544. Crossref
(2018) Generalization properties of doubly stochastic learning algorithms. Journal of Complexity 47, 42-61. Crossref
(2018) On the Information-Adaptive Variants of the ADMM: An Iteration Complexity Perspective. Journal of Scientific Computing 76:1, 327-363. Crossref
2018. Robust Constant Modulus Algorithms. Simplified Robust Adaptive Detection and Beamforming for Wireless Communications, 225-261. Crossref
(2018) An incremental off-policy search in a model-free Markov decision process using a single sample path. Machine Learning 107:6, 969-1011. Crossref
(2018) Stochastic optimization using a trust-region method and random models. Mathematical Programming 169:2, 447-487. Crossref
(2018) Probability Maximization with Random Linear Inequalities: Alternative Formulations and Stochastic Approximation Schemes. 2018 Annual American Control Conference (ACC), 1396-1401. Crossref
(2018) Fast GPU Implementation of a Scan-Specific Deep Learning Reconstruction for Accelerated Magnetic Resonance Imaging. 2018 IEEE International Conference on Electro/Information Technology (EIT), 0399-0403. Crossref
(2018) Convergence and Efficiency of Adaptive Importance Sampling Techniques with Partial Biasing. Journal of Statistical Physics 171:2, 220-268. Crossref
(2018) Minimum deviation distribution machine for large scale regression. Knowledge-Based Systems 146, 167-180. Crossref
(2018) Dynamic Resource Caching in the IoT Application Layer for Smart Cities. IEEE Internet of Things Journal 5:2, 606-613. Crossref
(2018) Learning Trans-Dimensional Random Fields with Applications to Language Modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence 40:4, 876-890. Crossref
(2018) Online Principal Component Analysis in High Dimension: Which Algorithm to Choose?. International Statistical Review 86:1, 29-50. Crossref
(2018) Lexico-Acoustic Neural-Based Models for Dialog Act Classification. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6194-6198. Crossref
(2018) Calculating quantiles of noisy distribution functions using local linear regressions. Computational Statistics 33:1, 487-501. Crossref
(2018) Distributed Computation of Equilibria in Misspecified Convex Stochastic Nash Games. IEEE Transactions on Automatic Control 63:2, 360-371. Crossref
(2018) EXPLICIT HESTON SOLUTIONS AND STOCHASTIC APPROXIMATION FOR PATH-DEPENDENT OPTION PRICING. International Journal of Theoretical and Applied Finance 21:01, 1850006. Crossref
(2018) Online cross-validation-based ensemble learning. Statistics in Medicine 37:2, 249-260. Crossref
(2018) Sequential linear regression with online standardized data. PLOS ONE 13:1, e0191186. Crossref
(2018) Minimax Optimal Procedures for Locally Private Estimation. Journal of the American Statistical Association 113:521, 182-201. Crossref
, , , and . (2018) On Sampling Rates in Simulation-Based Recursions. SIAM Journal on Optimization 28:1, 45-73. Abstract | PDF (520 KB) 
, , and . (2018) ASTRO-DF: A Class of Adaptive Sampling Trust-Region Algorithms for Derivative-Free Stochastic Optimization. SIAM Journal on Optimization 28:4, 3145-3176. Abstract | PDF (732 KB) 
, , and . (2018) Optimization Methods for Large-Scale Machine Learning. SIAM Review 60:2, 223-311. Abstract | PDF (1735 KB) 
, , and . (2018) A Central Limit Theorem and Hypotheses Testing for Risk-averse Stochastic Programs. SIAM Journal on Optimization 28:2, 1337-1366. Abstract | PDF (751 KB) 
. (2018) Optimizing Adaptive Importance Sampling by Stochastic Approximation. SIAM Journal on Scientific Computing 40:4, A2774-A2800. Abstract | PDF (1837 KB) 
2018. Adaptive Ensemble Probabilistic Matrix Approximation for Recommendation. Pattern Recognition and Computer Vision, 328-339. Crossref
2018. Randomization in Robustness, Estimation, and Optimization. Uncertainty in Complex Networked Systems, 181-208. Crossref
2018. Research Questions in Data Science. Targeted Learning in Data Science, 3-14. Crossref
2018. Data-Adaptive Estimation in Cluster Randomized Trials. Targeted Learning in Data Science, 195-215. Crossref
2018. Averaged Stochastic Optimization for Medical Image Registration Based on Variance Reduction. Biomedical Image Registration, 69-79. Crossref
2018. Training Deep Neural Networks. Neural Networks and Deep Learning, 105-167. Crossref
2018. Overview of LifeCLEF 2018: A Large-Scale Evaluation of Species Identification and Recommendation Algorithms in the Era of AI. Experimental IR Meets Multilinguality, Multimodality, and Interaction, 247-266. Crossref
(2018) Online average-based system modelling method for batch process. Computers & Chemical Engineering 108, 128-138. Crossref
2018. Deep Semi-supervised Segmentation with Weight-Averaged Consistency Targets. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 12-19. Crossref
(2017) A stochastic approximation for finding an element of the core of uncertain cooperative games. 2017 11th Asian Control Conference (ASCC), 2071-2076. Crossref
(2017) History of seeking better solutions, AKA simulation optimization. 2017 Winter Simulation Conference (WSC), 131-157. Crossref
(2017) Application of a second-order stochastic optimization algorithm for fitting stochastic epidemiological models. 2017 Winter Simulation Conference (WSC), 2194-2206. Crossref
(2017) Learning Simpler Language Models with the Differential State Framework. Neural Computation 29:12, 3327-3352. Crossref
(2017) Constrained Stochastic Gradient Descent: The Good Practice. 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), 1-8. Crossref
(2017) Attention-Based Extraction of Structured Information from Street View Imagery. 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), 844-850. Crossref
(2017) Large margin distribution machine recursive feature elimination. 2017 4th International Conference on Systems and Informatics (ICSAI), 1518-1523. Crossref
(2017) On Projected Stochastic Gradient Descent Algorithm with Weighted Averaging for Least Squares Regression. IEEE Transactions on Automatic Control 62:11, 5974-5981. Crossref
(2017) A Robbins-Monro Algorithm for Non-Parametric Estimation of NAR Process with Markov Switching: Consistency. Journal of Time Series Analysis 38:6, 809-837. Crossref
(2017) Mean estimation from adaptive one-bit measurements. 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 1000-1007. Crossref
(2017) A Numerical Approach to Stability of Multiclass Queueing Networks. IEEE Transactions on Automatic Control 62:10, 5478-5484. Crossref
(2017) A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation. IEEE Transactions on Signal Processing 65:18, 4770-4783. Crossref
(2017) Individual confidence intervals for solutions to expected value formulations of stochastic variational inequalities. Mathematical Programming 165:1, 151-196. Crossref
(2017) On smoothing, regularization, and averaging in stochastic approximation methods for stochastic variational inequality problems. Mathematical Programming 165:1, 391-431. Crossref
(2017) Fast estimation of the median covariation matrix with application to online robust principal components analysis. TEST 26:3, 461-480. Crossref
(2017) Nexus: Bringing Efficient and Scalable Training to Deep Learning Frameworks. 2017 IEEE 25th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), 12-21. Crossref
(2017) Accelerating the rate of convergence for LMS-like on-line identification and adaptation algorithms. Part 1: Basic ideas. 2017 22nd International Conference on Methods and Models in Automation and Robotics (MMAR), 347-350. Crossref
(2017) Online Hybrid Likelihood Based Modulation Classification Using Multiple Sensors. IEEE Transactions on Wireless Communications 16:8, 4984-5000. Crossref
(2017) Regularized non-convex image reconstruction in digital holographic microscopy. Optics Express 25:14, 16491. Crossref
(2017) A Strong Interference Suppressor for Satellite Signals in GNSS Receivers. Circuits, Systems, and Signal Processing 36:7, 3004-3019. Crossref
(2017) Xception: Deep Learning with Depthwise Separable Convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1800-1807. Crossref
(2017) Adaptive Optimal Stochastic Control of Delay-Tolerant Networks. IEEE Transactions on Mobile Computing 16:7, 1815-1829. Crossref
(2017) Multistep stochastic mirror descent for risk-averse convex stochastic programs based on extended polyhedral risk measures. Mathematical Programming 163:1-2, 169-212. Crossref
(2017) Adaptive System Optimization Using Random Directions Stochastic Approximation. IEEE Transactions on Automatic Control 62:5, 2223-2238. Crossref
(2017) An Online and Distributed Approach for Modulation Classification Using Wireless Sensor Networks. IEEE Sensors Journal 17:6, 1781-1787. Crossref
(2017) Minimizing finite sums with the stochastic average gradient. Mathematical Programming 162:1-2, 83-112. Crossref
(2017) Finite-sample analysis of iterate averaging method for stochastic approximation with quadratic loss function. 2017 51st Annual Conference on Information Sciences and Systems (CISS), 1-6. Crossref
(2017) Online kernel nonnegative matrix factorization. Signal Processing 131, 143-153. Crossref
(2017) On the performance of random reshuffling in stochastic learning. 2017 Information Theory and Applications Workshop (ITA), 1-5. Crossref
, , , and . (2017) Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization. SIAM Journal on Optimization 27:2, 927-956. Abstract | PDF (1357 KB) 
2017. Batch and Online Mixture Learning: A Review with Extensions. Computational Information Geometry, 267-299. Crossref
2017. Improving Face Pose Estimation Using Long-Term Temporal Averaging for Stochastic Optimization. Engineering Applications of Neural Networks, 194-204. Crossref
2017. Directional Averages for Motion Segmentation in Discontinuity Preserving Image Registration. Medical Image Computing and Computer Assisted Intervention − MICCAI 2017, 249-256. Crossref
2017. A Localized Statistical Motion Model as a Reproducing Kernel for Non-rigid Image Registration. Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017, 261-269. Crossref
(2017) Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions. Mathematical Programming 161:1-2, 419-449. Crossref
(2017) Self-healing umbrella sampling: convergence and efficiency. Statistics and Computing 27:1, 147-168. Crossref
(2017) Brief Announcement. Proceedings of the ACM Symposium on Principles of Distributed Computing - PODC '17, 455-457. Crossref
(2017) Small Batch or Large Batch?. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '17, 1275-1284. Crossref
(2016) Variance-constrained actor-critic algorithms for discounted and average reward MDPs. Machine Learning 105:3, 367-417. Crossref
(2016) Optimizing cluster structures with inner product induced norm based dissimilarity measures: Theoretical development and convergence analysis. Information Sciences 372, 796-814. Crossref
(2016) An online primal-dual method for discounted Markov decision processes. 2016 IEEE 55th Conference on Decision and Control (CDC), 4516-4521. Crossref
(2016) Bound analysis of natural gradient descent in stochastic optimization setting. 2016 23rd International Conference on Pattern Recognition (ICPR), 4166-4171. Crossref
(2016) eg-VSSA: An extragradient variable sample-size stochastic approximation scheme: Error analysis and complexity trade-offs. 2016 Winter Simulation Conference (WSC), 690-701. Crossref
(2016) Mitigating Uncertainty via Compromise Decisions in Two-Stage Stochastic Linear Programming: Variance Reduction. Operations Research 64:6, 1422-1437. Crossref
(2016) Time Averaging Algorithms with Stopping Rules for Multi-Agent Consensus with Noisy Measurements. Asian Journal of Control 18:6, 1969-1982. Crossref
(2016) Optimal Bayesian Experimental Design for Priors of Compact Support with Application to Shock-Tube Experiments for Combustion Kinetics. International Journal for Numerical Methods in Engineering 108:2, 136-155. Crossref
(2016) A stochastic approximation approach to improve the convergence behavior of hierarchical atomistic-to-continuum multiscale models. Journal of the Mechanics and Physics of Solids 95, 480-500. Crossref
(2016) Excess-Risk of Distributed Stochastic Learners. IEEE Transactions on Information Theory 62:10, 5753-5785. Crossref
(2016) Analysis of a stochastic approximation algorithm for computing quasi-stationary distributions. Advances in Applied Probability 48:3, 792-811. Crossref
(2016) Stochastic gradient descent with finite samples sizes. 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), 1-6. Crossref
(2016) A Distributed Algorithm for Convex Constrained Optimization Under Noise. IEEE Transactions on Automatic Control 61:9, 2496-2511. Crossref
(2016) Large Margin Distribution Learning with Cost Interval and Unlabeled Data. IEEE Transactions on Knowledge and Data Engineering 28:7, 1749-1763. Crossref
(2016) The multivariate Révész’s online estimator of a regression function and its averaging. Mathematical Methods of Statistics 25:3, 151-167. Crossref
(2016) Stochastic approximation for combustion phase optimization of SI gasoline engines. 2016 European Control Conference (ECC), 1265-1270. Crossref
(2016) A Parallel Decomposition Method for Nonconvex Stochastic Multi-Agent Optimization Problems. IEEE Transactions on Signal Processing 64:11, 2949-2964. Crossref
(2016) High-Frequency Trading Meets Online Learning. Market Microstructure and Liquidity 02:01, 1650003. Crossref
(2016) Stopping Rules for Optimization Algorithms Based on Stochastic Approximation. Journal of Optimization Theory and Applications 169:2, 568-586. Crossref
(2016) Estimating the geometric median in Hilbert spaces with stochastic gradient algorithms: L p and almost sure rates of convergence. Journal of Multivariate Analysis 146, 209-222. Crossref
(2016) Accelerated gradient methods for nonconvex nonlinear and stochastic programming. Mathematical Programming 156:1-2, 59-99. Crossref
(2016) On projected stochastic gradient descent algorithm with weighted averaging for least squares regression. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2314-2318. Crossref
(2016) A Survey of Stochastic Simulation and Optimization Methods in Signal Processing. IEEE Journal of Selected Topics in Signal Processing 10:2, 224-241. Crossref
(2016) Randomized Algorithms for Distributed Nonlinear Optimization Under Sparsity Constraints. IEEE Transactions on Signal Processing 64:6, 1420-1434. Crossref
(2016) A hybrid stochastic-deterministic gradient descent algorithm for image reconstruction in cone-beam computed tomography. Biomedical Physics & Engineering Express 2:1, 015008. Crossref
(2016) Efficient control chart calibration by simulated stochastic approximation. IIE Transactions 48:1, 57-65. Crossref
and . (2016) On the Solution of Stochastic Optimization and Variational Problems in Imperfect Information Regimes. SIAM Journal on Optimization 26:4, 2394-2429. Abstract | PDF (648 KB) 
, , , and . (2016) An Empirical Interpolation and Model-Variance Reduction Method for Computing Statistical Outputs of Parametrized Stochastic Partial Differential Equations. SIAM/ASA Journal on Uncertainty Quantification 4:1, 244-265. Abstract | PDF (1003 KB) 
2016. Recent Results on Nonparametric Quantile Estimation in a Simulation Model. Challenges in Computational Statistics and Data Mining, 225-246. Crossref
2016. Székely Regularization for Uplift Modeling. Challenges in Computational Statistics and Data Mining, 135-154. Crossref
2016. Spatially-Dimension-Adaptive Sparse Grids for Online Learning. Sparse Grids and Applications - Stuttgart 2014, 133-162. Crossref
(2016) Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization. Mathematical Programming 155:1-2, 267-305. Crossref
2016. Deep learning of brain images and its application to multiple sclerosis. Machine Learning and Medical Imaging, 69-96. Crossref
(2016) Mobile App Tagging. Proceedings of the Ninth ACM International Conference on Web Search and Data Mining - WSDM '16, 63-72. Crossref
(2015) Common Subspace for Model and Similarity: Phrase Learning for Caption Generation from Images. 2015 IEEE International Conference on Computer Vision (ICCV), 2668-2676. Crossref
(2015) Supervised Single-Microphone Multi-Talker Speech Separation with Conditional Random Fields. IEEE/ACM Transactions on Audio, Speech, and Language Processing 23:12, 2334-2342. Crossref
(2015) Dynamic Spectrum Leasing Under Uncertainty: Modeling, Analysis and Algorithms. IEEE Transactions on Vehicular Technology 64:11, 5149-5159. Crossref
(2015) Stochastic Gradient Descent with Barzilai–Borwein update step for SVM. Information Sciences 316, 218-233. Crossref
(2015) Scalable estimation strategies based on stochastic approximations: classical results and new insights. Statistics and Computing 25:4, 781-795. Crossref
(2015) Distributed randomized algorithms for opinion formation, centrality computation and power systems estimation: A tutorial overview. European Journal of Control 24, 2-13. Crossref
(2015) Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-9. Crossref
(2015) Simulation Optimization: A Review and Exploration in the New Era of Cloud Computing and Big Data. Asia-Pacific Journal of Operational Research 32:03, 1550019. Crossref
(2015) Stochastic approximation search algorithms with randomization at the input. Automation and Remote Control 76:5, 762-775. Crossref
(2015) Approaches for solving the stochastic equilibrium assignment with variable demand: internal vs. external solution algorithms. Optimization Methods and Software 30:2, 338-364. Crossref
(2015) Ergodic Randomized Algorithms and Dynamics Over Networks. IEEE Transactions on Control of Network Systems 2:1, 78-87. Crossref
(2015) Recursive algorithms for parameter estimation with adaptive quantizer. Automatica 52, 192-201. Crossref
and . (2015) Stochastic Block Mirror Descent Methods for Nonsmooth and Stochastic Optimization. SIAM Journal on Optimization 25:2, 856-881. Abstract | PDF (446 KB) 
2015. An Overview of Stochastic Approximation. Handbook of Simulation Optimization, 149-178. Crossref
2015. Stochastic Approximation Methods and Their Finite-Time Convergence Properties. Handbook of Simulation Optimization, 179-206. Crossref
2015. A Guide to Sample Average Approximation. Handbook of Simulation Optimization, 207-243. Crossref
2015. Comparison of Phase II Control Charts Based on Variable Selection Methods. Frontiers in Statistical Quality Control 11, 151-162. Crossref
2015. Stochastic Optimization. International Encyclopedia of the Social & Behavioral Sciences, 473-481. Crossref
(2015) Central limit theorems for stochastic approximation with controlled Markov chain dynamics. ESAIM: Probability and Statistics 19, 60-80. Crossref
(2014) Privacy Aware Learning. Journal of the ACM 61:6, 1-57. Crossref
(2014) RES: Regularized Stochastic BFGS Algorithm. IEEE Transactions on Signal Processing 62:23, 6089-6104. Crossref
(2014) Simulation optimization: A tutorial overview and recent developments in gradient-based methods. Proceedings of the Winter Simulation Conference 2014, 21-35. Crossref
(2014) Noisy belief propagation decoder. 2014 48th Asilomar Conference on Signals, Systems and Computers, 2111-2115. Crossref
(2014) Nonparametric recursive quantile estimation. Statistics & Probability Letters 93, 102-107. Crossref
(2014) Improving training time of deep neural networkwith asynchronous averaged stochastic gradient descent. The 9th International Symposium on Chinese Spoken Language Processing, 446-449. Crossref
(2014) An Overview of Stochastic Approximation Monte Carlo. Wiley Interdisciplinary Reviews: Computational Statistics 6:4, 240-254. Crossref
(2014) Computing highly accurate confidence limits from discrete data using importance sampling. Statistics and Computing 24:4, 663-673. Crossref
(2014) A sparsity preserving stochastic gradient methods for sparse regression. Computational Optimization and Applications 58:2, 455-482. Crossref
(2014) Three Guidelines of Online Learning for Large-Scale Visual Recognition. 2014 IEEE Conference on Computer Vision and Pattern Recognition, 3574-3581. Crossref
(2014) Multi-agent consensus with noisy communication via time averaging. 2014 European Control Conference (ECC), 1530-1535. Crossref
(2014) An adaptive zero-variance importance sampling approximation for static network dependability evaluation. Computers & Operations Research 45, 51-59. Crossref
(2014) Energy-saving gossip algorithm for compressed sensing in multi-agent systems. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 5060-5064. Crossref
(2014) Exploring one pass learning for deep neural network training with averaged stochastic gradient descent. 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 6854-6858. Crossref
(2014) Estimation of a Ramsay-Curve Item Response Theory Model by the Metropolis–Hastings Robbins–Monro Algorithm. Educational and Psychological Measurement 74:2, 343-369. Crossref
(2014) A stochastic Kaczmarz algorithm for network tomography. Automatica 50:3, 910-914. Crossref
and . (2014) On Stochastic Subgradient Mirror-Descent Algorithm with Weighted Averaging. SIAM Journal on Optimization 24:1, 84-107. Abstract | PDF (361 KB) 
, , and . (2014) Optimal Primal-Dual Methods for a Class of Saddle Point Problems. SIAM Journal on Optimization 24:4, 1779-1814. Abstract | PDF (664 KB) 
2014. Fast LSTD Using Stochastic Approximation: Finite Time Analysis and Application to Traffic Control. Machine Learning and Knowledge Discovery in Databases, 66-81. Crossref
2014. Dynamic Sampling Schemes for Optimal Noise Learning Under Multiple Nonsmooth Constraints. System Modeling and Optimization, 85-95. Crossref
(2014) Monte Carlo sampling-based methods for stochastic optimization. Surveys in Operations Research and Management Science 19:1, 56-85. Crossref
(2014) Belief Propagation Algorithms on Noisy Hardware. IEEE Transactions on Communications, 1-1. Crossref
(2013) On the solution of stochastic optimization problems in imperfect information regimes. 2013 Winter Simulations Conference (WSC), 821-832. Crossref
(2013) An empirical sensitivity analysis of the Kiefer-Wolfowitz algorithm and its variants. 2013 Winter Simulations Conference (WSC), 945-956. Crossref
(2013) Performance of a Distributed Stochastic Approximation Algorithm. IEEE Transactions on Information Theory 59:11, 7405-7418. Crossref
(2013) Uncertainty Driven Probabilistic Voxel Selection for Image Registration. IEEE Transactions on Medical Imaging 32:10, 1777-1790. Crossref
(2013) Machine learning of molecular electronic properties in chemical compound space. New Journal of Physics 15:9, 095003. Crossref
(2013) A hybrid optimization method for acceleration of building linear classification models. The 2013 International Joint Conference on Neural Networks (IJCNN), 1-8. Crossref
(2013) Adaptive Learning Vector Quantization for Online Parametric Estimation. IEEE Transactions on Signal Processing 61:12, 3119-3128. Crossref
(2013) Fast waveform inversion without source-encoding. Geophysical Prospecting 61, 10-19. Crossref
(2013) Using dynamic conditional random field on single-microphone speech separation. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 146-150. Crossref
(2013) An empirical study of learning rates in deep neural networks for speech recognition. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 6724-6728. Crossref
(2013) Distributed Pareto Optimization via Diffusion Strategies. IEEE Journal of Selected Topics in Signal Processing 7:2, 205-220. Crossref
(2013) A new hybrid stochastic approximation algorithm. Optimization Letters 7:3, 593-606. Crossref
and . (2013) Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms. SIAM Journal on Optimization 23:4, 2061-2089. Abstract | PDF (542 KB) 
and . (2013) Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming. SIAM Journal on Optimization 23:4, 2341-2368. Abstract | PDF (294 KB) 
2013. Stochastic Approximation Algorithms. Stochastic Recursive Algorithms for Optimization, 17-28. Crossref
2013. Reliable Quantification and Efficient Estimation of Credit Risk. Risk Measures and Attitudes, 35-43. Crossref
2013. Algorithms: A Survey. Design of Experiments in Nonlinear Models, 277-333. Crossref
(2013) A Simple Model for On-Sensor Phase-Detection Autofocusing Algorithm. Journal of Computer and Communications 01:06, 11-17. Crossref
2013. Stochastic Approximation. Encyclopedia of Operations Research and Management Science, 1470-1476. Crossref
(2012) Compensating for CQI aging by channel prediction: The LTE downlink. 2012 IEEE Global Communications Conference (GLOBECOM), 4821-4827. Crossref
(2012) Averaging and derivative estimation within Stochastic Approximation algorithms. Proceedings Title: Proceedings of the 2012 Winter Simulation Conference (WSC), 1-9. Crossref
(2012) Optimal resource allocation in wireless communication and networking. EURASIP Journal on Wireless Communications and Networking 2012:1. Crossref
(2012) Adaptive Generalized Likelihood Ratio Control Charts for Detecting Unknown Patterned Mean Shifts. Journal of Quality Technology 44:4, 281-303. Crossref
(2012) Validation analysis of mirror descent stochastic approximation method. Mathematical Programming 134:2, 425-458. Crossref
(2012) Sample size selection in optimization methods for machine learning. Mathematical Programming 134:1, 127-155. Crossref
(2012) An optimal method for stochastic composite optimization. Mathematical Programming 133:1-2, 365-397. Crossref
(2012) A fast and recursive algorithm for clustering large datasets with -medians. Computational Statistics & Data Analysis 56:6, 1434-1449. Crossref
(2012) Some multivariate risk indicators: Minimization by using a Kiefer–Wolfowitz approach to the mirror stochastic algorithm. Statistics & Risk Modeling 29:1, 47-72. Crossref
, , , and . (2012) Ergodic Mirror Descent. SIAM Journal on Optimization 22:4, 1549-1578. Abstract | PDF (473 KB) 
and . (2012) Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework. SIAM Journal on Optimization 22:4, 1469-1492. Abstract | PDF (304 KB) 
2012. Stochastic Gradient Descent Tricks. Neural Networks: Tricks of the Trade, 421-436. Crossref
2012. Practical Recommendations for Gradient-Based Training of Deep Architectures. Neural Networks: Tricks of the Trade, 437-478. Crossref
(2012) On stochastic gradient and subgradient methods with adaptive steplength sequences. Automatica 48:1, 56-67. Crossref
2011. Bibliography. Statistical Learning and Data Science, 205-223. Crossref
(2011) An introspective on the Retrospective-Approximation paradigm. Proceedings of the 2011 Winter Simulation Conference (WSC), 412-421. Crossref
(2011) A regularized adaptive steplength stochastic approximation scheme for monotone stochastic variational inequalities. Proceedings of the 2011 Winter Simulation Conference (WSC), 4110-4121. Crossref
(2011) Subsampling Algorithms for Semidefinite Programming. Stochastic Systems 1:2, 274-305. Crossref
(2011) Information-Based Complexity, Feedback and Dynamics in Convex Programming. IEEE Transactions on Information Theory 57:10, 7036-7056. Crossref
(2011) General Bounds and Finite-Time Improvement for the Kiefer-Wolfowitz Stochastic Approximation Algorithm. Operations Research 59:5, 1211-1224. Crossref
(2011) Ergodic mirror descent. 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 701-706. Crossref
(2011) A Least Angle Regression Control Chart for Multidimensional Data. Technometrics 53:3, 285-296. Crossref
(2011) Quasi stochastic approximation. Proceedings of the 2011 American Control Conference, 2429-2435. Crossref
(2011) Large-scale image classification: Fast feature extraction and SVM training. CVPR 2011, 1689-1696. Crossref
(2011) Evaluating knowledge transfer and zero-shot learning in a large-scale setting. CVPR 2011, 1641-1648. Crossref
2011. Online Expectation Maximisation. Mixtures, 31-53. Crossref
(2011) The stochastic root-finding problem. ACM Transactions on Modeling and Computer Simulation 21:3, 1-23. Crossref
2011. Simultaneous Perturbation and Finite Difference Methods. Wiley Encyclopedia of Operations Research and Management Science. Crossref
and . (2011) A Generalization of the Averaging Procedure: The Use of Two-Time-Scale Algorithms. SIAM Journal on Control and Optimization 49:4, 1523-1543. Abstract | PDF (263 KB) 
2011. Generic Stochastic Gradient Methods. Wiley Encyclopedia of Operations Research and Management Science, 1-8. Crossref
(2011) Seismic Waveform Inversion by Stochastic Optimization. International Journal of Geophysics 2011, 1-18. Crossref
(2011) A framework for adaptive Monte Carlo procedures. Monte Carlo Methods and Applications 17:1. Crossref
(2010) Recursive Identification for Hammerstein Systems with State-space Model. Acta Automatica Sinica 36:10, 1460-1467. Crossref
2010. System Identification. The Control Systems Handbook, Second Edition, 57-1-57-39. Crossref
(2010) A stopping rule for linear stochastic approximation. 49th IEEE Conference on Decision and Control (CDC), 4171-4176. Crossref
(2010) On Complexity Issues of Online Learning Algorithms. IEEE Transactions on Information Theory 56:12, 6470-6481. Crossref
(2010) Ergodic Stochastic Optimization Algorithms for Wireless Communication and Networking. IEEE Transactions on Signal Processing 58:12, 6369-6386. Crossref
(2010) Using simulation-based Stochastic Approximation to optimize staffing of systems with Skills-Based-Routing. Proceedings of the 2010 Winter Simulation Conference, 3307-3317. Crossref
(2010) Root finding via DARTS — Dynamic Adaptive Random Target Shooting. Proceedings of the 2010 Winter Simulation Conference, 1255-1262. Crossref
(2010) The initial transient in steady-state point estimation: Contexts, a bibliography, the mse criterion, and the MSER statistic. Proceedings of the 2010 Winter Simulation Conference, 184-197. Crossref
(2010) Stochastic Root Finding and Efficient Estimation of Convex Risk Measures. Operations Research 58:5, 1505-1521. Crossref
(2010) Adaptive random search for continuous simulation optimization. Naval Research Logistics (NRL) 57:6, 583-604. Crossref
2010. References. Advanced Markov Chain Monte Carlo Methods, 327-352. Crossref
(2010) Stochastic learning algorithms for optimal design of wireless networks. 2010 IEEE 11th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 1-5. Crossref
(2010) The Robbins-Monro type stochastic differential equations. III. Polyak's averaging. Stochastics 82:2, 165-188. Crossref
(2010) An EM and a Stochastic Version of the EM Algorithm for Nonparametric Hidden Semi-Markov Models. Communications in Statistics - Simulation and Computation 39:2, 240-261. Crossref
2010. Large-Scale Machine Learning with Stochastic Gradient Descent. Proceedings of COMPSTAT'2010, 177-186. Crossref
2010. Stochastic Approximation for Multivariate and Functional Median. Proceedings of COMPSTAT'2010, 421-428. Crossref
(2010) Optimal two-hop routing in delay-tolerant networks. 2010 European Wireless Conference (EW), 881-888. Crossref
(2010) A tutorial on stochastic approximation algorithms for training Restricted Boltzmann Machines and Deep Belief Nets. 2010 Information Theory and Applications Workshop (ITA), 1-10. Crossref
(2010) Algorithms for Reinforcement Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 4:1, 1-103. Crossref
(2010) Stochastic approximation: a survey. WIREs Computational Statistics 2:1, 87-96. Crossref
(2009) LMS-2: Towards an algorithm that is as cheap as LMS and almost as efficient as RLS. Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, 1181-1188. Crossref
(2009) How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths. Mathematical Programming 120:1, 67-99. Crossref
(2009) A stochastic gradient type algorithm for closed-loop problems. Mathematical Programming 119:1, 51-78. Crossref
(2009) Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm. IEEE Transactions on Automatic Control 54:6, 1216-1229. Crossref
(2009) On-line expectation-maximization algorithm for latent data models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71:3, 593-613. Crossref
(2009) A stochastic approximation algorithm with multiplicative step size modification. Mathematical Methods of Statistics 18:2, 185-200. Crossref
(2009) Decentralized Stochastic Control of Delay Tolerant Networks. IEEE INFOCOM 2009 - The 28th Conference on Computer Communications, 1134-1142. Crossref
(2009) Bootstrap-based design of residual control charts. IIE Transactions 41:4, 275-286. Crossref
and . (2009) Smoothing Projected Gradient Method and Its Application to Stochastic Linear Complementarity Problems. SIAM Journal on Optimization 20:2, 627-649. Abstract | PDF (545 KB) 
, , , and . (2009) Robust Stochastic Approximation Approach to Stochastic Programming. SIAM Journal on Optimization 19:4, 1574-1609. Abstract | PDF (392 KB) 
2009. Recursive Computation of Value-at-Risk and Conditional Value-at-Risk using MC and QMC. Monte Carlo and Quasi-Monte Carlo Methods 2008, 193-208. Crossref
(2009) Low-Complexity Variable Step-Size Mechanism for Code-Constrained Constant Modulus Stochastic Gradient Algorithms Applied to CDMA Interference Suppression. IEEE Transactions on Signal Processing 57:1, 313-323. Crossref
(2009) A Stochastic Approximation Method and Its Application to Confidence Intervals. Journal of Computational and Graphical Statistics 18:1, 184-200. Crossref
(2009) Computing VaR and CVaR using stochastic approximation and adaptive unconstrained importance sampling. Monte Carlo Methods and Applications 15:3. Crossref
(2008) The mathematics of continuous-variable simulation optimization. 2008 Winter Simulation Conference, 122-132. Crossref
(2008) Adaptive downlink OFDMA resource allocation. 2008 42nd Asilomar Conference on Signals, Systems and Computers, 2203-2207. Crossref
(2008) Semimartingale stochastic approximation procedure and recursive estimation. Journal of Mathematical Sciences 153:3, 211-261. Crossref
(2008) Practical Design of Generalized Likelihood Ratio Control Charts for Autocorrelated Data. Technometrics 50:3, 357-370. Crossref
(2008) Stochastic Approximation Approaches to the Stochastic Variational Inequality Problem. IEEE Transactions on Automatic Control 53:6, 1462-1475. Crossref
(2008) Asynchronous distributed power and rate control in ad hoc networks: a game-theoretic approach. IEEE Transactions on Wireless Communications 7:7, 2536-2548. Crossref
(2008) Blind Adaptive Constrained Reduced-Rank Parameter Estimation Based on Constant Modulus Design for CDMA Interference Suppression. IEEE Transactions on Signal Processing 56:6, 2470-2482. Crossref
(2008) Asymptotic expansions of the Robbins-Monro process. Mathematical Methods of Statistics 17:2, 138-145. Crossref
(2008) Analysis and Optimization of Diagonally Layered Lattice Schemes for MIMO Fading Channels. IEEE Transactions on Information Theory 54:3, 1162-1185. Crossref
(2008) Edgeworth expansions for stochastic approximation theory. Mathematical Methods of Statistics 17:1, 44-65. Crossref
(2008) Application of kernel-based stochastic gradient algorithms to option pricing. Monte Carlo Methods and Applications 14:2. Crossref
(2007) Distributed Utility-Based Power Control: Objectives and Algorithms. IEEE Transactions on Signal Processing 55:10, 5058-5068. Crossref
(2007) Maximum likelihood from spatial random effects models via the stochastic approximation expectation maximization algorithm. Statistics and Computing 17:2, 163-177. Crossref
(2007) Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm. 2007 41st Annual Conference on Information Sciences and Systems, 35-40. Crossref
(2007) Combining Monte Carlo and Mean-Field-Like Methods for Inference in Hidden Markov Random Fields. IEEE Transactions on Image Processing 16:3, 824-837. Crossref
, , and . (2007) Stochastic $R_0$ Matrix Linear Complementarity Problems. SIAM Journal on Optimization 18:2, 482-506. Abstract | PDF (288 KB) 
2007. Stochastic Approximation Algorithms for Estimation of Spatial Mixed Models. Handbook of Latent Variable and Related Models, 399-421. Crossref
2007. 18 Stochastic Approximation Algorithms for Estimation of Spatial Mixed Models. Handbook of Latent Variable and Related Models, 399-421. Crossref
(2007) Quantile Estimation with Adaptive Importance Sampling. SSRN Electronic Journal. Crossref
(2006) Gradient-Based Simulation Optimization. Proceedings of the 2006 Winter Simulation Conference, 159-167. Crossref
(2006) Implementing and Diagnosing the Stochastic Approximation EM Algorithm. Journal of Computational and Graphical Statistics 15:4, 803-829. Crossref
(2006) Low-complexity variable step-size mechanisms for stochastic gradient algorithms in minimum variance CDMA receivers. IEEE Transactions on Signal Processing 54:6, 2302-2317. Crossref
(2006) The Averaged Robbins – Monro Method for Linear Problems in a Banach Space. Journal of Theoretical Probability 19:1, 166-189. Crossref
(2006) Feedback and weighting mechanisms for improving Jacobian (Hessian) estimates in the adaptive simultaneous perturbation algorithm. 2006 American Control Conference, 6 pp.. Crossref
(2006) Convergence Analysis for Feedback-and Weighting-Based Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm. Proceedings of the 45th IEEE Conference on Decision and Control, 5669-5674. Crossref
(2006) A Framework for Traffic Assignment with Travel Information. 2006 IEEE Intelligent Transportation Systems Conference, 218-223. Crossref
(2005) A Baseline-free Procedure for Transformation Models Under Interval Censorship. Lifetime Data Analysis 11:4, 473-488. Crossref
(2005) Recursive Aggregation of Estimators by the Mirror Descent Algorithm with Averaging. Problems of Information Transmission 41:4, 368-384. Crossref
(2005) Recursive EM and SAGE-inspired algorithms with application to DOA estimation. IEEE Transactions on Signal Processing 53:8, 2664-2677. Crossref
(2005) LMS Algorithms for Tracking Slow Markov Chains With Applications to Hidden Markov Estimation and Adaptive Multiuser Detection. IEEE Transactions on Information Theory 51:7, 2475-2490. Crossref
(2005) Least mean square algorithms with Markov regime-switching limit. IEEE Transactions on Automatic Control 50:5, 577-593. Crossref
(2005) Ascent-based Monte Carlo expectation- maximization. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67:2, 235-251. Crossref
(2005) Recursive K-distribution parameter estimation. IEEE Transactions on Signal Processing 53:2, 397-402. Crossref
(2005) Particle filtering and Polyak averaging-based non-stationary noise tracking for ASR in noise. IEEE Workshop on Automatic Speech Recognition and Understanding, 2005., 337-342. Crossref
(2005) Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Transactions on Modeling and Computer Simulation 15:1, 74-107. Crossref
(2004) Adaptive Filtering With Averaging-Based Algorithm for Feedforward Active Noise Control Systems. IEEE Signal Processing Letters 11:6, 557-560. Crossref
2004. Optimization Techniques. Stochastic Processes, 167-221. Crossref
2004. Analysis of Recursive Algorithms. Stochastic Processes, 223-314. Crossref
(2004) A universal procedure for parametric frailty models. Journal of Statistical Computation and Simulation 74:1, 1-13. Crossref
(2002) Fast stochastic power control algorithms for nonlinear multiuser receivers. IEEE Transactions on Communications 50:11, 1817-1827. Crossref
(2002) On-line almost-sure parameter estimation for partially observed discrete-time linear systems with known noise characteristics. International Journal of Adaptive Control and Signal Processing 16:6, 435-453. Crossref
(2002) Asymptotically optimal smoothing of averaged LMS estimates for regression parameter tracking. Automatica 38:8, 1287-1293. Crossref
(2002) Recursive EM algorithm for stochastic ML DOA estimation. IEEE International Conference on Acoustics Speech and Signal Processing, III-3029-III-3032. Crossref
(2002) Blind adaptive multiuser detection for cellular systems using stochastic approximation with averaging. IEEE Journal on Selected Areas in Communications 20:2, 310-319. Crossref
(2002) ASYMPTOTICALLY OPTIMAL SMOOTHING OF AVERAGED LMS FOR REGRESSION PARAMETER TRACKING. IFAC Proceedings Volumes 35:1, 163-168. Crossref
(2001) Recursive least-squares and accelerated convergence in stochastic approximation schemes. International Journal of Adaptive Control and Signal Processing 15:2, 169-178. Crossref
(2001) On the design of a stable adaptive filter for state estimation in high dimensional systems. Automatica 37:3, 341-359. Crossref
(2000) Online Estimation of Allan Variance Parameters. Journal of Guidance, Control, and Dynamics 23:6, 980-987. Crossref
(2000) Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Transactions on Automatic Control 45:10, 1839-1853. Crossref
(2000) Suppression of multiple narrowband interferers in a spread-spectrum communication system. IEEE Journal on Selected Areas in Communications 18:8, 1365-1374. Crossref
(2000) Rates of Convergence of Adaptive Step-Size of Stochastic Approximation Algorithms. Journal of Mathematical Analysis and Applications 244:2, 333-347. Crossref
(2000) Recursive estimation of distributional fix-points. Journal of Applied Probability 37:01, 73-87. Crossref
(2000) Recursive estimation of distributional fix-points. Journal of Applied Probability 37:1, 73-87. Crossref
and . (2000) Martingale Models of Stochastic Approximation and Their Convergence. Theory of Probability & Its Applications 44:2, 333-360. Abstract | PDF (249 KB) 
and . (2000) On Stochastic Approximation Procedures with Averaging. Theory of Probability & Its Applications 44:3, 591-604. Abstract | PDF (181 KB) 
. (2000) Asymptotic Almost Sure Efficiency of Averaged Stochastic Algorithms. SIAM Journal on Control and Optimization 39:1, 49-72. Abstract | PDF (246 KB) 
(2000) On Recursive Estimation in Incomplete Data Models. Statistics 34:1, 27-51. Crossref
(2000) Averaged stochastic gradient algorithms for adaptive blind multiuser detection in DS/CDMA systems. IEEE Transactions on Communications 48:1, 125-134. Crossref
1999. Stochastic Optimization, Stochastic Approximation and Simulated Annealing. Wiley Encyclopedia of Electrical and Electronics Engineering. Crossref
(1999) Stochastic approximation algorithms: Overview and recent trends. Sadhana 24:4-5, 425-452. Crossref
(1999) Review Of Stochastic Approximation Algorithms And Applications [Book Reviews]. Proceedings of the IEEE 87:4, 688-690. Crossref
, , and . (1999) Asymptotic Efficiency of Perturbation-Analysis-Based Stochastic Approximation with Averaging. SIAM Journal on Control and Optimization 37:6, 1822-1847. Abstract | PDF (451 KB) 
(1999) On-line estimation of Allan variance parameters. 1999 Information, Decision and Control. Data and Information Fusion Symposium, Signal Processing and Communications Symposium and Decision and Control Symposium. Proceedings (Cat. No.99EX251), 439-444. Crossref
(1999) Stochastic optimization and the simultaneous perturbation method. Proceedings of the 31st conference on Winter simulation Simulation---a bridge to the future - WSC '99, 101-109. Crossref
(1998) Model-free control of nonlinear stochastic systems with discrete-time measurements. IEEE Transactions on Automatic Control 43:9, 1198-1210. Crossref
(1998) On closed-loop adaptive noise cancellation. IEEE Transactions on Automatic Control 43:8, 1103-1107. Crossref
(1998) Globally convergent stochastic optimization with optimal asymptotic distribution. Journal of Applied Probability 35:02, 395-406. Crossref
(1998) Globally convergent stochastic optimization with optimal asymptotic distribution. Journal of Applied Probability 35:2, 395-406. Crossref
(1998) Adaptive estimation of HMM transition probabilities. IEEE Transactions on Signal Processing 46:5, 1374-1385. Crossref
(1998) Review of Stochastic Approximation Algorithms and Applications [Book Reviews]. IEEE Transactions on Automatic Control 43:5, 753-755. Crossref
and . (1998) Budget-Dependent Convergence Rate of Stochastic Approximation. SIAM Journal on Optimization 8:1, 217-247. Abstract | PDF (435 KB) 
(1998) On adaptive HMM state estimation. IEEE Transactions on Signal Processing 46:2, 475-486. Crossref
(1998) Asymptotically efficient recursive estimation for incomplete data models using the observed information. Metrika 47:1, 119-145. Crossref
and . (1997) Weighted Means in Stochastic Approximation of Minima. SIAM Journal on Control and Optimization 35:5, 1811-1827. Abstract | PDF (713 KB) 
(1997) Stochastic optimization algorithms of a Bayesian design criterion for Bayesian parameter estimation of nonlinear regression models: Application in pharmacokinetics. Mathematical Biosciences 144:1, 45-70. Crossref
(1997) Non-Linear Black Box Models in System Identification. IFAC Proceedings Volumes 30:9, 1-12. Crossref
(1997) Optimization of computer simulation models with rare events. European Journal of Operational Research 99:1, 89-112. Crossref
(1997) Weighted averaging and stochastic approximation. Mathematics of Control, Signals, and Systems 10:1, 41-60. Crossref
1997. Dynamical Stability of a High-Dimensional Self-Organizing Map. Mathematics of Neural Networks, 389-393. Crossref
(1997) A one-measurement form of simultaneous perturbation stochastic approximation. Automatica 33:1, 109-112. Crossref
(1996) Learning algorithms for feedforward networks based on finite samples. IEEE Transactions on Neural Networks 7:4, 926-940. Crossref
(1996) Stochastic Version Of Second-Order Optimization Using Only Function Measurements. IFAC Proceedings Volumes 29:1, 1838-1843. Crossref
and . (1996) Stochastic Approximation Methods for Systems over an Infinite Horizon. SIAM Journal on Control and Optimization 34:2, 712-756. Abstract | PDF (5515 KB) 
(1996) Iterative and recursive estimators for hidden Markov errors-in-variables models. IEEE Transactions on Signal Processing 44:3, 629-639. Crossref
and . (1996) On the Averaged Stochastic Approximation for Linear Regression. SIAM Journal on Control and Optimization 34:1, 31-61. Abstract | PDF (2119 KB) 
1996. Stochastic Approximation. Optimization of Stochastic Models, 281-322. Crossref
1996. Searching for the Best: Stochastic Approximation, Simulated Annealing and Related Procedures. Identification, Adaptation, Learning, 514-549. Crossref
1996. On Neural Network Model Structures in System Identification. Identification, Adaptation, Learning, 366-399. Crossref
(1996) Simulation-based optimization—convergence analysis and statistical inference. Communications in Statistics. Stochastic Models 12:3, 425-454. Crossref
(1996) Стохастические дифференциальные уравнения: негладкость коэффициентов, регрессионные модели и стохастическая аппроксимация. Успехи математических наук 51:5, 43-136. Crossref
(1995) Some results about averaging in stochastic approximation. Metrika 42:1, 153-171. Crossref
(1995) Analysis of adaptive step-size SA algorithms for parameter tracking. IEEE Transactions on Automatic Control 40:8, 1403-1410. Crossref
(1995) Monte carlo estimation for guaranteed-coverage non-normal tolerance intervals. Journal of Statistical Computation and Simulation 51:2-4, 223-238. Crossref
(1995) Stochastic approximation with averaging and feedback: rapidly convergent "on-line" algorithms. IEEE Transactions on Automatic Control 40:1, 24-34. Crossref
1995. Stochastic Approximation with Averaging and Feedback: Faster Convergence. Adaptive Control, Filtering, and Signal Processing, 205-228. Crossref
1995. Adaptive Filtering with Averaging. Adaptive Control, Filtering, and Signal Processing, 375-396. Crossref
(1994) Letters to the Editor: COMMENT ON WHITE (1989). Journal of the American Statistical Association 89:428, 1571-1571. Crossref
(1994) Continuous-time stochastic approximation: Convergence and asymptotic efficiency. Stochastics and Stochastic Reports 51:3-4, 217-239. Crossref
(1994) On-line identification of hidden Markov models via recursive prediction error techniques. IEEE Transactions on Signal Processing 42:12, 3535-3539. Crossref
(1994) Continuous-time stochastic approximation: Convergence and asymptotic efficiency. Stochastics and Stochastic Reports 51:1-2, 111-132. Crossref
(1994) Neural Networks in System Identification. IFAC Proceedings Volumes 27:8, 359-382. Crossref
(1994) Adaptive estimation of hidden nearly completely decomposable Markov chains with applications in blind equalization. International Journal of Adaptive Control and Signal Processing 8:3, 237-260. Crossref
(1994) On-line estimation of dynamic shock-error models based on the Kullback Leibler information measure. IEEE Transactions on Automatic Control 39:5, 1129-1135. Crossref
and . (1993) Accelerated Stochastic Approximation. SIAM Journal on Optimization 3:4, 868-881. Abstract | PDF (1019 KB) 
(1993) On a continuous time stochastic approximation problem. Acta Applicandae Mathematicae 33:1, 3-20. Crossref
and . (1993) Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes. SIAM Journal on Control and Optimization 31:4, 1045-1062. Abstract | PDF (1462 KB) 
(1993) Recursive Prediction Error Techniques for Adaptive Estimation of Hidden Markov Models *. IFAC Proceedings Volumes 26:2, 349-352. Crossref
(1993) Recursive Estimation of Shock-error Models. IFAC Proceedings Volumes 26:2, 355-358. Crossref
The EM Algorithm, Its Randomized Implementation and Global Optimization: Some Challenges and Opportunities for Operations Research. Perspectives in Operations Research, 367-392. Crossref
Distributed Optimization and Duality in QoS Control for Wireless Best-Effort Traffic. Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005., 30-35. Crossref
Ergodic control applied to car suspension design. [1992] Proceedings of the 31st IEEE Conference on Decision and Control, 2056-2061. Crossref
Adaptive non-linear time-series estimation based on hidden Markov models. Proceedings of 32nd IEEE Conference on Decision and Control, 720-725. Crossref
Analysis of adaptive step size SA algorithms for parameter tracking. Proceedings of 1994 33rd IEEE Conference on Decision and Control, 730-737. Crossref
Quasi-efficient stochastic approximation on the basis of neural networks. Proceedings of 1994 33rd IEEE Conference on Decision and Control, 1163-1164. Crossref
Aspects on accelerated convergence in stochastic approximation schemes. Proceedings of 1994 33rd IEEE Conference on Decision and Control, 1649-1652. Crossref
Model-free control of nonlinear stochastic systems in discrete time. Proceedings of 1995 34th IEEE Conference on Decision and Control, 2199-2204. Crossref
Weighted averaging and stochastic approximation. Proceedings of 35th IEEE Conference on Decision and Control, 1071-1076. Crossref
Some guidelines for using iterate averaging in stochastic approximation. Proceedings of the 36th IEEE Conference on Decision and Control, 2287-2290. Crossref
Accelerated second-order stochastic optimization using only function measurements. Proceedings of the 36th IEEE Conference on Decision and Control, 1417-1424. Crossref
Adaptive stochastic approximation by the simultaneous perturbation method. Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171), 3872-3879. Crossref
Traffic Modeling with Gamma Mixtures and Dynamical Bandwidth Provisioning. 4th Annual Communication Networks and Services Research Conference (CNSR'06), 123-130. Crossref
Averaging methods for blind multiuser detection in DS/CDMA systems. IEEE GLOBECOM 1998 (Cat. NO. 98CH36250), 3566-3571. Crossref
Recursive estimation of K-distribution parameters. 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)., II-425-8. Crossref
Stochastic power control with bit estimation and base station processing. 2002 IEEE International Conference on Communications. Conference Proceedings. ICC 2002 (Cat. No.02CH37333), 1616-1620. Crossref
Stochastic power control with averaging. 2000 IEEE International Conference on Personal Wireless Communications. Conference Proceedings (Cat. No.00TH8488), 325-329. Crossref
DOA estimation of multiple moving sources using recursive EM algorithms. Sensor Array and Multichannel Signal Processing Workshop Proceedings, 2002, 323-326. Crossref
An improved averaged stochastic gradient algorithms for blind multiuser detection in DS/CDMA systems. IEEE VTS 53rd Vehicular Technology Conference, Spring 2001. Proceedings (Cat. No.01CH37202), 1947-1952. Crossref
Stochastic version of second-order (Newton-Raphson) optimization using only function measurements. Winter Simulation Conference Proceedings, 1995., 347-352. Crossref
A review of simulation optimization techniques. 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274), 151-158. Crossref
Confidence regions for stochastic approximation algorithms. Proceedings of the Winter Simulation Conference, 370-376. Crossref