Projected Nonlinear Least Squares for Exponential Fitting

The modern ability to collect vast quantities of data provides a challenge for parameter estimation. When posed as a nonlinear least squares problem fitting a model to data, the cost of each iteration grows linearly with the amount of data and with large data it can easily become too expensive to perform many iterations. Here we reduce the cost of each iteration by orthogonally projecting the data onto a low-dimensional subspace preserving the quality of the resulting parameter estimates. We provide results from both an optimization and a statistical perspective that show accurate parameter estimates are recovered when the subspace angles between this subspace and the range Jacobian of the model at the current iterate remain small. However, for this approach to reduce computational complexity, both the projected model and projected Jacobian must be computed inexpensively. This places a constraint on the pairs of models and subspaces for which this approach provides a computational speedup. Here we consider the exponential fitting problem projected onto the range of a Vandermonde matrix for which both the projected model and projected Jacobian can be computed in closed form using a generalized geometric sum formula. We further provide an inexpensive heuristic for choosing this Vandermonde matrix which ensures the subspace angles with the Jacobian remain small, and use this heuristic to update the subspace during optimization. Although the asymptotic cost still depends on the data dimension, the overall cost of solving this sequence of projected problems is significantly less expensive than the original.