We present a new approach to clustering of time series based on a minimization of the averaged clustering functional. The proposed functional describes the mean distance between observation data and its representation in terms of $\mathbf{K}$ abstract models of a certain predefined class (not necessarily given by some probability distribution). For a fixed time series $x(t)$ this functional depends on $\mathbf{K}$ sets of model parameters $\Theta=(\theta_1,\dots,\theta_\mathbf{K})$ and $\mathbf{K}$ functions of cluster affiliations $\Gamma=(\gamma_1(t),\dots,\gamma_{\mathbf{K}}(t))$ (characterizing the affiliation of any element $x(t)$ of the analyzed time series to one of the $\mathbf{K}$ clusters defined by the considered model parameters). We demonstrate that for a fixed set of model parameters $\Theta$ the appropriate Tykhonov-type regularization of this functional with some regularization factor $\epsilon^2$ results in a minimization problem similar to a variational problem usually associated with one-dimensional nonhomogeneous partial differential equations. This analogy allows us to apply the finite element framework to the problem of time series analysis and to propose a numerical scheme for time series clustering. We investigate the conditions under which the proposed scheme allows a monotone improvement of the initial parameter guess with respect to the minimization of the discretized version of the regularized functional. We also discuss the interpretation of the regularization factor in the Markovian case and show its connection to metastability and exit times. The computational performance of the resulting method is investigated numerically on multidimensional test data and is applied to the analysis of multidimensional historical stock market data.

MSC codes

  1. 62-07
  2. 62H30
  3. 62H25
  4. 65M60
  5. 60J10


  1. time series analysis
  2. inverse problems
  3. regularization
  4. finite element method

Get full access to this article

View all available purchase options and get full access to this article.


A. Sutera, R., Benzi, G. Parisi, and A. Vulpiani, Stochastic resonance in climatic change, Tellus, 3 (1982), pp. 10–16.
C. Nicolis, Stochastic aspects of climatic transitions—response to a periodic forcing, Tellus, 34 (1982), pp. 1–9.
A. A. Tsonis and J. B. Elsner, Multiple attractors, fractal basins and long term climate dynamics, Beitr. Phys. Atmosph., 63 (1990), pp. 171–176.
T. N. Palmer, A nonlinear dynamical perspective on climate prediction, J. Climate, 12 (1999), pp. 575–591.
J. D. Hamilton, A new approach to the economic analysis of nonstationary time series and the business cycle, Econometrica, 57 (1989), pp. 357–384.
R. S. Tsay, Analysis of Financial Time Series, 2nd ed., Wiley-Interscience, Hoboken, NJ, 2005.
R. Elber and M. Karplus, Multiple conformational states of proteins: A molecular dynamics analysis of Myoglobin, Science, 235 (1987), pp. 318–321.
H. Frauenfelder, P. J. Steinbach, and R. D. Young, Conformational relaxation in proteins, Chem. Soc., 29A (1989), pp. 145–150.
C. Schütte, A. Fischer, W. Huisinga, and P. Deuflhard, A direct approach to conformational dynamics based on hybrid Monte Carlo, J. Comput. Phys., 151 (1999), pp. 146–168.
J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York, 1981.
F. Höppner, F. Klawonn, R. Kruse, and T. Runkler, Fuzzy Cluster Analysis, John Wiley and Sons, New York, 1999.
J. C. Bezdek, R. H. Hathaway, M. J. Sabin, and W. T. Tucker, Convergence theory for fuzzy c-means: Counterexamples and repairs, IEEE Trans. Systems, 17 (1987), pp. 873–877.
R. H. Hathaway and J. C. Bezdek, Switching regression models and fuzzy clustering, IEEE Trans. Fuzzy Systems, 1 (1993), pp. 195–204.
G. J. McLachlan and D. Peel, Finite Mixture Models, Wiley, New York, 2000.
S. Fruhwirth-Schnatter, Finite Mixture and Markov Switching Models, Springer-Verlag, New York, 2006.
J. A. Bilmes, A Gentle Tutorial of the EM Algorithm and Its Applications to Parameter Estimation for Gaussian Mixture and Hidden Markov Models, International Computer Science Institute, Berkeley, CA, 1998.
I. Horenko, J. Schmidt-Ehrenberg, and C. Schütte, Set-oriented dimension reduction: Localizing principal component analysis via hidden Markov models, in R. Glen, M. R. Berthold, and I. Fischer, eds., CompLife 2006, Lecture Notes in Bioinformatics 4216, Springer-Verlag, Berlin, Heidelberg, 2006, pp. 98–115.
I. Horenko, R. Klein, S. Dolaptchiev, and C. Schütte, Automated generation of reduced stochastic weather models I: Simultaneous dimension and model reduction for time series analysis, Multiscale Model Simul., 6 (2008), pp. 1125–1145.
I. Horenko, On simultaneous data–based dimension reduction and hidden phase identification, J. Atmospheric Sci., 65 (2008), pp. 1941–1954.
A. H. Monahan, Nonlinear principal component analysis by neural networks: Theory and application to the Lorenz system, J. Climate, 13 (2000), pp. 821–835.
W. Härdle and L. Simar, Applied Multivariate Statistical Analysis, Springer-Verlag, New York, 2003.
I. T. Jolliffe, Principal Component Analysis, Springer-Verlag, New York, 2002.
P. Deuflhard, Newton Methods for Nonlinear Problems. Affine Invariance and Adaptive Algorithms, Springer Ser. Comput. Math. 35, Springer-Verlag, Berlin, 2004.
A. Tikhonov, On the stability of inverse problems, Dokl. Akad. Nauk SSSR, 39 (1943), pp. 195–198.
A. Hoerl, Application of ridge analysis to regression problems, Chemical Engineering Progress, 58 (1962), pp. 54–59.
G. Wahba, Spline Models for Observational Data, CBMS-NSF Regional Conf. Ser. in Appl. Math. 59, SIAM, Philadelphia, 1990.
D. Braess, Finite Elements: Theory, Fast Solvers and Applications to Solid Mechanics, 3rd ed., Cambridge University Press, Cambridge, UK, 2007.
M. K. Kozlov, L. G. Kachiyan, and S. P. Tarasov, Polynomial solvability of convex quadratic programming, Sov. Math. Dokl., 20 (1979), pp. 1108–1111.
J. Nocedal and S. J. Wright, Numerical Optimization, Springer-Verlag, New York, 1999.
B. de Finetti, Theory of Probability, Vol. I, Wiley, New York, 1974.
B. de Finetti, Theory of Probability, Vol. II, Wiley, New York, 1975.
H. Gardiner, Handbook of Stochastical Methods, Springer-Verlag, Berlin, 2000.
D. Calvetti, S. Morigi, L. Reichel, and F. Sgallari, Tikhonov regularization and the l-curve for large discrete ill-posed problems, J. Comput. Appl. Math., 123 (2000), pp. 423–446.
D. Calvetti, L. Reichel, and A. Shuibi, L-curve and curvature bounds for Tikhonov regularization, Numer. Algorithms, 35 (2004), pp. 301–314.
P. Metzner, I. Horenko, and C. Schuette, Generator estimation of Markov jump processes based on incomplete observations nonequidistant in time, Phys. Rev. E, 76 (2007), 066702.
C. Schütte and W. Huisinga, Biomolecular conformations can be identified as metastable sets of molecular dynamics, in P. G. Ciarlet and J.-L. Lions, eds., Handbook of Numerical Analysis, Volume X, North–Holland, Amsterdam, 2003, pp. 699–744.
M. Weber and P. Deuflhard, Perron cluster cluster analysis, J. Chem. Phys., 5 (2003), pp. 802–827.
H. Engl and A. Neubauer, An improved version of Marti's method for solving ill-posed linear integral equations, Math. Comp., 45 (1985), pp. 405–416.
E. Schock, Morozov<#0092>s discrepancy principle for Tikhonov regularization of severely ill-posed problems in finite-dimensional subspaces, Numer. Funct. Anal. Optim., 21 (2000), pp. 901–916.
B. Kedem and K. Fokianos, Regression Models for Time Series Analysis, Wiley Series in Probability and Statistics, Wiley-Interscience, Hoboken, NJ, 2002.

Information & Authors


Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: 62 - 83
ISSN (online): 1095-7197


Submitted: 18 February 2008
Accepted: 6 February 2009
Published online: 5 February 2010

MSC codes

  1. 62-07
  2. 62H30
  3. 62H25
  4. 65M60
  5. 60J10


  1. time series analysis
  2. inverse problems
  3. regularization
  4. finite element method



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options


View PDF







Copy the content Link

Share with email

Email a colleague

Share on social media

On May 28, 2024, our site will enter Read Only mode for a limited time in order to complete a platform upgrade. As a result, the following functions will be temporarily unavailable: registering new user accounts, any updates to existing user accounts, access token activations, and shopping cart transactions. Contact [email protected] with any questions.