SIAM Digital Library
 
 
 

SIAM J. on Control and Optimization

All Journal content published prior to 1997 is part of LOCUS.

Search Issue | RSS Feeds RSS
Previous Issue

1975

Volume 13, Issue 6, pp. 1115-1281


A Semigroup Representation of the Maximum Expected Reward Vector in Continuous Parameter Markov Decision Theory

Stanley R. Pliska

SIAM J. Control 13, pp. 1115-1129 (15 pages) | Cited 2 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
The maximum expected reward vector that arises in continuous parameter Markov decision problems is frequently characterized as the unique solution of a certain Cauchy problem. This paper generalizes this characterization by viewing the maximum expected reward vector as a nonlinear semigroup in an appropriate Banach space. This perspective has several advantages. First, the semi-group may exist even though the corresponding Cauchy problem does not have a solution. Second, this approach is often useful in showing when the Cauchy problem does have a solution. Third, these methods are useful in the study of the method of successive approximations. Finally, these methods appear likely to unify some diverse results in Markov decision theory.
The results in this paper are very general. First, sufficient conditions are given for the semigroup to exist. The discounted reward case is studied next ; a certain operator is shown to have a unique singular point that is the strong limit of the semigroup as the parameter $t \to \infty $. The asymptotic properties of the semigroup in the case of undiscounted rewards are studied with the aid of some fixed point theorems for monotone and nonexpansive operators; the transient, positive, negative and optimal stopping cases are studied in this context. The paper concludes with two examples. The first is a controlled diffusion process on a compact interval of the real line. The second is a controlled jump process with general state and action spaces.

The Infinite Dimensional Riccati Equation with Applications to Affine Hereditary Differential Systems

Ruth F. Curtain

SIAM J. Control 13, pp. 1130-1143 (14 pages)

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
The infinite-dimensional versions of the linear quadratic cost control problem and of the linear filtering problem lead to an infinite-dimensional Riccati equation with unbounded operators. Existence and uniqueness theorems for mild solutions of these were established in The infinite dimensional Riccati equations, Ruth F. Curtain and A. J. Pritchard (to appear in J. Math. Anal. Appl.) using a semigroup and evolution operator approach. Although this formulation was very general, covering a large class of parabolic partial differential control systems, it does not cover the semigroup formulation of linear hereditary differential equations introduced by Delfour and Mitter. This paper remedies this and applies the theory to the linear quadratic cost control problem for the afBne linear hereditary differential case.

Coprime Factorizations and Stability of Multivariable Distributed Feedback Systems

Mathukumalli Vidyasagar

SIAM J. Control 13, pp. 1144-1155 (12 pages) | Cited 8 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
The stability of multivariable feedback systems presents different problems from the stability of single loop feedback systems, owing mainly to the complexities of “pole”-“zero” cancellation in the multivariate case. In this paper, the “coprime factorization” of a nonrational transfer function matrix is defined and is used in studying the stability of multivariable distributed feedback systems. However, the stability results based on coprime factorizations, though they are quite elegant, do not lead to readily applicable testing procedures. For this reason, we introduce the notion of “pseudo-coprime” factorizations. These also lead to many stability theorems. As a special case of these stability results, we obtain explicit necessary and sufficient conditions for the stability of a multivariable feedback system whose open loop transfer function contains a finite number of poles in the closed right half-plane, but is otherwise stable. These results significantly generalize those of Callier and Desoer [8].

Stability Criteria for Time-Varying Systems in Hilbert Space

Andrew Acker

SIAM J. Control 13, pp. 1156-1171 (16 pages)

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
The Freedman and Zames logarithmic variation criterion for stability [1] is extended to two types of systems in Hilbert space : one involving a generalized causal convolution operator and a time-varying gain, the other involving a real causal convolution operator and a time-varying un-bounded operator. Also, the conditions on the Nyquist diagram are relaxed, and it is shown that a certain bound on either the average logarithmic increase or the average logarithmic decrease of the gain functions insures stability.

On Normality and Conjugate Point Criteria for Singular Extremals

Violet B. Haas

SIAM J. Control 13, pp. 1172-1182 (11 pages) | Cited 3 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
The Moore–Penrose generalized inverse is employed in the control problem of Lagrange to obtain necessary and sufficient conditions for normality, sufficient conditions for the nonexistence of conjugate points and sufficient conditions for the nonnegativity of the second variation. Our results are valid for both regular and singular extremal arcs.

Consistency of Least-Squares Estimates Used in Linear Systems Identification

D. O. Norris and L. E. Snyder

SIAM J. Control 13, pp. 1183-1193 (11 pages) | Cited 3 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
Least-squares estimation of the parameters of a single input-single output linear autonomous system is considered where both plant noise and observation noise are present. It is shown that under fairly general conditions that the estimates converge almost surely to the true system parameters.

An Arc Method for Nonlinear Programming

Garth P. McCormick

SIAM J. Control 13, pp. 1194-1216 (23 pages) | Cited 4 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
An algorithm using second derivatives for solving the optimization problem : minimize $f(x)$ subject to $g_i (x) \geqq 0$, $i = 1, \cdots ,m$, where the $g_i $ are not necessarily linear is presented. The basic idea is to generate a sequence of feasible points with decreasing objective value by movement along piecewise, smooth, quadratic arcs. Cluster points of the sequence generated are shown to be second order Kuhn–Tucker points. If the strict second order sufficiency conditions hold, the rate of convergence is shown to be at least quadratic.

On Optimal Stochastic Control of Discrete-Time Systems in Hilbert Space

Jerzy Zabczyk

SIAM J. Control 13, pp. 1217-1234 (18 pages) | Cited 5 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
A general system described by a linear difference equation in a Hilbert space is considered. Three types of disturbances, control-dependent noise, state-dependent noise and purely additive noise, are taken into account. The cost function is assumed to be quadratic. The existence of an optimal stationary strategy and the uniqueness of the stationary measure related to this strategy are proved.
Special attention is paid to the related Riccati operator difference equation and the asymptotic behavior of the solution of such an equation is investigated. Under certain assumptions, the existence and uniqueness of the solution of the algebraic Riccati equation are proved, too.

Periodicity, Detectability and the Matrix Riccati Equation

G. A. Hewer

SIAM J. Control 13, pp. 1235-1251 (17 pages) | Cited 22 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
This paper discusses the periodic solution of matrix Riccati differential equations with periodic coefficients. Such equations arise in linear filtering and control and in many other applications. The principal result: the existence of a periodic solution is equivalent to detectability and stabilizability of certain coefficient pairs. This result generalizes the Kalman–Wonham–Kucera theorem for algebraic Riccati equations. Among the numerous preliminaries is a discussion, apparently new, of detectability for linear periodic control systems. Another important result, for a linear matrix differential equation, is the equivalence of a bounded solution, an exponentially stable solution and a periodic solution. Finally, the periodic solution is shown to be an equilibrium solution in the sense of Kalman.

Time-Varying Systems

Michael A. Arbib and Ernest G. Manes

SIAM J. Control 13, pp. 1252-1270 (19 pages) | Cited 1 time

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
Section 1 provides a theory of reachability, observability, minimal realization and duality for time-varying linear systems, using only the basic language of linear algebra. Section 2 uses category theory to show that time-varying dynamics for adjoint processes in a category $\mathcal{K}$ may be defined as ad joint processes in a suitable new category $\mathcal{K}^{\bf z} $.

On Controllability by Means of Two Vector Fields

Norman Levitt and Héctor J. Sussmann

SIAM J. Control 13, pp. 1271-1281 (11 pages) | Cited 13 times

Online Publication Date: July 18, 2006

Full Text: | Download PDF

Show Abstract
A set $S$ of vector fields on a differentiable manifold $M$ is said to be completely controllable if for every pair $(m,m')$ of points of $M$ there exists a trajectory of $S$ from $m$ to $m'$. Here a trajectory of $S$ is a curve which is an integral curve of some $X \in S$ or a finite concatenation of such curves so that, in general, a trajectory of $S$ run in reverse is no longer a trajectory. Our main theorem is: on every connected paracompact manifold of class $C^k $, $2 \leqq k \leqq \infty $, or $k = \omega $, there exists a completely controllable set $S$ consisting of two vector fields of class $C^{k - 1} $.
Close

Your Recent History Recent History Help

Recently Viewed

  1. Simulating Size-constrained Galton–Watson Trees
  2. Locally Testable Codes Require Redundant Testers
  3. Unique Games with Entangled Provers Are Easy
  4. Programmable Control of Nucleation for Algorithmic Self-Assembly
  5. Faster Integer Multiplication
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close