SIAM Digital Library
 
 
 

You are not logged in Logged Out Log In

SIAM J. on Control and Optimization / Volume 50 / Issue 1

SIAM J. Control Optim. 50, pp. 23-47 (25 pages)

Linear Programming and Constrained Average Optimality for General Continuous-Time Markov Decision Processes in History-Dependent Policies

Xianping Guo, Yonghui Huang, and Xinyuan Song

Full Text: Download PDF | Buy PDF (US$25) | View Cart
This paper attempts to study the constrained average optimality for continuous-time Markov decision processes in the class of randomized history-dependent policies. The states and actions are in general Polish spaces, and the transition rates are allowed to be unbounded. The optimality criterion to be optimized is expected average costs, multiple constraints are imposed on similar expected average costs, and all costs may be unbounded from above and from below. Under suitable conditions, we first show the existence of a constrained optimal policy by improving the concept of a stable policy in the previous literature and using the analogue of the forward Kolmogorov equation. Then, we develop a linear program (LP), which is equivalent to the constrained optimality problem and is used to obtain a constrained optimal policy. By introducing suitable operators and conditions, we further establish the dual program (DP) of the LP, show that the LP and DP are solvable, and show that there is no duality gap between them. Finally, we use a cash flow model and a controlled birth and death system to illustrate the applications of our main results.

© 2012 Society for Industrial and Applied Mathematics

RELATED DATABASES

To view database links for this article, you need to log in.

KEYWORDS

AMS Subject Headings

90C40, 60J27

PUBLICATION DATA

ISSN

0363-0129 (print)  
1095-7138 (online)

Publisher

$abstract.society.value is a Member of CrossRef Society for Industrial and Applied Mathematics

ARTICLE DATA

History
Received August 12, 2010
Accepted September 27, 2011
Published online January 03, 2012
Digital Object Identifier
http://dx.doi.org/10.1137/100805169

For access to fully linked references, you need to log in.

Close

Your Recent History Recent History Help

Recently Viewed

  1. Linear Programming and Constrained Average Optimality for General Continuous-Time Markov Decision Pr...
  2. Some Compact Classes of Open Sets under Hausdorff Distance and Application to Shape Optimization
  3. A Lyapunov Inequality Characterization of and a Riccati Inequality Approach to $L_{\infty}$ and $L_{...
  4. Asynchronous Rendezvous Analysis via Set-valued Consensus Theory
  5. On the Lavrentiev Phenomenon for Optimal Control Problems with Second-Order Dynamics
© SIAM By using SIAM Publications Online you agree to abide by the Terms and Conditions of Use. Banner art adapted from a figure by Hinke M. Osinga and Bernd Krauskopf (University of Bristol, UK).
close