Computational Methods in Optimal Control: Theory and Practice

The front matter includes the Title page, Author page, Copyright page, Table of Contents, List of Figures, List of Tables, and Preface.

1.1 ■ Some History

Optimal control is a field that emerged in the 1950s from the much older calculus of variations. Some of the first problems from the calculus of variations include Newton's minimal resistance problem (1687) [53]: find the solid of revolution that experiences the smallest resistance when moving through a fluid in the direction of its axis of rotation at constant velocity; or Bernoulli's brachistochrone problem (1696) [8]: find the curve of fastest descent that connects two points when a ball falls frictionlessly under the force of gravity. The first-order optimality conditions for a problem from the calculus of variations often have the property that their solutions are relatively smooth. The simplest example is the problem where f : [0, 1] → ℝ lies in ℒ²([0, 1]), the space of square integrable functions, and the minimization is over y ∈ ℋ₀¹([0, 1]), the space of functions with a square integrable derivative that satisfy the boundary conditions y (0) = y (1) = 0.

2.1 ■ The Abstract Setting

The principal focus of this book is the convergence of numerical algorithms for solving optimal control problems. The optimal control problem is formulated in an infinite-dimensional space of controls and states that are functions of time. The numerical algorithms are formulated in a finite-dimensional, discrete space. The goal is to prove convergence of solutions of the discrete problem to a solution of the continuous problem. This chapter provides the mathematical foundation that is used to establish convergence. The setting where we work is the following: Our numerical algorithm involves two spaces, a solution space denoted 𝒳 and a second space 𝒴 used to formulate constraints on the solution. The solution space 𝒳 is a Banach space, while 𝒴 is a normed linear space. A norm is a map, which we denote by || · ||, into the nonnegative real numbers that has the following three properties:

1. ||y || ≥ 0 for all y ∊ 𝒴 and ||y || = 0 if and only if y = 0.

2. ||αy || = |α|||y || for all y ∊ 𝒴 and scalars α.

3. ||x + y || ≤ ||x || + ||y || for all x and y ∊ 𝒴.

3.1 ■ First-Order Optimality Conditions

To analyze Runge-Kutta integration schemes in optimal control, it is necessary to develop the connection between the first-order optimality conditions for the continuous control problem (the Pontryagin Minimum Principle) and the first-order optimality conditions for the discrete approximating problem (the Karush-Kuhn-Tucker or KKT conditions). William Karush developed the first-order optimality conditions for finite-dimensional optimization problems in his December 1939 Master's thesis [47] at the University of Chicago, under the direction of Lawrence Murray Graves, a mathematician working in the calculus of variations. The thesis, however, was not published as a journal article, and Karush was soon involved in the Manhattan Project, which was racing to beat the Nazis to develop an atom bomb. Kuhn and Tucker, professors at Princeton, published the first-order conditions for finite-dimensional optimization in the proceedings of a Berkeley symposium in 1950 [48], unaware of Karush's thesis. The first-order optimality conditions were initially known as the KT conditions, until Karush's thesis was uncovered and the acronym was updated to include the initial K. The connection between the first-order optimality conditions for the continuous control problem and its discrete approximation is first developed in the context of the control constrained problem (1.21), while the connection is extended to include terminal constraints and singular controls in Chapter 6.

This chapter uses Theorem 2.8 to analyze the convergence of Runge-Kutta discretizations of optimal control problems. The generalized equation 𝒯 (θ) ∈ ℱ (θ) is the first-order optimality condition for a solution of the Runge-Kutta discretization. The variable θ in the theorem corresponds to the triple (X, U, Λ), where the components of X are the states x_k and the intermediate states y_kj, the components of U are the controls u_kj, and the components of Λ are the costates p_k and the intermediate costates such as λ_kj or the transformed variables q_kj. The generalized structure of the equation 𝒯 (θ) ∈ ℱ (θ) arises from the control minimum principle, which is formulated as an inequality. For the symplectic Runge-Kutta schemes, the minimum principle was given in (3.78), which is repeated here for convenience: Using the normal cone, defined in (2.4), this condition is reformulated as the generalized equation There is one generalized equation for each discrete control u_kj and the associated components of ℱ(θ) are the sets -N_𝒰 (u_kj). The remaining components of ℱ, associated with the discrete state or the discrete costate dynamics, are the vector 0. In the context of Theorem 2.8, the bounds for the local truncation errors obtained in the previous chapter represent bounds for the distance from components of 𝒯(θ*) to the associated components of ℱ (θ*). The reference point θ* was the continuous optimal state, costate, and control evaluated at mesh points or other time instants. In this chapter, we introduce a linear operator L for each scheme that is analyzed, where L approximates the Fréchet derivative ∇ 𝒯(θ*). We show that (ℱ - L)^-1 is single-valued and Lipschitz continuous. Combining the local truncation error bounds with these new results, Theorem 2.8 yields an ∞-norm error bound for the Runge-Kutta discretized control problem.

5.1 ■ Polynomial Approximations

Runge-Kutta methods based on a single fixed tableau are known as h methods since convergence is achieved by letting h tend to zero. In a p-method, the state is approximated by a polynomial and convergence is achieved by letting p increase. A p-method can potentially converge extremely quickly when the state is smooth. The solutions to optimal control problems, however, are often nonsmooth, with potential jumps in the control or its derivative at points where the active constraints change, while the control is smooth between the jump points. If we knew the location of the jump points, then we could use a different polynomial over each of the intervals where the solution is smooth, and achieve rapid convergence by simply increasing the polynomial degree. But typically, finding the jump locations is part of the optimization problem. This leads us to consider an hp implementation where the problem domain is partitioned into subintervals, with a different polynomial on each subinterval, and the number of subintervals, their boundary points, and the degree of the polynomials on each subinterval are adjusted in an effort to exploit the rapid convergence that can be achieved by increasing the polynomial degree on the intervals where the solution is smooth. In this chapter, the hp framework is developed. The software package 𝔾ℙ𝕆ℙ𝕊-𝕀𝕀 [2, 54, 56, 63, 64] employs this hp framework.

In this chapter, our problem formulation is generalized in two ways, first to include problems with terminal constraints, and then to develop strategies for solving problems with discontinuous controls by optimizing over the location of the discontinuities.

Let us consider our standard control problem for almost every t ∈ [0, T ], where x : [0, T ] → ℝⁿ, f : ℝ^n+m → ℝⁿ, u : [0, T ] → ℝ^m is bounded and measurable, and 𝒰⊂ℝ^m. We assume the following: (A1) 𝒰 is a closed and bounded set (compact set). (A3) (strengthened version of (A2) in Section 1.5) f is uniformly Lipschitz continuous on ℝⁿ × 𝒰. That is, there exists a constant L such that for all x₁, x₂ ∈ ℝⁿ and u₁, u₂ ∈ 𝒰. Moreover, ∇_x f (x, u) is continuous in x, uniformly for (x, u) ∈ ℬ_β (0) × 𝒰, where β is defined in (1.30). By taking u ₂ = u₁ ∈ 𝒰 in (A3), we see that (A2) holds, which implies that the dynamics of (A.1) has a solution for any feasible u and the solution lies in ℬ_β (0).

The back matter includes bibliography, index, and backcover.