Practical Methods for Optimal Control and Estimation Using Nonlinear Programming, Second Edition

The front matter includes the title page, series page, copyright page, dedication, TOC, and preface

1.1 Preliminaries

This book concentrates on numerical methods for solving the optimal control problem. The fundamental principle of all effective numerical optimization methods is to solve a difficult problem by solving a sequence of simpler subproblems. In particular, the solution of an optimal control problem will require the solution of one or more finite-dimensional subproblems. As a prelude to our discussions on optimal control, this chapter will focus on the nonlinear programming (NLP) problem. The NLP problem requires finding a finite number of variables such that an objective function or performance index is optimized without violating a set of constraints. The NLP problem is often referred to as parameter optimization. Important special cases of the NLP problem include linear programming (LP), quadratic programming (QP), and least squares problems.

Before proceeding further, it is worthwhile to establish the notational conventions used throughout the book. This is especially important since the subject matter covers a number of different disciplines, each with its own notational conventions. Our goal is to present a unified treatment of all these fields. As a rule, scalar quantities will be denoted by lowercase letters (e.g., α). Vectors will be denoted by boldface lowercase letters and will usually be considered column vectors, as in x= [ x1 x2 ⋮ xn ] , 1.1 where the individual components of the vector are x_k for k = 1,…, n. To save space, it will often be convenient to define the transpose, as in x⊤ = ( x1 , x2 ,…, xn ) . 1.2 A sequence of vectors will often be denoted as x_k,x_k+1,….

2.1 Overview: Large, Sparse NLP Issues

Chapter 1 presents background on nonlinear programming (NLP) methods that are applicable to most problems. In this chapter, we will focus on NLP methods that are appropriate for problems that are both large and sparse. To set the stage for what follows, it is useful to define what we mean by large and sparse. The definition of “large” is closely tied with the tools available for solving linear systems of equations. For our purposes, we will consider a problem “small” if the underlying linear systems can be solved using a dense, direct matrix factorization. Solving a dense linear system using a direct factorization requires O ( n3 ) arithmetic operations. Thus, for today's computers, this suggests that the problem size is probably limited to matrices of order n ≈ 1000. If the linear equations are solved using methods that exploit sparsity in the matrices but still use direct matrix factorizations, then with current computer hardware, the upper limit on the size of the problem is n ≈ 10⁶. This problem is considered “large” and is the focus of methods in this book. Although some of the NLP methods can be extended, as a rule we will not address “huge” problems for which n > 10⁶. Typically, linear systems of this size are solved by iterative (as opposed to direct) methods. The second discriminator of interest concerns matrix sparsity. A matrix is said to be “sparse“ when many of the elements are zero. For most applications of interest, the number of nonzero elements in the Hessian matrix H and Jacobian matrix G is less than 1%.

Many of the techniques described in Chapter 1 extend, without change, to large, sparse problems. On the other hand, there are some techniques that cannot be used for large, sparse applications. In this chapter, we will focus on those issues that are unique to the large, sparse NLP problem.

The first item concerns how to calculate Jacobian and Hessian information for large, sparse problems. All of the Newton-based methods described in Chapter 1 rely on the availability of first and second derivative information. For most practical applications, first derivatives are computed using finite difference approximations. Unfortunately, a single finite difference gradient evaluation can require n additional function evaluations, and when n is large, this computational cost can be excessive.

3.1 The Transcription Method

The preceding chapters focus on methods for solving NLP problems. In the remainder of the book, we turn our attention to the optimal control problem. An NLP problem is characterized by a finite set of variables x and constraints c. In contrast, optimal control problems can involve continuous functions such as y(t) and u(t). It will be convenient to view the optimal control problem as an infinite-dimensional extension of an NLP problem. However, practical methods for solving optimal control problems require Newton-based iterations with a finite set of variables and constraints. This goal can be achieved by transcribing or converting the infinite-dimensional problem into a finite-dimensional approximation.

Thus, the transcription method has three fundamental steps:

1. convert the dynamic system into a problem with a finite set of variables; then

2. solve the finite-dimensional problem using a parameter optimization method (i.e., the NLP subproblem); and then

3. assess the accuracy of the finite-dimensional approximation and if necessary repeat the transcription and optimization steps.

We will begin the discussion by focusing on the first step in the process, namely identifying the NLP variables, constraints, and objective function for common applications. In simple terms, we will focus on how to convert an optimal control problem into an NLP problem.

3.2 Dynamic Systems

A dynamic system is usually characterized mathematically by a set of ordinary differential equations (ODEs).

4.1 Introduction

4.1.1 Dynamic Constraints

The optimal control problem may be interpreted as an extension of the NLP problem to an infinite number of variables. For fundamental background in the associated calculus of variations, the reader should refer to Bliss [42]. First, let us consider a simple problem with a single phase and no path constraints. Specifically, suppose we must choose the control functions u(t) to minimize J=ϕ [y ( tF ), tF ] 4.1 subject to the state equations y ˙ =f [y (t),u (t) ] 4.2 and the boundary conditions ψ [y ( tF ),u ( tF ), tF ]=0, 4.3 where the initial conditions y(t_I) = y_I are given at the fixed initial time t_I and the final time t_F is free. This is a very simplified version of the problem (called an autonomous system) that will be addressed later in the chapter. We have intentionally chosen a problem with only equality constraints. However, in contrast to the discussion in Chapters 1 and 2, we now have two types of constraints. The equality constraint (4.2) may be viewed as “continuous” since it must be satisfied over the entire interval t_I ≤t ≤ t_F, whereas the equality (4.3) may be viewed as “discrete” since it is imposed at the specific time t_F. In a manner analogous to the definition of the Lagrangian function (1.55), we form an augmented performance index J ^ = [ϕ+ ν⊤ ψ] tF − ∫ tI tF λ⊤ (t) { y ˙ −f [y (t),u (t) ] } dt. 4.4 Notice that, in addition to the Lagrange multipliers v for the discrete constraints, we also have multipliers λ(t), referred to as adjoint or costate variables for the continuous (differential equation) constraints.

5.1 Introduction

The behavior of many physical processes can be described mathematically by ordinary differential or differential-algebraic equations. Commonly a finite number of parameters appear in the description of the system dynamics. A parameter estimation problem arises when it is necessary to compute values for these parameters based on observations of the system dynamics. Methods for solving these so-called inverse problems have been used for many years [155]. In fact, most techniques in use today are based on ideas proposed by Gauss nearly 200 years ago that he used to solve orbit determination problems.

Traditional methods for solving inverse problems usually combine a standard initial-value numerical integration technique with a Gauss—Newton method for optimization. An approach that uses neither of the traditional processes can be constructed using the same techniques introduced in Chapter 4 for solving optimal control problems. Initial value integration is replaced by a discretization of the relevant differential-algebraic equations. We then exploit sparse finite difference approximations to the Hessian matrix, which permits us to construct a quadratically convergent algorithm for solving parameter estimation problems.

5.2 The Parameter Estimation Problem

Typically the dynamics of the system are defined for t_I ≤ t ≤ t_F by a set of ODEs written in explicit form given by (4.32) and restated here as y ˙ =f [y (t),u (t),p,t ] , 5.1 where y is the n_y dimension state vector, and u is an n_u dimension vector of algebraic variables.

6.1 Space Shuttle Reentry Trajectory

Construction of the reentry trajectory for the space shuttle is a classic example of an optimal control problem. The problem is of considerable practical interest and is nearly intractable using a simple shooting method because of its nonlinear behavior. Early results were presented by Bulirsch [55] on one version of the problem, as well by Dickmanns [73]. Ascher, Mattheij, and Russell present a similar problem [2, p. 23] and Brenan, Campbell, and Pet-zold discuss a closely related path control problem [48, p. 157]. Let us consider a particular variant of the problem originally described in [175].

The motion of the vehicle is defined by the following set of DAEs: h ˙ =υ sin γ, 6.1 ϕ ˙ = υ r cos γ sin ψ/cos θ, 6.2 θ ˙ = υ r cos γ cos ψ, 6.3 υ ˙ =− D m −g sin γ, 6.4 γ ˙ = L mυ cos (β) +cos γ ( υ r − g υ ) , 6.5 ψ ˙ = 1 m υ cos γ L sin (β) + υ r cos θ cos γ sin ψ sin θ, 6.6 q≤ qU , 6.7 where the aerodynamic heating on the vehicle wing leading edge is q = q_aq_r and the dynamic variables are

h altitude (ft), γ flight path angle (rad),

ϕ longitude (rad), ψ azimuth (rad),

θ latitude (rad), α angle of attack (rad),

υ velocity (ft/sec), β bank angle (rad).

Most effective numerical methods are based on the following premise:

A hard problem can be broken into a sequence of easy subproblems.

For example, to compute the root of a nonlinear constraint c(x) = 0 using Newton's method, one must solve the sequence of linear approximations x_k+1 = x_k − c(x_k)/c′(x_k). Similarly, when using a nonlinear programming (NLP) algorithm to solve a nonlinear optimization problem, one solves a sequence of (a) quadratic programming subproblems (an SQP method) or (b) unconstrained subproblems (a barrier method). In like fashion an optimal control solution can be obtained by solving a sequence of NLP subproblems. Of course there may be no clear definition of an “easy subproblem.” For Newton's method is it better to compute c′(x_k) or use a secant approximation? Is a quadratic program easier than an unconstrained subproblem? Ultimately the solution technique should consist of a sequence of subproblems that can be solved efficiently and reliably.

This chapter presents a number of advanced applications that require using a combination of the techniques presented in the book.

7.1 Optimal Lunar Swingby Trajectories

7.1.1 Background and Motivation

With the growth of the space age, it has become desirable to place spacecraft into a wide variety of mission orbits. Typically a launch vehicle is used to deploy a satellite into a low altitude park orbit. It then becomes necessary to transfer from the park orbit to the mission orbit. A major goal of mission design is to compute this orbit transfer to minimize some performance objective. The most common objective is to minimize the fuel consumed by the orbit transfer. For spacecraft using propulsion systems whose thrust is high compared to the mass of the vehicle, i.e., so-called high thrust systems, the duration of the burns is very short compared to the duration of the orbit transfer itself. Consequently it is common to model high thrust systems using an instantaneous or impulsive velocity change. In essence a minimum fuel orbit transfer is approximated using a so-called minimum Δυ transfer.

Solving a problem in optimal control or estimation is not easy. Pieces of the puzzle are found scattered throughout many different disciplines. At the very least, one needs an “optimization” method and a “differential equation” method. A rudimentary understanding of modern control theory is helpful, to say nothing of expertise in the domain of application. In my experience, the most challenging practical problems originate with experts in a particular domain. An aerodynamicist feels comfortable discussing “lift” and “drag” but is less familiar with index-two DAEs. Chemical engineers can readily describe a batch feed process but become uneasy when discussing the KKT conditions. Often this domain expertise has been painstakingly developed over many years and involves complex computer simulations and/or expensive experimental studies.

Thus, when faced with an optimal control or estimation problem it is tempting for the domain expert (i.e., the structural engineer, biochemist, etc.) to simply “paste” together packages for optimization and numerical integration. While naive approaches such as this may be moderately successful, the goal of this book is to suggest that there is a better way! The methods used to solve the differential equations and optimize the functions are intimately related. Furthermore design and development of the “simulation” or experimental trial should be done with the intent to optimize. Optimization is not an afterthought. Indeed, perhaps the most important issue needed to successfully solve a problem is the proper formulation. And so I close with the following:

THEERUM

If there is a flaw in the problem formulation, the optimization algorithm will find it.

All of the algorithms described in the book have been implemented and tested on realistic test problems. In addition, all of the examples presented have been solved using this software, and FORTRAN implementations of the examples can be obtained by contacting the author. The SOCS [38] library, which contains all of the software, has a great deal of functionality and it is not intended that this appendix should be viewed as a user's manual. On the other hand, it is worthwhile to outline the basic capability of the tool to assist the reader in the formulation and solution of practical problems. To that end, the following sections present a brief overview of the library. Additional information can be obtained by contacting the author at [email protected].

A.1 Simplified Usage Dense NLP

The subroutine HDNLPD provides the most basic functionality for solving small, dense NLP problems. The primary information that must be supplied by the user (outlined with a double box in Figure A.1) is a subroutine called FUNBOX that evaluates the objective and constraint functions. This is the function generator described in Section 3.8. All algorithm parameters are given default values, which are appropriate for a simple problem. If desired, the default values can be redefined using a utility routine HHSNLP that is common to all software in the SOCS library. The simplified usage software uses a forward communication format and, as such, the optimization algorithm HDNLPD calls the user-supplied subroutine, which has a fixed calling argument list. For more sophisticated applications, the reverse-communication subroutine HDNLPR can be used. HDNLPR is appropriate for small, dense applications when the user can supply gradient and Hessian information. It is also appropriate for use with complicated simulation programs and when parallel processing is used.

A.2 Sparse NLP with Sparse Finite Differences

Large, sparse NLP problems necessarily demand more information from the user because of the need to specify matrix sparsity and provide corresponding derivative information.

The back matter includes Bibliography and Index