SIAM Journal on Control and Optimization

This paper is concerned with the properties of the sets of strategic measures induced by admissible team policies in decentralized stochastic control and the convexity properties in dynamic team problems. To facilitate a convex analytical approach, strategic measures for team problems are introduced. Properties such as convexity, compactness, and Borel measurability under weak convergence topology are studied, and sufficient conditions for each of these properties are presented. These lead to existence of and structural results for optimal policies. It will be shown that the set of strategic measures for teams which are not classical is in general nonconvex, but the extreme points of a relaxed set consist of deterministic team policies, which lead to their optimality for a given team problem under an expected cost criterion. Externally provided independent common randomness for static teams or private randomness for dynamic teams do not improve the team performance. The problem of when a sequential team problem is convex is studied and necessary and sufficient conditions for problems which include teams with a nonclassical information structure are presented. Implications of this analysis in identifying probability and information structure dependent convexity properties are presented.

We show the appearance of an essentially nonlocal dynamics describing the limit behavior of trajectories of a class of dynamical systems defined by classical autonomous ODEs with smooth right-hand sides containing a small parameter and becoming discontinuous in the formal limit. The limit dynamics is shown to be described by an explicitly constructed Nerode--Kohn hybrid dynamical system consisting of a continuous plant (an ODE) and a finite state machine which are interacting and producing hybrid dynamics with possible memory effects. We remark, however, that, from the “statistical” point of view, the limit behavior of an ensemble of trajectories can still be described by an ODE, with possibly time-dependent and discontinuous right-hand side depending on the chosen ensemble.

For the heat equation on a bounded subdomain $\Omega$ of ${\mathbb{R}^d}$, we investigate the optimal shape and location of the observation domain in observability inequalities. A new decomposition of $L^2({\mathbb{R}^d})$ into heat packets allows us to remove the randomization procedure and assumptions on the geometry of $\Omega$ in previous works. The explicit nature of the heat packets gives new information about the observability constant in the inverse problem.

In this paper, we consider a rather general linear evolution equation of fractional type, namely, a diffusion type problem in which the diffusion operator is the $s$th power of a positive definite operator having a discrete spectrum in ${\mathbb R}^+$. We prove existence, uniqueness, and differentiability properties with respect to the fractional parameter $s$. These results are then employed to derive existence as well as first-order necessary and second-order sufficient optimality conditions for a minimization problem, which is inspired by considerations in mathematical biology. In this problem, the fractional parameter $s$ serves as the “control parameter” that needs to be chosen in such a way as to minimize a given cost functional. This problem constitutes a new class of identification problems: while usually in identification problems the type of the differential operator is prescribed and one or several of its coefficient functions need to be identified, in the present case one has to determine the type of the differential operator itself. This problem exhibits the inherent analytical difficulty that with changing fractional parameter $s$ also the domain of definition, and thus the underlying function space, of the fractional operator changes.

We revisit the classical monotone-follower problem and consider it in a generalized formulation. Our approach is based on a compactness substitute for nondecreasing processes, the Meyer--Zheng weak convergence, and the maximum principle of Pontryagin. It establishes existence under weak conditions, produces general approximation results, and further elucidates the celebrated connection between singular stochastic control and stopping.

This article is concerned with the exponential stability and the uniform propagation of chaos properties of a class of extended ensemble Kalman--Bucy filters with respect to the time horizon. This class of nonlinear filters can be interpreted as the conditional expectations of nonlinear McKean--Vlasov-type diffusions with respect to the observation process. We consider filtering problems with Langevin-type signal processes observed by some noisy linear and Gaussian-type sensors. In contrast with more conventional Langevin nonlinear drift type processes, the mean field interaction is encapsulated in the covariance matrix of the diffusion. The main results discussed in the article are quantitative estimates of the exponential stability properties of these nonlinear diffusions. These stability properties are used to derive uniform and nonasymptotic estimates of the propagation of chaos properties of extended ensemble Kalman filters, including exponential concentration inequalities. To our knowledge these results seem to be the first results of this type for this class of nonlinear ensemble type Kalman--Bucy filters.

We extend a formula for the computation of the shape derivative of an integral cost functional with respect to a class of convex domains, using the so-called support functions and gauge functions to express it. This is a priori a formula in shape optimization theory. However, the result also happens to be an extension of a well-known formula from the Brunn--Minkowski theory of convex bodies.

Formation control deals with the design of decentralized control laws that stabilize mobile, autonomous agents at prescribed distances from each other. We call any configuration of the agents a target configuration if it satisfies the interagent distance conditions. It is well known that when the distance conditions are defined by a rigid graph, there is a finite number of target configurations modulo rotations and translations of the entire formation. We can thus recast the objective of formation control as stabilizing one or many of the target configurations. A major issue is that such control laws will also have equilibria corresponding to configurations which do not meet the desired interagent distance conditions; we refer to these as undesirable configurations. The undesirable configurations become problematic if they are also stable. Designing decentralized control laws whose stable equilibria are all target configurations in the case of a general rigid graph is still an open problem. We provide here a new point of view on this problem and propose a partial solution by exhibiting a class of rigid graphs and control laws for which all stable equilibria are target configurations.

We consider the class of spatially decaying systems, where the underlying dynamics are spatially decaying and the sensing and controls are spatially distributed. This class of systems arises in various applications where there is a notion of spatial distance with respect to which couplings between the subsystems can be quantified using a class of coupling weights. We exploit the spatial decay property of the underlying dynamics of this class of systems by introducing a class of sparsity and spatial localization measures. We develop a new methodology based on concepts of $q$-Banach algebras of spatially decaying matrices that enables us to establish a relationship between spatial decay properties of spatially decaying systems and their sparsity and spatial localization features. Moreover, it is shown that the inverse-closedness property of matrix algebras plays a central role in exploiting various structural properties of spatially decaying systems. We characterize conditions for exponential stability of spatially decaying system over $q$-Banach algebras and prove that the unique solutions of the Lyapunov and Riccati equations over a proper $q$-Banach algebra also belong to the same $q$-Banach algebra. It is shown that the quadratically optimal state feedback controllers for spatially decaying systems are sparse and spatially localized in the sense that they have near-optimal sparse information structures.

A linear time-invariant singularly perturbed system with multiple pointwise and distributed small time delays is considered. A novel (direct) approach to exact slow-fast decomposition of this system is proposed. In contrast with the existing method, this approach uses neither a preliminary transformation of the original differential system to an integral one nor rather complicated integral manifold and operator techniques. Moreover, the approach of the present paper does not assume the exponential stability of the fast subsystem. Based on this decomposition, an exact slow-fast decomposition of the spectrum of a singularly perturbed system with a single pointwise small delay is carried out. Using the theoretical results, the stability of a multilink single-sink optical network is analyzed.

In this paper, the synchronization of discrete-time linear multiagent systems is studied with finite communication data rate and switching topology flows. A class of quantized-observer-based communication schemes and a class of certainty-equivalence-principle-based cooperative control laws are proposed with adaptive encoders and decoders. It is shown that if the pairs of agents' state matrices and control matrices multiplied by Laplacian eigenvalues of the weakly connected components are simultaneously stabilizable, and the communication topology flow is frequently connected, then there exist such protocols leading to synchronization exponentially fast. Furthermore, only finite bits of information exchange per step are required to guarantee the synchronization if the communication channels are frequently active. For first-order dynamics, the dwell time and the number of bits are both related to the unstable mode of agent dynamics, the number of agents, the frequency of connectivity, and the Laplacian eigenvalue ratio of the switching topology flow.

In this paper we investigate the exact controllability of $n \times n$ first order one-dimensional quasi-linear hyperbolic systems by $m<n$ internal controls that are localized in space in some part of the domain. We distinguish two situations. The first one is when the equations of the system have the same speed. In this case, we can use the method of characteristics and obtain a simple and complete characterization for linear systems. Thanks to a linear test this also provides some sufficient conditions for the local exact controllability around the trajectories of semilinear systems. However, when the speed of the equations is not the same, we see that we encounter the problem of loss of derivatives if we try to control quasi-linear systems with a reduced number of controls. To solve this problem, as in a prior article by Coron and Lissy on a Navier--Stokes control system, we first use the notion of algebraic solvability due to Gromov. However, in contrast with this prior article where a standard fixed point argument could be used to treat the nonlinearities, we use here a fixed point theorem of Nash--Moser type due to Gromov in order to handle the problem of loss of derivatives.

We consider the wave equation with mixed boundary conditions in a bounded domain; on one portion of the boundary, we have dynamic Wentzell boundary conditions, and on the other portion, we have homogeneous Dirichlet boundary conditions. First, using an appropriate geometric partition of the boundary, we prove some Carleman estimates for this system. Then, we apply those estimates to prove a boundary controllability result for a nonconservative model of the system under consideration. Our results improve earlier Carleman estimates and boundary controllability results established in the Dirichlet boundary conditions setting.

Motivated by a control problem of a certain queueing network we consider a control problem where the dynamics is constrained in the nonnegative orthant $\mathbb{R}^{d}_{+}$ of the $d$-dimensional Euclidean space and controlled by the reflections at the faces/boundaries. We define a discounted value function associated to this problem and show that the value function is a viscosity solution to a certain HJB equation in $\mathbb{R}^{d}_{+}$ with nonlinear Neumann type boundary condition. Under certain conditions, we also characterize this value function as the unique solution to this HJB equation.

This paper explores the deliberate introduction of delays in feedback loops and their positive impact on linear time-invariant, single-input, single-output (LTI-SISO) systems. The delay, considered as a design parameter, is intentionally induced using a class of delay-based controllers, aiming at improving closed-loop system performance by deploying a pole-placement technique to attain the fastest velocity of response to control commands. The technique, based on an algebraic geometric analysis, is able to extract exact tuning formulas for the controller gains and the delay.

The nonlinear diffusion equation $\rho_t-\Delta(\rho H(\rho-\rho_c))=0$ in $(0,\infty)\times \mathbb{R}^n$, $\rho(0,x)=\rho_0(x)$, where $H$ is the Heaviside function, describes the sandpile model [P. Bak, C. Tang, and K. Wiesenfeld, Phys. Rev. A (3), pp. 364--374; P. Bantay and I. M. Janosi, Phys. A, 185 (1992), pp. 11--18; R. Cafiero et al., Europhys. Lett., 29 (1995), pp. 111--116] with critical state $\rho_c\in L^\infty(\mathbb{R}^n)\cap L^1(\mathbb{R}^n)$. Here, one proves that a solution $\rho=\rho(t,x)$ can be obtained as the limit of the time-stepping approximation scheme associated with the variational problem $\rho_k={\rm arg}\,\min_{\rho\in\mathcal{P}}\{\frac1h\ d^2(\rho_{k-1},\rho)+\mathbb{E}(\rho)\}$, where $d$ is the $2$-Wasserstein distance and $\mathbb{E}$ is the energy functional corresponding to the above nonlinear diffusion process. This result is on the line of that previously obtained for the linear Fokker--Planck and porous media equations by Jordan, Kinderlehrer, and Otto [SIAM J. Math. Anal., 29 (1998), pp. 1--17] and Otto [Arch. Ration. Mech. Anal., 141 (1998), pp. 63--103; Comm. Partial Differential Equations, 26 (2001), pp. 101--174].

This paper investigates social optima of mean field linear-quadratic-Gaussian (LQG) control models with Markov jump parameters. The common objective of the agents is to minimize a social cost---the cost average of the whole society. In the cost functions there are coupled mean field terms. First, we consider the centralized case and get a parameterized equation of mean field effect. Then, we design a set of distributed strategies by solving a limiting optimal control problem in an augmented state space subject to the consistency requirement for mean field approximation. It is shown that the set of distributed strategies is asymptotically team-optimal, and the asymptotically optimal social cost value can be obtained explicitly. The optimal social average cost is compared with the optimal individual cost in mean field games by virtue of the explicit expressions, and the difference is further illustrated by a numerical example.

A common approach to controlling complex networks is to directly control a subset of input nodes, which then controls the remaining nodes via network interactions. Current approaches for selecting input nodes assume that either all system matrix entries are known and fixed, or are independent free parameters, and focus either on performance or controllability. In this paper, we make two contributions towards input selection in networked systems. First, we propose polynomial-time algorithms for input selection in structured linear descriptor systems, which are systems with dependencies between free parameters due to physical laws or design constraints. Second, we develop a framework for input selection based on joint consideration of controllability and performance. We make both contributions by mapping input selection to a submodular optimization problem under two matroid constraints, which enables development of polynomial-time algorithms with provable optimality guarantees. We provide improved optimality guarantees for special cases such as strongly connected networks, consensus networks, double integrators, and networks where all system parameters can take any arbitrary real values.

This paper considers continuously differentiable functions of two vector variables that have (possibly a continuum of) min-max saddle points. We study the asymptotic convergence properties of the associated saddle-point dynamics (gradient descent in the first variable and gradient ascent in the second one). We identify a suite of complementary conditions under which the set of saddle points is asymptotically stable under the saddle-point dynamics. Our first set of results is based on the convexity-concavity of the function defining the saddle-point dynamics to establish the convergence guarantees. For functions that do not enjoy this feature, our second set of results relies on properties of the linearization of the dynamics, the function along the proximal normals to the saddle set, and the linearity of the function in one variable. We also provide global versions of the asymptotic convergence results. Various examples illustrate our discussion.

Stabilization problems for parabolic equations with polynomial nonlinearities are investigated in the context of an optimal control formulation with a sparsity enhancing cost functional. This formulation allows that the optimal control completely shuts down once the trajectory is sufficiently close to a stable steady state. Such a property is not present for commonly chosen control mechanisms. To establish these results it is necessary to develop a function space framework for a class of optimal control problems posed on infinite time horizons, which is otherwise not available.

A recent work by Mazenc and Malisoff provides a trajectory-based approach for proving stability of time-varying systems with time-varying delays. Here, we provide several significant applications of their approach. In two results, we use a Lyapunov function for a corresponding undelayed system to provide a new method for proving stability of linear continuous-time time-varying systems with bounded time-varying delays. We allow uncertainties in the coefficient matrices of the systems. Our main results use upper bounds on an integral average involving the delay. The results establish input-to-state stability with respect to disturbances. We also provide a novel reduction model approach that ensures global exponential stabilization of linear systems with a time-varying pointwise delay in the input, which allows the delay to be discontinuous and uncertain. Finally, we provide an alternative to the reduction model method, based on a different dynamic extension. Our examples demonstrate the usefulness of our findings in several settings.

We present a novel method for solving a class of time-inconsistent optimal stopping problems by reducing them to a family of standard stochastic optimal control problems. In particular, we convert an optimal stopping problem with a nonlinear function of the expected stopping time in the objective into optimization over an auxiliary value function for a standard stochastic control problem with an additional state variable. This approach differs from those in the previous literature which primarily employ Lagrange multiplier methods or rely on exact solutions. In contrast, we characterize the auxiliary value function as the unique viscosity solution of a nonlinear elliptic PDE which satisfies certain growth constraints, and investigate basic regularity properties. We demonstrate a connection between optimal stopping times for the original problem and optimal controls of the auxiliary control problem. Finally, we discuss extensions to more general dynamics and time-inconsistencies, as well as potential connections to degenerate Monge--Ampère equations.

This paper considers a structured separable convex optimization problem, motivated by the deployment of model predictive control on multiagent systems that are interacting via nondelayed couplings. We show that the dual decomposition of this problem yields a numerical structure in the Hessian of the dual function. This numerical structure allows for deploying a quasi-Newton method in the dual space. For large problems, this approach yields a large reduction of the computational complexity of solving the problem, and for geographically distributed problems a reduction in the communication burden.