Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics: A Tribute to Michael K. Sain (Systems & Control: Foundations & Applications)

Systems & Control: Foundations & Applications Series Editor Tamer Bas¸ar, University of Illinois at Urbana-Champaign Editorial Board ˚ om, Lund University of Technology, Lund, Sweden Karl Johan Astr¨ Han-Fu Chen, Academia Sinica, Beijing William Helton, University of California, San Diego Alberto Isidori, University of Rome (Italy) and Washington University, St. Louis Petar V. Kokotovi´c, University of California, Santa Barbara Alexander Kurzhanski, Russian Academy of Sciences, Moscow and University of California, Berkeley H. Vincent Poor, Princeton University Mete Soner, Koc¸ University, Istanbul

Chang-Hee Won Cheryl B. Schrader Anthony N. Michel

Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics A Tribute to Michael K. Sain

Birkh¨auser Boston • Basel • Berlin

Chang-Hee Won Temple University Department of Electrical & Computer Engineering Philadelphia, PA 19122 USA [email protected]

Cheryl B. Schrader Boise State University College of Engineering Boise, ID 83725-2100 USA [email protected]

Anthony N. Michel University of Notre Dame Department of Electrical Engineering Notre Dame, IN 46556-5637 USA [email protected] Series Editor Tamer Bas¸ar Coordinated Science Laboratory University of Illinois at Urbana-Champaign 1308 W. Main St. Urbana, IL 61801-2307 USA

ISBN: 978-0-8176-4794-0 DOI: 10.1007/978-0-8176-4795-7

e-ISBN: 978-0-8176-4795-7

Library of Congress Control Number: 2008923475 Mathematics Subject Classification: 93-02, 93-06 c 2008 Birkh¨auser Boston, a part of Springer Science+Business Media, LLC  All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Birkh¨auser Boston, c/o Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

Printed on acid-free paper. 9 8 7 6 5 4 3 2 1 www.birkhauser.com

Dedicated to our mentor and colleague Michael K. Sain on the occasion of his seventieth birthday

A group picture during the workshop, “Advances in Statistical Control, System Theory, and Engineering Education: A Workshop in Honor of Dr. Michael K. Sain,” on Saturday, 27 October 2007. Bottom row, from the left: Joe Cruz, Mary Sain, Elizabeth Sain, John Sain, Frances Sain, Mike Sain, Patrick Sain, Barbara Sain, Peter Hoppner, and Shirley Dyke. Middle row, from the left: Giuseppe Conte, Anna Maria Perdon, Jody O’Sullivan, Anthony Michel, Edward Davison, Frank Lewis, Peter Dorato, Bostwick Wyman, Khanh Pham, Cheryl Schrader, Ron Cubalchini, Erik Johnson, Matthew Zyskowski, Ying Shang, and Qingmin Liu. Top row, from the left: Ken Dudek, Panos Antsaklis, Steve Yurkovich, Mike Schafer, B. F. Spencer, Gang Jin, Eric Kuehner, Chang-Hee Won, Leo McWilliams, Stan Liberty, and Ronald Diersing.

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii List of Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xv

Part I Statistical Control Introduction and Literature Survey of Statistical Control: Going Beyond the Mean Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich . . . . . . . . . . . . . .

3

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case Luis Cosenza, Michael K. Sain, Ronald W. Diersing, and Chang-Hee Won . . . . 29 Statistical Control of Stochastic Systems Incorporating Integral Feedback: Performance Robustness Analysis Khanh D. Pham . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Multi-Cumulant and Pareto Solutions for Tactics Change Prediction and Performance Analysis in Stochastic Multi-Team Noncooperative Games Khanh D. Pham, Stanley R. Liberty, and Gang Jin . . . . . . . . . . . . . . . . . . . . . . . 65 A Multiobjective Cumulant Control Problem Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won . . . . . . . . . . . . . . . . 99

Part II Algebraic Systems Theory Systems over a Semiring: Realization and Decoupling Ying Shang and Michael K. Sain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

viii

Contents

Modules of Zeros for Linear Multivariable Systems Cheryl B. Schrader and Bostwick F. Wyman . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Zeros in Linear Time-Delay Systems Giuseppe Conte and Anna Maria Perdon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Part III Dynamic Systems Characteristics On the Status of the Stability Theory of Discontinuous Dynamical Systems Anthony N. Michel and Ling Hou . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Direct Adaptive Optimal Control: Biologically Inspired Feedback Control Draguna Vrabie and Frank Lewis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Characterization and Calculation of Approximate Decentralized Fixed Modes (ADFMs) Edward J. Davison and Amir G. Aghdam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Some New Nonlinear and Symbol Manipulation Techniques to Mitigate Adverse Effects of High PAPR in OFDM Wireless Communications Byung Moo Lee and Rui J.P. de Figueiredo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Homogeneous Domination and the Decentralized Control Problem for Nonlinear System Stabilization Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader . . . . . . . . . . . . . . . . . . 257 Volterra Control Synthesis Patrick M. Sain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Part IV Engineering Education The First Professional Degree: Master of Engineering? Peter Dorato . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Theology and Engineering: A Conversation in Two Languages Barbara K. Sain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 Publications of Michael K. Sain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

Preface

Life has many surprises. One of the best surprises is meeting a caring mentor, an encouraging collaborator, or an enthusiastic friend. This volume is a tribute to Professor Michael K. Sain, who is such a teacher, colleague, and friend. On the beautiful fall day of October 27, 2007, friends, families, colleagues, and former students gathered at a workshop held in Notre Dame, Indiana. This workshop brought together many people whose lives have been touched by Mike to celebrate his milestone 70th birthday, and to congratulate him on his contributions in the fields of systems, circuits, and control. Mike was born on March 22, 1937, in St. Louis, Missouri. After obtaining his B.S.E.E. and M.S.E.E. at St. Louis University, he went on to study at the University of Illinois at Urbana-Champaign for his doctoral degree. With his Ph.D. degree complete, he came to the University of Notre Dame in 1965 as an assistant professor. He became an associate professor in 1968, a full professor in 1972, and the Frank M. Freimann Chair in Electrical Engineering in 1982. He has remained at and loved the University of Notre Dame for over 40 years. Mike also held a number of consulting jobs throughout his career. Most notably, he consulted with the Energy Controls Division of Allied-Bendix Aerospace from 1976 to 1988 and the North American Operations branch of the Research and Development Laboratory of General Motors Corporation for a decade, 1984–1994. Mike’s research interests have been wide and varied. He worked on statistical control and game theory with a focus on the use of cumulants, system theory on semirings, generalized pole and zero techniques, nonlinear multivariable feedback control with tensors, structural control for buildings and bridges subject to high winds and earthquakes, jet engine gas turbine control, algebraic systems theory, and generalization of H∞ control. Mike is a pioneer in statistical control theory, which generalizes traditional linearquadratic-Gaussian control by optimizing with respect to any of the cost cumulants instead of just the mean. For over 30 years, Mike and his students have contributed to the development of minimal cost variance control, kth cumulant control, and statistical game theory. In statistical game theory, the statistical paradigm generalized mixed H2 /H∞ control and stochastic H∞ control concepts. Although there is more

x

Preface

work to be done in this area, Mike has pioneered a promising new stochastic optimal control method. Another major contribution of Mike’s research is in the field of algebraic systems theory, expanding the algebraic system-theoretic concepts of poles and zeros of a linear system. Mike and his collaborators also researched a module-theoretic approach to zeros of a linear system and the application of these ideas to inverse systems. Mike’s 1981 monograph Introduction to Algebraic Systems Theory bridged the gap between systems theory and algebraic theory and is considered a definitive introduction to algebraic systems theory. More recently, Mike has applied concepts from feedback control theory to model Catholic moral teachings and decision making, showing that analogous structures exist in the two fields, and that one can construct a framework to support selection of “good” outcomes and rejection of what is “not good.” Mike has also been a valuable resource to the Institute of Electrical and Electronics Engineers (IEEE). In particular, he was the founding editor-in-chief of the flagship Circuits and Systems Magazine. Mike, with the support of then IEEE Circuits and Systems Society president Rui De Figueiredo, changed the IEEE Circuits and Systems Society Newsletter into the Circuits and Systems Magazine, a highly regarded magazine within the IEEE. Mike was also the editor-in-chief of the journal of record in the field of control systems, the IEEE Transactions on Automatic Control. He also served on numerous award committees, including the IEEE Award Board, where he chaired the Baker Prize Committee which annually determines the best publication from among all those in the transactions and journals of the IEEE. During his 42 years of service, he has received numerous awards and honors including the IEEE Centennial Medal, IEEE Circuits and Systems Society Golden Jubilee Medal, IEEE Fellow, and University of Notre Dame President’s Award. Perhaps more importantly, Mike is widely recognized by his peers and students as an outstanding educator, and he has received several teaching awards for his excellent pedagogy. He has directed over 47 theses and dissertations, 19 of which are doctoral dissertations, and his students have become leaders in academic research, teaching, and administration, and in industry and government. This Festschrift volume is divided into four parts: statistical control theory, algebraic systems theory, dynamic systems characteristics, and engineering education. The statistical control theory part begins with a survey. Statistical control is a generalization of Kalman’s linear-quadratic-Gaussian regulator. Here, we view the optimal cost function as a random variable and optimize the cost cumulants. The current state of research is discussed in the first chapter. In the second chapter, Cumulant Control Systems: The Cost-Variance, Discrete-Time Case, the authors address the second cumulant optimization for a discrete-time system. In this digital world, this is an important addition to statistical control theory. The third chapter, by Pham, discusses statistical control for a system with integral feedback, and extends the statistical control idea to both regulation and tracking problems. The fourth chapter uses a statistical control paradigm for decision making, using multi-player game theory. The final chapter of Part I deals with multi-objective cumulant control. Here the cumulant idea is applied to mixed H2 /H∞ control. Instead of optimizing the mean in the H2

Preface

xi

cost function, the authors optimize the variance while constraining the system’s H∞ norm. Interestingly, this idea generalizes stochastic H∞ control. The second part of the book is dedicated to algebraic systems theory. Its first chapter describes a new system theory for linear time-invariant systems with coefficients in a semiring motivated by applications in communication networks, manufacturing systems, and queueing systems. In addition to revealing realization issues of systems over semirings, this theory connects geometric control with the frequency domain and provides methods to compute invariant sets associated with decoupling. The second chapter, by Schrader and Wyman, discusses the module-theoretic approach to zeros and poles of a linear multivariable system. By examining the intuition that the zeros of a linear system should become the poles of its inverse system, this chapter emphasizes Mike’s contributions to this body of knowledge. The main result provides a complete understanding of the connection between all poles and zeros of a transfer function matrix, including those at infinity and those resulting from singularities. The final chapter of this section, by Conte and Perdon, presents the notion of zeros for linear time-delay systems by generalizing the algebraic notion of a zero module. Additional control problems such as inversion and tracking are also addressed using this framework. The third part starts with the overview of stability results for discontinuous hybrid dynamical systems. Michel and Hou show that if the hypotheses of a classical Lyapunov stability and boundedness result are satisfied for a given Lyapunov function, then the hypotheses of the corresponding stability and boundedness result for discontinuous dynamical systems are also satisfied for the same Lyapunov function. They also show that the converse is not true in general. The second chapter solves complex systems using a neural network structure. In particular, it discusses two algorithms, based on a biologically inspired structure, in solving for an optimal state feedback controller. The third chapter tackles the characterization and calculation of approximate decentralized fixed modes. The fourth chapter is concerned with a communications system, wherein Lee and de Figueiredo discuss two approaches to mitigate adverse effects due to the high peak-to-average power ratio in orthogonal frequency division multiplexing systems. Then Polendo et al. discuss constructive techniques for stabilization of nonlinear systems with uncertainties and limited information. The final chapter of this part presents a systematic method for deriving and realizing nonlinear controllers and nonlinear closed-loop systems using Volterra control synthesis. Mike has been a lifelong mentor and teacher to many students. So, appropriately, we have chosen two important subjects in education for this volume. One important topic is the issue of the first professional degree in engineering. In this context, Dorato argues that the first professional degree in engineering should be the Master of Engineering degree rather than the bachelor’s degree. In order to maintain America’s competitiveness, advances in engineering education are prerequisite. This chapter should generate some insight into the question of what constitutes a true engineering education. A relatively new interest of Mike has been the research of the relationship between theology and engineering. In this research he has been collaborating with his daughter at St. Thomas University. Thus, it is appropriate to end this volume with a chapter about theology and engineering, authored by Barbara Sain.

xii

Preface

There she answers the question: What does the discipline of engineering have to do with the life of faith? It is interesting and insightful to see models of the will in block diagrams! Religion is an important part of Mike’s life. He is a devoted Catholic with a great love and devotion for the Virgin Mary. He attends daily Mass and has visited Medjugorje in Bosnia-Herzegovina four times. Perhaps this is why his view on life is larger than just research or teaching. We would like to end this preface with a prayer—the same prayer that begins all Mike’s classes—because this is another commencement for Mike. Our Father, Who art in heaven Hallowed be Thy Name; Thy kingdom come, Thy will be done, on earth as it is in heaven. Give us this day our daily bread, and forgive us our trespasses, as we forgive those who trespass against us; and lead us not into temptation, but deliver us from evil. Amen.

Notre Dame, IN Evansville, IN Notre Dame, IN Los Angeles, CA Boise, ID Philadelphia, PA Columbus, OH October 2007

P. Antsaklis R. Diersing E. Kuehner P. Sain C. Schrader C. Won S. Yurkovich Workshop Organizing Committee

Acknowledgments

The editors are most grateful to the Notre Dame Electrical Engineering Department and the authors of the papers for the support of the workshop and this volume, especially Dr. Thomas Fuja. We thank all the participants of the workshop. There was much enthusiastic support from the planning stage from many people, including Drs. Bostwick Wyman, Edward Davison, Frank Lewis, and Tony Michel. For the logistics support, we acknowledge the aid from Ms. Lisa Vervynckt and Ms. Fanny Wheeler of the University of Notre Dame. We also acknowledge valuable advice from our colleagues and friends, especially Drs. Derong Liu, Panos Antsaklis, Rui deFigueiredo, and Peter Dorato. The first editor would like to acknowledge the support from Dr. Saroj Biswas and Dr. Keya Sadeghipour. For the LaTeX typesetting help, we thank Jong-Ha Lee, Zexi Liu, and Bei Kang of Temple University. Jong-Ha was responsible for the wonderful workshop web site. The editors thank Mr. Tom Grasso and Ms. Regina Gorenshteyn of Birkh¨auser Boston for their professional support and advice.

List of Contributors

Amir G. Aghdam Dept. of Elect. and Comp Eng. Concordia University Montreal, Quebec H3G 1M8, Canada [email protected] Panos Antsaklis Dept. of Elect. Eng. University of Notre Dame Notre Dame, IN 46556, USA [email protected] Giuseppe Conte Dip. di Ingegneria Informatica Gestionale e dell’Automazione Universit`a Politecnica delle Marche Via Brecce Bianche 60131 Ancona - Italy [email protected] Luis Cosenza 4412 Residencial Pinares El Hatillo, Tegucigalpa Honduras luis [email protected] Edward J. Davison Dept. of Elect. and Comp. Eng. University of Toronto Toronto, Ontario M5S 1A4, Canada [email protected]

Rui J.P. de Figueiredo California Institute for Telecommunications and Information Technology University of California Irvine, CA 92697-2800, USA [email protected]

Ronald W. Diersing Department of Engineering University of Southern Indiana Evansville, IN 47712, USA [email protected]

Peter Dorato Department of Electrical and Computer Engineering MSC01 1100 University of New Mexico Albuquerque, NM 87131-0001, USA [email protected]

Ling Hou Dept. of Electrical and Computer Engineering St. Cloud State University St. Cloud, MN, USA [email protected]

xvi

List of Contributors

Gang Jin Electronics & Electrical Engineering Ford Motor Company Dearborn, MI 48124, USA [email protected] Eric Kuehner Dept. of Electrical Engineering University of Notre Dame Notre Dame, IN 46556, USA [email protected] Byung Moo Lee Infra Laboratory, KT 17 Woomyeon-dong, Seocho-gu Seoul, 137-792, South Korea [email protected] Frank Lewis Automation and Robotics Research Institute The University of Texas at Arlington 7300 Jack Newell Blvd. S. Ft. Worth, TX 76118, USA [email protected] Stanley R. Liberty Office of President Kettering University Flint, MI 48504, USA [email protected] Anthony N. Michel Dept. of Electrical Engineering University of Notre Dame Notre Dame, IN, 46556 USA [email protected] Anna Maria Perdon Dip. di Ingegneria Informatica Gestionale e dell’Automazione Universit`a Politecnica delle Marche Via Brecce Bianche 60131 Ancona - Italy [email protected]

Khanh D. Pham Space Vehicles Directorate Air Force Research Laboratory Kirtland AFB, NM 87117, USA Jason Polendo Southwest Research Institute 6220 Culebra Rd. San Antonio, TX 78229, USA [email protected] Chunjiang Qian Dept. of Electrical & Computer Engineering University of Texas at San Antonio One UTSA Circle San Antonio, Texas 78249, USA [email protected] Barbara K. Sain University of St. Thomas 2115 Summit Ave. Saint Paul, MN 55105, USA [email protected] Michael K. Sain Department of Electrical Engineering University of Notre Dame Notre Dame, IN 46556, USA [email protected] Patrick M. Sain Raytheon Company P.O. Box 902 El Segundo, CA 90245, USA [email protected] Cheryl B. Schrader College of Engineering Boise State University 1910 University Drive Boise, ID 83725-2100, USA [email protected]

List of Contributors

xvii

Ying Shang Department of Electrical and Computer Engineering Southern Illinois University Edwardsville, IL 62026, USA [email protected]

Bostwick F. Wyman Department of Mathematics The Ohio State University 231 West 18th Avenue Columbus, OH 43210, USA [email protected]

D. Vrabie Automation and Robotics Research Institute The University of Texas at Arlington 7300 Jack Newell Blvd. S Ft. Worth, TX 76118, USA [email protected]

Stephen Yurkovich Department of Electrical Engineering The Ohio State University Columbus, OH 43210, USA [email protected]

Chang-Hee Won Department of Electrical and Computer Engineering Temple University Philadelphia, PA 19122, USA [email protected]

Part I

Statistical Control

Introduction and Literature Survey of Statistical Control: Going Beyond the Mean Chang-Hee Won,1 Ronald W. Diersing,2 and Stephen Yurkovich3 1 2 3

Department of Electrical and Computer Engineering, Temple University, Philadelphia, PA 19122, USA. [email protected] Department of Engineering, University of Southern Indiana, Evansville, IN 47712, USA. [email protected] Department of Electrical Engineering, The Ohio State University, Columbus, OH 43210, USA. [email protected]

Summary. In traditional optimal control, the system is modeled as a stochastic differential equation and an optimal controller is determined such that the expected value of a cost function is minimized. An example is the well-known linear-quadratic-Gaussian problem that has been studied extensively since the 1960s. The mean or the first cumulant is a useful performance metric, however, the mean is only one of the cumulants that describe the distribution of a random variable. It is possible for the operator to optimize the whole distribution of the cost function instead of just the mean. In fact, a denumerable sum of all the cost cumulants has been optimized in risk-sensitive control. The key idea behind statistical control is to optimize other statistical quantities such as the variance, skewness, and kurtosis of the cost function. This leads to the optimal performance shaping concept. Furthermore, we use this statistical concept to generalize H∞ and multiple player game theory. In both traditional H∞ theory and game theory, the mean of the cost function was the object of optimization, and we can extend this to the optimization of any cumulants if we utilize the statistical control concept. In this chapter, we formulate and provide a literature survey of statistical control. We also review minimal cost variance (second cumulant) control, kth cost cumulant control, and multiobjective cumulant games. Furthermore, risk-sensitive control is presented as a special case of statistical control. When we view the cost function as a random variable, and optimize any of the cost cumulants, linear-quadratic-Gaussian, minimal cost variance, risk-sensitive, game theoretic, and H∞ control all fall under the umbrella of statistical control. Finally, we interpret the cost functions via utility functions.

1 A Brief Introduction to Statistical Control In statistical control, the cost function is viewed as a random variable and the performance is shaped through the cost cumulants. In other words, the density function is shaped through the mean (first cumulant), variance (second cumulant), skewness (third cumulant), kurtosis (fourth cumulant), and other higher order C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 1, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

4

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

cumulants. The density function shaping shown in the cover of this book is a cartoon representation of this density shaping concept. In this respect, statistical control generalizes classical linear-quadratic-Gaussian optimal control, where only the mean of the cost function is minimized. Moreover, when applied to stochastic game theory, the statistical control approach also generalizes H-infinity control. This chapter is a brief survey of historical and recent developments in statistical control. Mathematically, the statistical control problem is formulated as follows. Consider the Ito-sense stochastic differential equation with control, dx(t) = f (t, x, k)dt + E(t)dw(t), y(t)dt = g(t, x, k)dt + dv(t), and the cost function,

 tF J(t, x(t), k) =

L(t, x, k)dt. t

Here, x(t) is a state vector, k(t, x) is an input vector, w(t) is a disturbance vector of Brownian motion, y(t) is an output vector, and v(t) is an output noise vector of Brownian motions. To define the cumulants, we need the following first and second characteristic equations: φ (t) = E{exp(−sJ)} and

∞

(−1)i i βi s , i i=1

ψ (s) = log φ (s) ∑

where {βi } are known as the cumulants of the cost function, J. Functions of these cumulants are maximized or minimized to shape the cost distribution. The statistical control problem finds the optimal controller that optimizes the distribution of the cost function through cost cumulants. In the linear-quadratic-Gaussian (LQG) optimal control problem, the system is linear, f (t, x, k) = A(t)x(t)+B(t)k(t, x), and the cost function is quadratic, L(t, x, k) = xT Qx + kT Rk, where the superscript T denotes the transpose. Then one finds the contollers, k, such that the expected value of the quadratic cost is minimum. This problem was popularized by R. E. Kalman [Kal60]. This is the first cumulant (mean) problem and it is a special case of statistical control. This is extensively studied in the literature with a number of textbooks about the subject, for example see [Ast70, KS72]. Minimal cost variance (MCV) control, where the idea is to minimize the variance (i.e., the second cumulant) of the cost function instead of the mean, has been pioneered by Sain [Sai65, Sai66]. Sain and Souza examined the MCV concept for problems of estimation in [SS68]. In 1971, an open loop result on the MCV optimal control problem was solved in [SL71]. They observed that all cost cumulants of any finite-horizon integral quadratic form in the system state were quadratic affine

Introduction and Literature Survey of Statistical Control

5

in the inital state mean for a linear system with Gaussian input noise. Some years later, Liberty and Hartwig published the results of generating cost cumulants in the time domain [LH76]. Recently, Sain, Won, and Spencer showed the relationship between risk-sensitive control and MCV control in [SWS92, Won95]. The full-statefeedback MCV control results are given in [Won95]. There, the problem formulation is for a nonlinear system with a nonquadratic cost function. Further development was made by Pham et al. for a linear system and quadratic cost function [Pha04]. This is described in Section 3. Won developed the necessary condition of MCV control for a nonlinear system and a non quadratic cost function in [Won05]. Finally, the statistical control idea was generalized to game theory by Diersing [Die06], see Section 4. Surprisingly, this concept leads to the generalization of the H∞ control theory. The development of statistical control is summarized in the Table 1. Here, we note that minimal variance control and cost moment control are covered in the literature, but not statistical (cost cumulant) control. Minimal variance control, which minimizes the variance of the output, has not been very successful. ˚ This phrase, minimal variance control, was coined by Karl Astrom and his students. It is not cost variance, but rather output variance. From an engineering point of ˚ view, Astrom’s variance control is very similar to LQG, E{xT x} = traceE{xxT }. As such, we may lump it together with the work of Kalman. Therefore, it is in essence the same idea as cost average control. Minimal cost moment control also exists, however, it is different from cost cumulant control, and in fact Sain was one of the first researchers to study cost moment control in the 1960s [Sai67]. However, this research was not very successful. Even though from a mathematician’s point of view, cost moment may be similar to cost cumulant, in control engineering applications a cumulant gives very different results from a moment. For example, in cost moment minimization, a nonlinear controller may result from a linear system, quadratic cost case. However cost cumulant control gives a linear controller. Cost moment controllers are much too complicated, giving nonlinear controllers for the LQ case. Moreover, the control of higher moments is problematic. If we control just the first few of them, the neglected higher moments may have more effect than the ones that we have chosen to control. This makes moment control very sensitive to model errors. This is not desirable in theory, in computation, in approximation, or in application. With cumulants, controlling the first few is an excellent approximation to controlling them all, as the neglected ones tend to produce increasingly smaller effects. This is true at least in the multiple large applications that we have studied. Bas¸ar and Bernhard noted the relationship between deterministic dynamic games and H∞ optimal control in their book [BB91]. Various researchers have pointed out that the time domain characterization of H∞ controllers contains a “generalized” Riccati-type equation that is also found in linear-quadratic zero-sum differential games or risk-sensitive linear-exponential-quadratic stochastic control [GD88], suggesting a possible relation between different approaches to robust control. Recently, in the area of robust control, we are noticing just such a synthesis of various different areas such as H∞ , deterministic differential game theory, and risk-sensitive control.

6

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich Table 1. The development of statistical control.

Year 1965

Authors Sain

1966

Sain

1968

Sain and Souza

1969

Cosenza

1971

Sain and Liberty

1971

Liberty

1976

Liberty Hartwig

1978

Liberty Hartwig

1992

Sain, Won Spencer

1995

Won

2000

Sain, Won, Spencer, Liberty

2004

Pham

2006

Diersing

Title On Minimal-Variance Control of Linear Systems with Quadratic Loss Control of Linear Systems According to the Minimal Variance Criterion A Theory for Linear Estimators Minimizing the Variance of the Error Squared On the Minimum Variance Control of Discrete-Time Systems Performance Measure Densities for a Class of LQG Control Systems Characteristic Functions of LQG Control

Remarks Started minimal cost variance control research

On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants Design-PerformanceMeasure Statistics for Stochastic Linear Control Systems Cumulant Minimization and Robust Control

Published in Information and Control journal

Cost Cumulants in Risk-Sensitive and Minimal Cost Variance Control Cumulants and Risk-Sensitive Control: A Cost Mean and Variance Theory with Application to Seismic Protection of Structures Statistical Control Paradigm for Structural Vibration Suppression H∞ , Cumulants and Games

Open loop minimal cost variance control Cost variance estimators

Cost variance control in discrete time Performance measure densities Characteristic function of the quadratic cost

Performance measure densities for nth cumulant case, linear case Relations between cost variance control and risk-sensitive control Full state feedback cost variance control Published in Annals of the International Society of Dynamic Games

kth cumulant control, linear system kth cumulant, nonlinear problem formulation, game theory

Introduction and Literature Survey of Statistical Control

7

Fig. 1. Relations between various robust controls.

See Figure 1 for an overview of the connection between various different areas of robust control. For more detailed descriptions of these relationships, see [Won04]. In Section 2 we review the MCV control, where we minimize the variance of the cost function while keeping the mean at a prespecified level. In Section 3, we review kth cumulant control results for a linear system and quadratic cost function. In Section 4, we survey the cumulant game results and show that this statistical control concept generalizes mixed H2 /H∞ as well as H∞ control. In Section 5, we survey risk-sensitive control and relate to the statistical control by showing that the exponential cost function is a denumerable sum of all the cumulants. Then we provide some interpretation of the cost functions through the utility function concept in Section 6. Our conclusions and future work are provided Section 7.

2 Minimal Cost Variance Control: Second Cumulant Case In statistical control, certain linear combinations of the cost cumulants are constrained or minimized. Thus, the classical minimal mean cost problem can be seen as a special case of statistical control, in which the first cumulant is minimized. This idea of minimizing the expected value of the cost function was developed in the 1960s, and it is well known in the literature. For example, see [Ath71,Dav77,Sag68]. The fundamental idea behind minimal cost variance (MCV) control is to minimize the variance of the cost function while keeping the mean at a prespecified level. MCV

8

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

control is a special case of statistical control where the second cumulant (variance) is optimized. MCV control was first developed in a dissertation in 1965 [Sai65], and appeared in a journal in 1966 [Sai66]. In [SS68], Sain and Souza examined the MCV concept for problems of estimation. The discrete time version of MCV control was investigated by Cosenza [Cos69]. In 1971, Sain and Liberty published an open loop result on minimizing the performance variance while keeping the performance mean at or below a prespecified value [SL71]. Liberty continued to study characteristic functions of integral quadratic forms, further developing the open loop MCV control idea in a Hilbert space setting [Lib71]. Some years later, Liberty and Hartwig published the results of generating cost cumulants in the time domain [LH76]. Sain, Won, and Spencer showed that MCV control is related to risk-sensitive control under some appropriate assumptions [SWS92]. The MCV formulation is for a nonlinear system and a non-quadratic cost framework, however, the controller is solved for a linear system and quadratic cost function [Won95, SWS95, SWS00]. This result is summarized in this section. Both risk-sensitive and MCV control are a special case of statistical control. In [SWS00], we also show that MCV control is an approximation of risk-sensitive control, where first two cumulants are optimized. A time-line comparison of MCV control and risk-sensitive control is shown in Figure 2. Minimal Cost Variance Control [Sain] 1965 [Sain] 1966 [Sain and Souza] 1968 [Liberty], [Sain and Liberty] 1971

RS Control

1973 [Jacobson] 1974 [Speyer, Deyst, Jacobson] [Liberty and Hartwig] 1976 1976 [Speyer] [Liberty and Hartwig] 1978 1981 [Kumar and van Schuppen], [Whittle] 1985 [Bensoussan, van Schuppen] 1990 [Whittle] 1991 [Whittle] [Sain, Won and Spencer] 1992 1994 [Won, Sain, and Spencer] [James, Baras, Elliott], [Runolfsson] [Won] 1995 Fig. 2. A time-line comparison of MCV and RS control.

Introduction and Literature Survey of Statistical Control

9

2.1 Open Loop Minimal Cost Variance Control In this section, we present the solution of the open loop MCV control problem. This is a summary of the results presented in [Sai66]. Consider a linear system x(t) ˙ = A(t)x(t) + B(t)u(t) + E(t)w(t) and the performance measure  tF J = [xT (t)Qx(t) + uT (t)Ru(t)] dt + xT (tF )Px(tF ),

(1)

(2)

0

where w(t) is zero mean with white characteristics relative to the system, tF is the fixed final time, x(t) ∈ Rn is the state of the system, and u(t) ∈ Rm is the control action. The weighting matrices P and Q are symmetric and positive semidefinite, and R is a symmetric and positive definite matrix. Note that E{w(t)wT (σ )} = Sδ (t − σ ),

(3)

where δ denotes the Dirac delta function and the superscript T denotes the transposition. The fundamental idea behind minimal cost variance control [Sai65, Sai66] is to minimize the variance of the cost function J JMV = VARk {J}

(4)

Ek {J} = M,

(5)

while satisfying a constraint where J is the cost function and the subscript k on E denotes the expectation based upon a control law k generating the control action u(t) from the state x(t) or from a measurement history arising from that state. By means of a Lagrange multiplier μ , corresponding to the constraint (5), one can form the function JMV = μ (Ek {J} − M) + VARkJ,

(6)

which is equivalent to minimizing J˜MV = μ Ek {J} + VARk{J}.

(7)

In [SL71], a Riccati solution to J˜MV minimization is developed for the open loop case u(t) = k(t, x(0)). (8) The solution is based upon the differential equations 1 z˙(t) = A(t)z(t) − B(t)R−1 BT (t)ρˆ (t) 2

(9)

10

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

ρ˙ˆ (t) = −AT (t)ρˆ (t) − 2Qz(t) − 8 μ Qv(t) T

(10)

v(t) ˙ = A(t)v(t) + E(t)SE (t)y(t)

(11)

y(t) ˙ = −A (t)y(t) − Qz(t)

(12)

z(0) = x(0) ρˆ (tF ) = 2Pz(tF ) + 8μ Pv(tF )

(13) (14)

v(0) = 0 y(tF ) = Pz(tF )

(15) (16)

1 u(t) = − R−1 BT (t)ρˆ (t). 2

(17)

T

with boundary conditions

and the control action relationship

2.2 Full-State Feedback Minimal Cost Variance Control This section deals with feedback MCV control in the completely observed case as a special case of statistical control. The admissible controller is defined, then the cost variance is minimized within that admissible controller. If T = {t : t0 ≤ t ≤ tF } is a set of real time instants, then the system which is to be controlled has the stochastic differential equation dx(t) = f (t, x(t), u(t))dt + E(t)dw(t), t ∈ T,

(18)

where x(t) ∈ Rn is an n-tuple state at time t, u(t) ∈ Rm is an m-tuple control action at time t, and f is a continuous mapping from T × Rn × Rm to Rn . We assume that f and the gradient of f with respect to x are bounded. The stochastic nature of the problem arises from the Wiener process (equivalently Brownian motion) w(t). Equation (18) must be viewed as a formal representation of the way in which many stochastic models arise: by means of a classical derivation of a differential equation and the addition of a noise process to describe, and to compensate for, uncertainties and approximations. These uncertainties may include assumptions in obtaining f (including neglected higher order terms) and physical disturbances such as thermal noise or interference. The interpretation of (18) is given in terms of Ito’s integral equation t t x(t) − x(t0 ) = f (s, x(s), u(s))ds + E(s) dw(s), (19) t0

t0

in which the second integral is a Wiener integral. In order to assess the performance of (18), it is convenient to associate with each realization of w(t) a penalty

Introduction and Literature Survey of Statistical Control

 tF J(t0 , x0 ; u) =

L(s, x(s), u(s))ds + ψ (x(tF )),

11

(20)

t0

where L is a continuous mapping from T × Rn × Rm to the nonnegative real line R+ , u = {u(s) : s ∈ T }

(21)

is a control action segment, and x0 = x(t0 ). We assume that L and the gradient of L with respect to x are bounded. With regard to observing (18), it is assumed that a noise-free measurement of x(t) can be made over T . The partially observed case will be treated in the next section. In order to control the performance of (18), a memoryless feedback control law is introduced in the manner u(t) = k(t, x(t)), t ∈ T,

(22)

where k is a nonrandom function with random arguments. The Markovian nature of the problem suggests that it is sufficient to consider the process of equation (22) [FS92, p. 136]. Accordingly, the notation of (20) is replaced by J(t0 , x0 ; k). The class of admissible control laws, and comparison of control laws within the class, is defined in terms of the first and second moments of (20). Let E{·} denote the mathematical expectation. Define V1 (t0 , x0 ; k) = E{J(t0 , x0 ; k)|x(t0 ) = x0 ; k} 2

V2 (t0 , x0 ; k) = E{J (t0 , x0 ; k)|x(t0 ) = x0 ; k}.

(23) (24)

Definition 1. A function M(t, x), from T × Rn to R+ , is an admissible mean cost function if it has continuous second partial derivatives with response to x and a continuous partial derivative with respect to t, and if there exists a continuous control law k such that V1 (t, x; k) = M(t, x) (25) for t ∈ T and x ∈ Rn . ∗ satisfies A minimal mean cost (MMC) control law kM ∗ V1 (t, x; kM ) = V1∗ (t, x) ≤ V1 (t, x; k),

(26)

∗ . Clearly, M(t, x) ≥ V ∗ (t, x). for t ∈ T , x ∈ Rn , whenever k = kM 1

Definition 2. Every admissible M(t, x) defines a class KM of admissible control laws k corresponding to M in the manner that k ∈ KM if and only if k satisfies (25). It is now possible to define an MCV control law kV∗ . Definition 3. Let M(t, x) be an admissible mean cost function, and let KM be its induced class of admissible control laws. An MCV control law kV∗ then satisfies V2 (t, x; kV∗ ) = V2∗ (t, x) ≤ V2 (t, x; k),

(27)

12

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

for t ∈ T , x ∈ Rn , whenever k ∈ KM is such that k = kV∗ . The corresponding cost variance is given by V ∗ (t, x) = V2∗ (t, x) − M 2 (t, x) (28) for t ∈ T , x ∈ Rn . An MCV control problem, therefore, is quite general in its scope. It presupposes that an average cost M(t, x) has been specified (arbitrarily, within the bounds set by Definition 1), and it seeks the control law which minimizes the variance of (20) about the average (25). The main result of MCV optimal control is summarized in the following theorem. This is a necessary condition for optimality. Theorem 1. Let M(t, x), t ∈ T , x ∈ Rn , be an admissible mean cost function, and let M induce a nonempty class KM of admissible control laws. Then the MCV function V ∗ (t, x) satisfies a Hamilton–Jacobi–Bellman equation    ∂ M(t, x) 2  min O(k)[V ∗ (t, x)] +  = 0,  ∂x  k∈KM E(t)W (t)E T (t)

(29)

for t ∈ T , x ∈ Rn , together with the terminal condition V ∗ (tF , x) = 0.

(30)

In equation (29), a2A = a, Aa . The proof of Theorem 1 is given in [SWS00]. The sufficient condition—verification theorem—is also given in [SWS00]. The solution of the general nonlinear system, nonquadratic cost function MCV problem is still an ongoing research area, however, for a linear system, f (t, x, k) = A(t)x(t) + B(t)k(t, x), and the quadratic cost function, L(t, x, k) = xT Qx + kT Rk, the solution is as follows [SWS00, Won95]. Theorem 2. Out of all the admissible controllers that satisfy (25), the optimal linear controller that minimizes the following:   V ∗ (t, x) = min E{J 2 (t, x, k)} − M 2 (t, x) k∈KM

is given by

kV∗ (t) = −R−1 BT [M + γ V ]x(t),

where M and V are the solutions of the coupled Riccati equations M˙ + AT M + M A + Q − M BR−1BT M + γ 2 V BR−1 BT V = 0

(31)

and V˙ + 4M EWE T M + AT V + V A − M BR−1BT V −V BR−1 BT M − 2γ V BR−1 BT V = 0 with boundary conditions M (tF ) = QF and V (tF ) = 0.

(32)

Introduction and Literature Survey of Statistical Control

13

3 kth Cost Cumulant Control: kth Cumulant Case Pham developed finite horizon kth cumulant state feedback optimal control [PSL02]. Won and Pham also investigated the infinite horizon version of the MCV and kth cumulant cases, respectively [WSL03, PSL04]. The output feedback statistical control for the kth cumulant case is considered in [PSL02]. Here we will introduce the kth cost cumulant kcc control problem for a linear system with a quadratic cost function. The cost function is a linear combination of the first kth cumulants of a finite horizon integral quadratic form cost. In kCC control, we minimize this cost function. The following results were given by Pham in [PSL04]. Theorem 3. Consider the stochastic linear-quadratic control problem defined on [t0 ,tF ] dx(t) = (A(t)x(t) + B(t)u(t))dt + E(t)dw(t), x(t0 ) = x0 , (33) and the performance measure  tF J(t0 , x0 ; u) = [xT (τ )Qx(τ ) + uT (τ )Ru(τ )] d τ + xT (tF )Q f x(tF ),

(34)

0

where coefficients A ∈ C ([t0 ,tF ]; Rn×n ); B ∈ C ([t0 ,tF ]; Rn×m ); E ∈ C ([t0 ,tF ]; Rn×p ); Q ∈ C ([t0 ,tF ]; Sn ) positive semidefinite; R ∈ C ([t0 ,tF ]; Sm ) positive definite; and W ∈ S p ). Suppose further that both k ∈ Z+ and the sequence μ = {μi ≥ 0}ki=1 with μ1 > 0 are fixed. Then the optimal state-feedback kCC control is achieved by the gain k

K ∗ (α ) = −R−1 (α )BT (α ) ∑ μˆ r H ∗ (α , r), α ∈ [t0 ,tF ],

(35)

r=1

where the real constants μˆ r = μi /μ1 represent parametric control design freedom and {H ∗ (α , r) ≥ 0}kr=1 are symmetric solutions of the backward differential equations d ∗ H (α , 1) = −[A(α ) + B(α )K ∗(α )]T H ∗ (α , 1) − H ∗(α , 1)[A(α ) + B(α )K ∗ (α )] dα −K ∗T (α )R(α )K ∗ (α ) − Q(α ), d ∗ H (α , r) = −[A(α ) + B(α )K ∗(α )]T H ∗ (α , r) − H ∗ (α , r)[A(α ) + B(α )K ∗ (α )] dα r−1 2r! H ∗ (α , s)E(α )W E T (α )H ∗ (α , r − s), −∑ s!(r − s)! s=1 with the terminal conditions H ∗ (tF , 1) = Q f , and H ∗ (tF , r) = 0 when 2 ≤ r ≤ k, whenever these solutions exist. The solution is obtained using modified dynamic programming, which is an iterative algorithm to find approximate solutions. See [PSL04] for the complete description of the solution procedure.

14

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

4 Cumulant Games: Multiobjective Case A natural extension from cumulant control is into game theory. Stochastic differential game theory has a great deal of work using the mean of cost functions as the performance indices for the players. It makes sense therefore to use further cumulants as performance indices. Furthermore, application of game theory is in the area of H2 /H∞ and H∞ control. With the use of cumulants, we can find a cumulant generalization of these two well-known control techniques. The game is given through a stochastic differential equation dx(t) = f (t, x(t), u(t), w(t))dt + σ (t, x(t))d ξ (t),

(36)

where u is the control, w is the second player (the disturbance), and ξ is Brownian motion. The costs for the players are given as  tF J1 (t, x, u, w) = L1 (t, x(t), u(t), w(t))dt + ψ1 (tF , x(tF )) t0

 tF J2 (t, x, u, w) =

(37) L2 (t, x(t), u(t), w(t))dt + ψ2 (tF , x(tF ))

t0

with J1 being the control’s cost and J2 , the disturbance’s cost. This result is found in [Die06]. ¯ with Theorem 4. Let M be an admissible mean cost function, M ∈ C1,2 p (Q) ∩C(Q), 1,2 ¯ an associated UM . Also consider the function V ∈ C p (Q)∩C(Q) that is a solution to  ∂V ∂V (t, x) + f T (t, x, μ , ν ∗ ) (t, x) min μ ∈UM ∂t ∂x   ∂ 2V 1 T (38) + tr σ (t, x)W (t)σ (t, x) 2 (t, x) 2 ∂x 2   ∂M (t, x) =0 +  ∂x σ (t,x)W (t)σ T (t,x) ¯ that satisfies with V (tF , x f ) = 0 and the function P ∈ C1,2 p (Q) ∩C(Q)

∂P ∂P (t, x) + f T (t, x, μ ∗ , ν ) (t, x) min ν ∈WF ∂t ∂x   1 ∂ 2P T + tr σ (t, x)W (t)σ (t, x) 2 (t, x) 2 ∂x  + L2 (t, x, μ ∗ , ν ) = 0

(39)

with P(tF , x f ) = ψ2 (x f ). If the strategies μ ∗ and ν ∗ are the minimizing arguments of (38) and (39), then the pair (μ ∗ , ν ∗ ) constitutes a Nash equilibrium solution.

Introduction and Literature Survey of Statistical Control

15

Now consider a linear system, f (t, x(t), u(t), w(t)) = dx(t) = (Ax(t) + Bu(t) + Dw(t))dt + Ed η (t), and the quadratic costs L1 (t, x(t), u(t), w(t)) = zT1 (t)z1 (t) L2 (t, x(t), u(t), w(t)) = δ 2 wT (t)w(t) − zT2 (t)z2 (t), where z1 (t) = G1 (t)x(t) + H1 (t)u(t) z2 (t) = G2 (t)x(t) + H2 (t)u(t), where H1T H1 = R1 > 0, H2T H2 = R2 > 0, GT1 G1 = Q1 ≥ 0, and GT2 G2 = Q2 ≥ 0. In traditional mixed H2 /H∞ control such as the one in [LAH94], one finds a controller, u, such that the mean of the first cost function, J1 , is minimized while the second player, w, maximizes the mean of the second cost function, J2 . Maximizing the mean of J2 is equivalent to z2 2,[t0 ,tF ] ≤ δ, sup w w2,[t0 ,tF ] which implies that δ is a constraint on the H∞ norm of the system. The solution to this mixed H2 /H∞ control problem is given in [LAH94]. One can generalize this problem to determine a controller, u, such that the variance of the first cost function, J1 , is minimized while the mean is kept at a prespecified level. This can be called mixed MCV/H∞ control. The equilibrium solution of mixed MCV/H∞ control is determined as T u∗ (t) = μ ∗ (t, x(t)) = −R−1 1 B (t)(M (t) + γ V (t))x(t) 1 w∗ (t) = ν ∗ (t, x(t)) = − 2 DT (t)P(t)x(t) δ

(40)

with Riccati equations T M˙ + AT M + M A + Q1 − M BR−1 1 B M 1 1 − 2 PDDT M − 2 M DDT P δ δ T + γ 2 V BR−1 1 B V = 0,

(41)

T −1 T V˙ + AT V + V A − γ M BR−1 1 B V − γ V BR1 B M 1 1 T − 2 PDDT V − 2 V DDT P − 2γ V BR−1 1 B V δ δ + 4M EWE T M = 0,

(42)

where M (tF ) = Q1f ,

16

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

with V (tF ) = 0, and T P˙ + AT P + PA − (M + γ V )BR−1 1 B P 1 T T − PBR−1 1 B (M + γ V ) − 2 PDD P δ −1 T − Q2 − M BR−1 1 R2 R1 B M

(43)

−1 T −1 −1 T − γ M BR−1 1 R2 R1 B V − γ V BR1 R2 R1 B M −1 T − γ 2 V BR−1 1 R2 R1 B V = 0,

where P(tF ) = Q2f . One can notice that as γ goes to zero, the equilibrium solution becomes the solution to the H2 /H∞ control problem. Furthermore, utilizing the statistical control concept, one may generalize the optimization criteria to higher order cumulants. For example, the mixed MCV/MCV is discussed in [Die07]. Similar to this mixed H2 /H∞ generalization, the minimax H∞ game problem can be generalized to include higher order cumulants instead of just the mean. See [Die06, Die07].

5 Risk-Sensitive Control: Sum of all the Cumulants Risk-sensitive (RS) control started with Jacobson in 1973. Jacobson extended LQG results by replacing the quadratic criterion with an exponential of a quadratic criterion, and related linear-exponential-quadratic-Gaussian (LEQG) control to differential games [Jac73]. Many years later, Whittle noted Jacobson’s results in an RS control [Whi91]. Speyer et al. [SDJ74] extended Jacobson’s results to the noisy linear measurements case in discrete time. In [SDJ74], optimal control becomes a linear function of the smoothed history of the state, and the solutions are acquired by defining an enlarged state space composed of the entire state history. This enlarged state vector grows at every new stage but retains the feature of being a discrete linear system with additive white Gaussian noise. The continuous time terminal LEQG problem is also briefly discussed in [SDJ74], and the solutions are achieved by taking a formal limit of the discrete case solutions. In 1976, Speyer considered the noisy measurement case again in continuous time, but with zero state weighting in the cost function [Spe76]. Unlike the method of the previous work [SDJ74], the Hamilton–Jacobi–Bellman equation was used to produce the solutions in [Spe76]. Kumar and van Schuppen derived the general solution of the partially observed exponential-of-integral (EOI) problem in continuous time with zero plant noise in 1981 [KV81]. Whittle then published his results for the general solution of the partially observed logarithm-exponential-of-integral (LEOI) problem in discrete time [Whi81]. Four years later, Bensoussan and van Schuppen reported the solution to the general case of a continuous time partially observed stochastic EOI problem using a different method [BV85]. Unexpectedly in 1988, Glover and Doyle related H∞ and minimum entropy criteria to the infinite horizon version of

Introduction and Literature Survey of Statistical Control

17

LEOI theory in discrete time, thus establishing a relationship between Whittle’s RS control and H∞ optimal control [GD88]. This result was extended to continuous time by Glover [Glo89]. In 1990, Whittle published the RS maximum principle in book form [Whi90], and published a journal article about the RS maximum principle for the case of partially observed states using large deviation ideas [Whi91]. A good reference for large deviation theory is [FW84]. A couple of years after Whittle’s paper [Whi90], Bensoussan published a book with all solutions (including the partially observed case) of the EOI problem [Ben92]. In 1992, James states that the RS optimal control problem with full-state-feedback information is equivalent to a stochastic differential game problem [Jam92]. Fleming and McEneaney pointed out independent, but similar, results in [FM92]. In 1994, Won, Sain, and Spencer used Runolfsson’s infinite horizon LEOI control results in a structural control application [WSS94]. Hopkins presented discounted EOI solutions in [Hop94]. In 1994, James et al. published RS control and dynamic games solutions for partially observed discrete time nonlinear systems [Jam92, Jam94]. In 1994, Runolfsson presented the relationship between Whittle’s RS control and stochastic differential games in the infinite horizon case using large deviation ideas [Run94]. The following is a summary of the development of RS control theory. Note that Whittle’s RS control cost function is just a logarithmic transformation of the EOI control cost criterion. • Jacobson, 1973 [Jac73] – linear-exponential-quadratic-Gaussian (LEQG) problem

 1 J = E μ exp( μψ ) 2  tF ψ = (xT Qx + uT Ru) dt + xT (tF )QF x(tF )

(44)

(45)

t0

– μ : + or − – completely observed case – continuous and discrete time cases – linear combination of all the moments of ψ • Speyer, Deyst, and Jacobson, 1974 [SDJ74] – LEQG problem

 1 J = E μ exp( μψ ) 2

(46)

N

ψ = ∑ (xTi Qi xi + ui Ri ui ) + xTN+1 QN+1 xN+1 i=1

– μ : nonzero constant – partially observed case · discrete time case – terminal cost (Qi = 0) problem · discrete and continuous time cases

(47)

18

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

– general problem (discrete time), but controller dependent upon an infinite dimensional state space – no process noise problem (discrete time) – no certainty equivalence principle – linear combination of all the moments of ψ • Speyer, 1976 [Spe76] – LEQG problem

 1 J = E μ exp( μψ ) (48) 2  tF ψ = (uT Ru) dt + xT (tF )QF x(tF ). (49) t0

– μ : nonzero constant – only terminal state penalized, zero state weighting – partially observed case – continuous time case – no certainty equivalence principle • Whittle, 1981 [Whi81] – risk-sensitive (RS) problem   θ 2 γ (θ ) = − log E{exp(− ψ )} θ 2

ψ=

(50)

N−1

∑ (xTi Qxi + uTi Rui) + xTN ΠN xN

(51)

i=1

– (θ = 0) risk-neutral case, (θ > 0) risk-seeking case, and (θ < 0) riskaversive case – completely and partially observed cases – discrete time – new (relative to LQG) certainty equivalence principle – linear combination of all the cumulants of ψ • Kumar and van Schuppen, 1981 [KV81] – exponential-of-integral (EOI) problem

  μ T J = E μ exp( c(x, u) dt) (52) 2 0 – partially observed case – continuous time – zero plant noise dx = Fx + Gu – linear combination of all the moments – [SDJ74] candidate for optimality wrong • Bensoussan and van Schuppen, 1985 [BV85] – EOI

  1 J = E μ exp μψ 2

(53)

Introduction and Literature Survey of Statistical Control

ψ=

 tF

(xT Qx + uT Nu) dt

19

(54)

0

– μ : real constant – partially observed case – continuous time case – method different than Whittle’s method [Whi81] • Whittle, 1991 [Whi91] – RS control J = −(kθ )−1 log E exp(−kθ φ )  tF φ= c(x, u)dt + φtF (x(tF ))

(55) (56)

0

– θ : real constant – logarithm of exponential-of-integral (LEOI) control – RS maximum principle • Bensoussan, 1992 [Ben92] – EOI control J = E θ exp(θ φ )  tF φ= c(x, u)dt + φtF (x(tF ))

(57) (58)

0

– θ : real constant – full and partial observation cases – continuous time case – nonlinear filtering theory – stochastic maximum principle • Runolfsson, 1994 [Run94] – LEOI control (infinite horizon version of Whittle’s RS cost function) T 1 J = lim log E{exp(μ 2 c(x, u) dt)} (59) T →∞ T 0 – μ : nonzero constant – solution for infinite horizon, completely observed, risk-aversive case – continuous time case – relates infinite horizon RS problem to stochastic differential games – no solution for infinite horizon partially observed case • James, 1994 [Jam94] – EOI cost function given by 

μ M−1 ε ε J = E exp ∑ c(x j , u j ) + Φ (xM ) ε j=0 – finite horizon, partially observed, RS, discrete time, nonlinear case – relates RS control to the dynamic game problem

(60)

20

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

6 Relations between RS, MCV, and Statistical Control Through Utility Functions We relate RS control with other statistical control using the concept of utility function. Before introducing the notion of risk in terms of utility functions, a brief review of utility theory is in order. The concept of utility is introduced in order to establish a uniform scale for measuring the overall value of a choice. We can define utility as a true measure of value to the decision maker. Utility theory provides a method to compare and measure values consistently with respect to a decision maker. Therefore, the choice with the highest utility value will be preferred. A utility function, denoted U (·), quantifies the order of preferences, such that U : degree of preference → R. Different decision makers faced with the same decision problem may have different utility functions. Thus utility functions can be viewed as a characterization of a decision maker in some sense. The problem of selecting an optimal decision is reduced to the problem of maximizing the expected value of the utility function, U . The utility function can be determined experimentally, or we can use one of the known utility functions. Most utility functions are convex, which implies risk-aversive behavior of a decision maker. In classical LQG optimal control we can view the cost function as having a quadratic utility function. Now, the notion of risk can be defined precisely using a utility or a disutility function (a disutility function being the negative of a utility function) as Bertsekas [Ber76] or Whittle [Whi90] did in their books. For a decision maker with a disutility function, D, defined over an interval X of real numbers and a cost function, J, we can define a risk-aversive or pessimistic attitude on the part of the decision maker by Jensen’s inequality, E{D(J)} ≥ D(E{J}), (61) for all probability distributions P on X for which the expectations above are finite. Furthermore, convexity of the disutility function can be shown to be equivalent to a risk-aversive case [Ber76, Whi90]. On the other hand, if the disutility function is concave, then the inequality is reversed in equation (61) and this becomes a riskseeking or an optimistic case. Lastly, a linear disutility function implies a risk-neutral case. An RS problem can be characterized by a sensitivity parameter θ . Depending on the sign of θ , we obtain risk-seeking, risk-aversive, or risk-neutral cases [SWS92], [Whi90]. On the other hand, if we define risk-aversive, risk-seeking, and risk-neutral cases using the disutility function, D, then the LEQG problem with an exponential disutility function is an RS problem as Whittle points out in his book [Whi90]. Note that Whittle defines the problem with an exponential disutility function as an RS problem; we will define problems with any disutility function as a statistical control problem, thus the classical LQG problem is a special case of the statistical control problem. Moreover, we can also view an MCV problem as a statistical control

Introduction and Literature Survey of Statistical Control

21

problem with a different disutility function. Thus, in this sense, the LEQG problem and the MCV problem are special cases of the statistical control problem with different disutility functions. In fact, the MCV problem uses just the first two cumulants, while the LEQG problem uses all the moments of the cost function. Whittle discusses the RS case with the disutility function, DLEQG (J) = exp(

−θ J ), 2

(62)

where θ is a real number. We can easily show that the LEQG problem is an RS problem with the disutility function (62) as Whittle presents in the form of an exercise in his book [Whi90]. Here we show that the MCV problem is a type of statistical control problem with a different disutility function. Consider a disutility function of the form DMCV (J) = μ (J − M)2 + λ (J − M),

(63)

where μ and λ are Lagrange multipliers and E{J} = M. Proposition 1. The disutility function, DMCV (J), is a convex function when μ > 0. Proof. First we say that a disutility function, DMCV (J), is convex when DMCV (α J1 + (1 − α )J2) ≤ α DMCV (J1 ) + (1 − α )DMCV (J2 )

(64)

for all α ∈ [0, 1]. By direct substitution, DMCV (α J1 + (1 − α )J2) = λ [α J1 + (1 − α )J2 − α M1 − (1 − α )M2] + μα 2 (J1 − M1 )2 + μ (1 − α )2(J2 − M2 )2 + 2α (1 − α )μ (J1 − M1 )(J2 − M2 )

(65)

and DMCV (J1 ) + (1 − α )DMCV (J2 ) = α μ (J1 − M1 )2 + λ (J1 − M1 )α + (1 − α )μ (J2 − M2 )2 + λ (J2 − M2 )(1 − α ).

(66)

Now let X = (J1 − M1 ) and Y = (J2 − M2 ). We need to show that

μα 2 X 2 + μ (1 − α )2Y 2 + 2α (1 − α )μ XY ≤ α μ X 2 + μ (1 − α )Y 2 .

(67)

Since μ > 0 for the risk-aversive case, the above equation reduces to showing whether α (α − 1)(X − Y )2 ≤ 0. (68) Because α ≥ 0, (α − 1) ≤ 0, and (X − Y )2 ≥ 0, inequality (68) is satisfied rather obviously. This also satisfies inequality (64). Therefore, DMCV is a convex function. 2

22

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

Proposition 2. For μ > 0, the disutility function, DMCV (J), describes a risk-aversive problem. Proof. This follows directly from Proposition 1 and the definition of the risk-aversive problem. 2 Proposition 3. For μ < 0, the disutility function, DMCV (J), describes a risk-seeking problem. Proof. A similar line of argument holds as for Proposition 1, except that μ < 0 and the inequality in equation (67) would be reversed. 2 Proposition 4. For μ = 0, the disutility function, DMCV (J), describes a risk-neutral problem. Proof. The disutility function becomes DMCV = λ J − λ M, which is a linear function. Thus by definition, this is a risk-neutral case. 2 Note that we may also use the first three cumulants and define the disutility function as D3CC (J) = μ1 (J − M) + μ2(J − M)2 + μ3 (J − M)3 . (69) In fact, we could define any linear combination of cumulants, but then we will probably have to place more stringent conditions on J for the utility function to be convex or concave.

7 Conclusions and Future Work When we view the cost function as a random variable, it is natural to optimize the distribution of the cost function. In this way, classical LQG and RS control become special cases of statistical control, where the first cumulant and sum of all the cumulants are optimized, respectively. We have formulated and presented solutions to the MCV control problem where the second cumulant, variance, is minimized. This concept is generalized to include kth cumulant and also multiobjective functions. In developing this general concept, we extended the mixed H2 /H∞ and H∞ game theory using cost cumulants. Full-state feedback statistical control is almost completely developed. We also have some output feedback results for linear systems with quadratic cost functions. However, nonlinear system, nonquadratic cost statistical control is an open area. Infinite time horizon statistical control also needs more research. There are results using statistical control in structural vibration control applications, but more applications are needed. Statistical control is intuitive and provides an ability to shape the performance of the system through cost cumulants. Further development of this area will lead to more exciting results.

Introduction and Literature Survey of Statistical Control

23

References [Ast70] [Ath71]

[BB91] [BV85]

[Ben92] [Ber76] [Bor89] [BH75]

[CJ93]

[Cos69]

[Dav77] [Die06] [Die07]

[Doo53] [Doy89]

[FR75] [FS92] [FM92]

[FW84] [Gar85]

˚ K. J. Astrom, Introduction to Stochastic Control Theory, New York: Academic Press, 1970. M. Athans, The Role and Use of the Stochastic Linear-Quadratic-Gaussian Problem in Control System Design, IEEE Transactions on Automatic Control, AC-16, Number 6, pp. 529–551, December 1971. T. Bas¸ar and P. Bernhard, H∞ –Optimal Control and Related Minimax Design Problems, Boston: Birkh¨auser, 1991. A. Bensoussan and J. H. van Schuppen, Optimal Control of Partially Observable Stochastic Systems with an Exponential-of-Integral Performance Index, SIAM Journal on Control and Optimization, Volume 23, pp. 599–613, 1985. A. Bensoussan, Stochastic Control of Partially Observable Systems, London: Cambridge University Press, 1992. D. P. Bertsekas, Dynamic Programming and Stochastic Control, London: Academic Press, 1976. V. S. Borkar, Optimal Control of Diffusion Processes, England: Longman Scientific & Technical, 1989. A. E. Bryson, Jr., and Y.-C. Ho, Applied and Optimal Control, Optimization, Estimation, and Control, Revised Printing, New York: Hemisphere Publishing Co., 1975. M. C. Campi and M. R. James, Risk-Sensitive Control: A Bridge Between H2 and H∞ Control, Proceedings of the 32nd Conference on Decision and Control, San Antonio, TX, December 1993. L. Cosenza, On the Minimum Variance Control of Discrete-Time Systems, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, January 1969. M. H. A. Davis, Linear Estimation and Stochastic Control, London: Halsted Press, 1977. R. W. Diersing, H∞ , Cumulants and Games, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, 2006. R. W. Diersing, M. K. Sain, and C.-H. Won, Bi-Cumulant Games: A Generalization of H-infinity and H2/H-infinity Control, IEEE Transactions on Automatic Control, submitted, 2007. J. L. Doob, Stochastic Processes, New York: John Wiley & Sons, Inc., 1953. J. Doyle, K. Glover, P. Khargonekar, and B. Francis, State-Space Solutions to Standard H2 and H∞ Control Problems, IEEE Transactions on Automatic Control, Vol. 34, No. 8, pp. 831–847, 1989. W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, New York: Springer-Verlag, 1975. W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, New York: Springer-Verlag, 1992. W. H. Fleming and W. M. McEneaney, “Risk Sensitive Optimal Control and Differential Games,” Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 185–197, 1992. M. I. Freidlin and A. D. Wentzell, Random Perturbations of Dynamical Systems, New York: Springer-Verlag, 1984. C. W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences, 2nd ed., New York: Springer-Verlag, 1985.

24 [GS69] [GS72] [GS79] [GD88]

[Glo89]

[Hop94]

[Igl01a] [Igl01b]

[Jac73]

[Jam92]

[Jam94]

[Kal60] [Kar87] [KV81]

[Kus71] [KS72] [Lib71] [LH76]

[LH78]

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich I. I. Gihman and A. V. Skorohod, Introduction to the Theory of Random Processes, London: W. B. Saunders, 1969. I. I. Gihman and A. V. Skorohod, Stochastic Differential Equations, New York: Springer-Verlag, 1972. I. I. Gihman and A. V. Skorohod, Controlled Stochastic Processes, New York: Springer-Verlag, 1979. K. Glover and J. C. Doyle, State-Space Formulae for All Stabilizing Controllers That Satisfy an H∞ -Norm Bound and Relations to Risk Sensitivity, Systems and Control Letters, Volume 11, pp. 167–172, 1988. K. Glover, Minimum Entropy and Risk-Sensitive Control: The Continuous Time Case, Proceedings 28th IEEE Conference on Decision and Control, pp. 388–391, December 1989. W. E. Hopkins, Jr., Exponential Linear Quadratic Optimal Control with Discounting, IEEE Transactions on Automatic Control, AC-39, No. 1, pp. 175–179, 1994. P. A. Iglesias, Tradeoffs in Linear Time-Varying Systems: An Analogue of Bode’s sensitivity integral, Automatica, Vol. 37, pp. 1541–1550, 2001. P. A. Iglesias, “An analogue of Bode’s integral for stable non linear systems: Relations to entropy,” Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, FL, pp. 3419–3420, December 2001. D. H. Jacobson, Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relationship to Deterministic Differential Games, IEEE Transactions on Automatic Control, AC-18, pp. 124–131, 1973. M. R. James, Asymptotic Analysis of Nonlinear Stochastic Risk-Sensitive Control and Differential Games, Mathematics of Control, Signals, and Systems, 5, pp. 401–417, 1992. M. R. James, J. S. Baras, and R. J. Elliott, Risk-Sensitive Control and Dynamic Games for Partially Observed Discrete-Time Nonlinear Systems, IEEE Transactions on Automatic Control, AC-39, No. 4, pp. 780–792, 1994. R. E. Kalman, Contributions to the Theory of Optimal Control, Bol. de Soc. Math. Mexicana, p. 102, 1960. I. Karatzas and S. E. Shereve, Brownian Motion and Stochastic Calculus, New York: Springer-Verlag, 1987. P. R. Kumar and J. H. van Schuppen, On the Optimal Control of Stochastic Systems with an Exponential-of-Integral Performance Index, Journal of Mathematical Analysis and Applications, Volume 80, pp. 312–332, 1981. H. Kushner, Introduction to Stochastic Control, New York: Holt, Rinehart and Winston, Inc., 1971. H. Kwakernaak and R. Sivan, Linear Optimal Control Systems, New York: John Wiley & Sons, Inc., 1972. S. R. Liberty, “Characteristic Functions of LQG Control,” Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, August 1971. S. R. Liberty and R. C. Hartwig, On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants, Information and Control, Volume 32, Number 3, pp. 276–305, 1976. S. R. Liberty and R. C. Hartwig, Design-Performance-Measure Statistics for Stochastic Linear Control Systems, IEEE Transactions on Automatic Control, AC-23, Number 6, pp. 1085–1090, December 1978.

Introduction and Literature Survey of Statistical Control [LAH94]

[MG90] [Oks89] [PI97]

[PBGM62]

[PSL02]

[PSL04]

[Pha04]

[RS93] [Run94]

[Sag68] [Sai65]

[Sai66]

[Sai67] [SS68]

[SL71]

[SWS92]

25

D. J. N. Limebeer, B. D. O. Anderson, and D. Hendel, A Nash Game Approach to Mixed H2 /H∞ Control, IEEE Transactions on Automatic Control, Vol. 39, Number 1, pp. 69–82, January 1994. Minimum entropy H∞ control, Vol. 146 of Lecture Notes in Control and Information Sciences, Springer-Verlag, Berlin, 1990. B. Øksendal, Stochastic Differential Equations, An Introduction with Applications, Second Edition, New York: Springer-Verlag, 1989. M. A. Peters and P. A. Iglesias, Minimum Entropy Control for Time-Varying Systems, Systems and Control: Foundations & Applications, Birkh¨auser, Boston, 1997. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkriledze, and E. F. Mischenko, The Mathematical Theory of Optimal Processes, New York: Interscience Publishers, 1962. K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: StateFeedback, Finite-Horizon Paradigm with Applications to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002. K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants, Proceedings American Control Conference, Boston, MA, USA, pp. 691–696, June 2004. K. D. Pham, “Statistical Control Paradigm for Structural Vibration Suppression”, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, May 2004. L. Ray and R. Stengel, A Monte Carlo Approach to the Analysis of Control System Robustness, Automatica, Vol. 29, No. 1, pp. 229–236, 1993. T. Runolfsson, The Equivalence Between Infinite-Horizon Optimal Control of Stochastic Systems with Exponential-of-Integral Performance Index and Stochastic Differential Games, IEEE Transactions on Automatic Control, Vol. 39, No. 8, pp. 1551–1563, 1994. A. P. Sage, Optimum Systems Control. Englewood Cliffs, NJ: Prentice-Hall Inc., 1968. M. K. Sain, On Minimal-Variance Control of Linear Systems with Quadratic Loss, Ph.D Thesis, Department of Electrical Engineering and Coordinated Science Laboratory, University of Illinois, Urbana, IL, January 1965. M. K. Sain, Control of Linear Systems According to the Minimal Variance Criterion—A New Approach to the Disturbance Problem, IEEE Transactions on Automatic Control, AC-11, No. 1, pp. 118–122, January 1966. M. K. Sain, Performance Moment Recursions, with Application to Equalizer Control Laws, Proc. 5th Allerton Conference, pp. 327–336, 1967. M. K. Sain and C. R. Souza, A Theory for Linear Estimators Minimizing the Variance of the Error Squared, IEEE Transactions on Information Theory, IT14, Number 5, pp. 768–770, September 1968. M. K. Sain and S. R. Liberty, Performance Measure Densities for a Class of LQG Control Systems, IEEE Transactions on Automatic Control, AC-16, Number 5, pp. 431–439, October 1971. M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulant Minimization and Robust Control, Stochastic Theory and Adaptive Control, Lecture Notes

26

Chang-Hee Won, Ronald W. Diersing, and Stephen Yurkovich

in Control and Information Sciences 184, T. E. Duncan and B. Pasik-Duncan (Eds.), Springer-Verlag, pp. 411–425, 1992. [SWS95] M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants in RiskSensitive Control: The Full-State Feedback Cost Variance Case, Proceedings of the Conference on Decision and Control, New Orleans, LA, pp. 1036–1041, 1995. [SWS00] M. K. Sain, Chang-Hee Won, and B. F. Spencer, Jr., Cumulants and Risk Sensitive Control: A Cost Mean and Variance Theory with Applications to Seismic Protection of Structures, Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games, Vol. 5, J. A. Filor, V. Gaisgory, K Mizukami (Eds), Birkh¨auser, Boston, 2000. [SSSS92] P. Sain, M., B. F. Spencer, Jr., M. K. Sain, and J. Suhardjo, Structural Control Design in the Presence of Time Delays, Proceedings of the ASCE Engineering Mechanics Conference, College Station, TX, pp. 812–815, 1992. [SSWKS93] B. F. Spencer, M. K. Sain, C.-H. Won, D. C. Kaspari, and P. M. Sain, ReliabilityBased Measures of Structural Control Robustness, Structural Safety, 15, pp. 111–129, 1993. [SDJ74] J. L. Speyer, J. Deyst, and D. H. Jacobson, Optimization of Stochastic Linear Systems with Additive Measurement and Process Noise Using Exponential Performance Criteria, IEEE Transactions on Automatic Control, AC-19, No. 4, pp. 358–366, August 1974. [Spe76] J. L. Speyer, An Adaptive Terminal Guidance Scheme Based on an Exponential Cost Criterion with Application to Homing Missile Guidance, IEEE Transactions on Automatic Control, AC-21, pp. 371–375, 1976. [SS96] A. A. Stoorvogel and J. H. Van Schuppen, System Identification with Information Theoretic Criteria, Identification, Adaptation, Learning: The Science of Learning Models from Data (NATO Asi Series. Series F, Computer and Systems Sciences, Vol. 153, Sergio Bittanti (Editor), Giorgio Picci (Editor), Springer, Berlin, 1996, pp. 289–338. [Uch89] K. Uchida and M. Fujita, On the Central Controller: Characterizations via Differential Games and LEQG Control Problems, Systems & Control Letters, Volume 13, pp9–13, 1989. [Whi81] P. Whittle, Risk-Sensitive Linear/Quadratic/Gaussian Control, Advances in Applied Probability, Vol. 13, pp. 764–777, 1981. [Whi90] P. Whittle, Risk Sensitive Optimal Control, New York: John Wiley & Sons, 1990. [Whi91] P. Whittle, A Risk-Sensitive Maximum Principle: The Case of Imperfect State Observation, IEEE Transactions on Automatic Control, Vol. 36, No. 7, pp. 793– 801, July 1991. [WK86] P. Whittle and J. Kuhn, A Hamiltonian formulation of risk-sensitive linear/ quadratic/Gaussian control, International Journal of Control, Vol. 43, pp. 1–12, 1986. [WSS94] C.-H. Won, M. Sain, and B. Spencer, Risk-Sensitive Structural Control Strategies, Proceedings of the Second International Conference on Computational Stochastic Mechanics, Athens, Greece, June 13–15, 1994. [Won95] C.-H. Won, Cost Cumulants in Risk-Sensitive and Minimal Cost Variance Control, Ph.D. Dissertation, University of Notre Dame, Notre Dame, IN, 1995. [WSL03] C.-H. Won, M. K. Sain, and S. Liberty. Infinite Time Minimal Cost Variance Control and Coupled Algebraic Riccati Equations Proceedings American Control Conference, Denver, Co, pp. 5155–5160, June 4–6, 2003.

Introduction and Literature Survey of Statistical Control [Won04]

[Won05]

[Won63] [Won74]

27

C.-H. Won, Cost Distribution Shaping: The Relations Between Bode Integral, Entropy, Risk-Sensitivity, and Cost Cumulant Control, Proceedings American Control Conference, Boston, MA, pp. 2160–2165, June 2004. C.-H. Won, Nonlinear n-th Cost Cumulant Control and Hamilton-JacobiBellman Equations for Markov Diffusion Process, Proceedings of 44th IEEE Conference on Decision and Control, Seville, Spain, pp. 4524–4529, 2005. W. M. Wonham, Stochastic Problems in Optimal Control, 1963 IEEE Int. Conv. Rec., part 2, pp. 114–124, 1963. W. M. Wonham, Linear Multivariable Control, A Geometric Approach, Lecture Notes in Economics and Mathematical Systems, 101, Springer-Verlag, 1974.

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case Luis Cosenza,1 Michael K. Sain,2 Ronald W. Diersing,3 and Chang-Hee Won4 1 2 3 4

Apartado 4461 Tegucigalpa, Honduras Central America. luis [email protected] Freimann Professor of Electrical Engineering, University of Notre Dame, Notre Dame, IN 46556, USA. [email protected] Department of Engineering, University of Southern Indiana, Evansville, IN 47712, USA. [email protected] Department of Electrical and Computer Engineering, Temple University, Philadelphia, PA 19122, USA. [email protected]

Summary. The expected value of a random cost may be viewed either as its first moment or as its first cumulant. Recently, the Kalman control gain formulas have been generalized to finite linear combinations of cost cumulants, when the systems are described in continuous time. This paper initiates the investigation of cost cumulant control for discrete-time systems. The cost variance is minimized, subject to a cost mean constraint. A new version of Bellman’s optimal cost recursion equation is obtained and solved for the case of full-state measurement. Application is made to the First Generation Structural Benchmark for seismically excited buildings.

1 Introduction The 1960s saw a burst of controls research whose impact upon theory and application has continued to this day, without much measurable lessening. Pivotal in this burst was the pioneering work of R. E. Kalman, embracing concepts such as linearquadratic-Gaussion (LQG) control, with linear dynamical systems, quadratic costs, and Gaussian noises. Kalman considered both discrete-time and continuous-time linear system models, and imported the ideas of Lyapunov analysis to incorporate notions of uniform controllability, uniform observability, and uniform asymptotic stability. The separation principle, Kalman–Bucy and Kalman filters, and the Kalman optimal control gain formulae, have become commonplace. The approach of Kalman to LQG problems was, of course, based upon minimizing the average cost. We remark that the average cost is the first entry in two famous sequences of random cost statistics. The first sequence is that of the cost moments; the second sequence is that of the cost cumulants. Without more information it would not be possible to surmise whether Kalman’s formulae derived their efficacy from the average cost being a moment or from the average cost being a cumulant. Recently, however, K. D. Pham, [Pha04], C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 2, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

30

Luis Cosenza, Michael K. Sain, Ronald W. Diersing, and Chang-Hee Won

[PSL02-1], [PSL02-2], has shown that the Kalman separation principle, filter, and optimal control gains generalize naturally to optimal control problems based upon finite linear combinations of cost cumulants. This suggests strongly that the successful operative methods in the Kalman advances were cost-cumulant enabled. Khanh’s work was in continuous-time. Moreover, the cost-cumulant control strategy families studied by Pham also display many of the same desirable features known to LQG designers. Indeed, Pham has carried out very promising applications of these algorithms to cable-stayed bridges [PSL04], structures excited by wind [PJSSL04], and buildings shaken by earthquakes [PSL02-3]. In view of these developments, it is both natural and desirable to examine the corresponding research issues for the other family of systems studied by Kalman, those in discrete time. This paper initiates such investigations. Cost variance is minimized, subject to cost mean constraint. A new version of Bellman’s optimal cost recursion equation is obtained, and solved for the case of full-state measurement. The theory is based upon the dissertation by Cosenza [Cos69]. Application is made to the First Generation Structural Benchmark for seismically excited buildings [SDD98].

2 Problem Definition Let I be a subset of the integers and R1 be the 1-fold product of the real line. Consider then the systems whose behavior is governed by the following stochastic difference equations: x( j + 1) = f ( j, x( j), u( j), w( j)) , y( j) = g( j, x( j), v( j)) ,

x(n0 ) = x0 ,

(1) (2)

where x( j) ∈ Rn is the system state, u( j) ∈ Rm is the control input, w( j) ∈ R p is the actuation noise, y( j) ∈ Rq is the system output, and v( j) ∈ Rr is the measurement noise, j ∈ I. The initial condition of equation (1) is given by x(n0 ), where n0 is the smallest element in I. Let f : I × Rn × Rm × R p → Rn and g : I × Rn × Rr → Rq be Borel measurable, with the probability density functions of w( j), v( j), and x0 given, j ∈ I. Define U( j)  {u(n0), u(n0 + 1), . . . , u( j)}, with a similar definition made for the remaining variables of equations (1) and (2), and let Z( j)  {Y ( j),U( j − 1)}, n0 < j, with Z(n0 ) = y(n0 ). It is then possible to denote the unique solution of equation (1) satisfying the initial condition x(n0 ) = x0 by θ ( j), where θ ( j)  θ ( j; n0 , x0 ;U( j − 1),W ( j − 1)), j ∈ I, and to specify that the control laws be of the form k( j)  k( j, Z( j)), where k( j, ·, ·) : Rq( j−n0 +1) × Rm( j−n0 ) → Rm , j ∈ I. Observe that Z( j) contains all the information available to the controller at time j, and that the form chosen for k( j), together with a boundedness requirement, contributes to the definition of the class of admissible controls.

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case

31

With the definitions and notation recently introduced it is now possible to formulate a performance index as J(n0 )  J(n0 , x0 ;U(N − 1),W (N − 1)), where N

J(n0 ) =

∑

L( j, θ ( j), u( j − 1)) ,

(3)

j=n0 +1

and where the loss function L : I × Rn × Rm → R+ (the nonnegative real line) is Borel measurable. Since f (·, ·, ·, ·), g(·, ·, ·) and L(·, ·, ·) are all Borel measurable, the performance index is a random variable and consequently one of its statistical moments must be selected for optimization. In this investigation it is desired to minimize the variance of J(n0 ) while its mean is forced to obey a constraint. In mathematical parlance, it is desired to find k( j, Z( j)), n0 ≤ j ≤ N − 1, such that   E J 2 (n0 )|Z(n0 ) − E 2 {J(n0 )|Z(n0 )} (4) is minimized, while E{J(n0)|Z(n0 )} = h(n0 , Z(n0 )) ,

(5)

where E{·|·} denotes the conditional expectation operator. The form of the function h : I × Rq → R+ is selected a priori based on practical considerations, such as desired response, permissible deviations from the desired response, complexity of the controller, etc. Observe that the choice of h(n0 , Z(n0 )) is not entirely arbitrary, for if

α (n0 , Z(n0 )) = inf E{J(n0 )|Z(n0 )} , U(N−1)

(6)

then h(n0 , Z(n0 )) must always be greater than α (n0 , Z(n0 )). This constraint on h, together with equation (5), completes the definition of the class of admissible controls.

3 Recursion Equation In this section, a recursion equation for the optimal variance cost is derived. The procedure employed is the standard procedure for this type of problem; first, the constraint equation is appended to the expression to be minimized by means of a Lagrange multiplier, μ (n0 ), and then the resulting equation is imbedded into the more general class of problems where n0 is a variable rather than a fixed initial time. It is clear that the solution of the more general problem leads trivially to the solution of the problem posed in Section 2. Consequently, it is desired to find μ ( j) and k(i, Z(i)), j ≤ i ≤ N − 1, j ∈ I, such that   E J 2 ( j)|Z( j) − E 2 {J( j)|Z( j)} + 4 μ ( j) [E{J( j)|Z( j)} − h( j, Z( j))] (7) is minimized, where μ ( j) ∈ R is a Lagrange multiplier, and where the 4 premultiplying μ ( j) has been introduced just for convenience.

32

Luis Cosenza, Michael K. Sain, Ronald W. Diersing, and Chang-Hee Won

Before proceeding with the development of the recursion equation, however, let k j  {k( j), k( j + 1), . . . , k(N − 1)}, j ∈ I, and let   VC( j, Z( j)|k j ) = E J 2 ( j)|Z( j) − E 2 {J( j)|Z( j)} (8) + 4 μ ( j) [E{J( j)|Z( j)} − h( j, Z( j))] , j ∈ I , where VC signifies “variance cost.” Define VC0 (N − 1, Z(N − 1)) to be the optimal value of VC(N − 1, Z(N − 1)|kN−1 ), that is,    VC0 (N − 1, Z(N − 1)) = E L2 (N)|Z(N − 1) min k(N−1), μ (N−1)

− E 2 {L(N)|Z(N − 1)} + 4 μ (N − 1)[E{L(N)|Z(N − 1)}  − h(N − 1, Z(N − 1))] , (9) where L( j)  L( j, θ ( j), k( j − 1)). Note in particular that if k0 (N − 1) is the control law which leads to VC0 (N − 1, Z(N − 1)), then

where

E{L(N, θ0 (N), k0 (N − 1))|Z(N − 1)} = h(N, Z(N − 1)) ,

(10)

θ0 (N) = f (N − 1, θ (N − 1), k0(N − 1), w(N − 1)) ,

(11)

and therefore, combining equations (10) and (9) it follows that   VC0 (N − 1, Z(N − 1)) = E L2 (N, θ0 (N), k0 (N − 1))|Z(N − 1) − E 2 {L(N, θ0 (N), k0 (N − 1))|Z(N − 1)} .

(12)

Similarly, VC0 (N − 2, Z(N − 2)) =

min

kN−2 , μ (N−2)

   E (L(N) + L(N − 1))2 |Z(N − 2)

− E 2{L(N) + L(N − 1)|Z(N − 2)} + 4 μ (N − 2)[E{L(N) + L(N − 1)|Z(N − 2)}  − h(N − 2, Z(N − 2))] ,

(13)

which after some manipulation may be written as  VC0 (N − 2, Z(N − 2)) = minkN−2 ,μ (N−2) Γ (N − 2) − E 2{L(N)|Z(N − 2)}     +E E L2 (N)|Z(N − 1) |Z(N − 2) +2E{L(N)L(N − 1)|Z(N − 2)}

 −2E{L(N)|Z(N − 2)}E{L(N − 1)|Z(N − 2)} , (14)

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case

33

where   Γ (N − 2) = E L2 (N − 1)|Z(N − 2) − E 2{L(N − 1)|Z(N − 2)} + 4 μ (N − 2)[E{L(N) + L(N − 1)|Z(N − 2)} − h(N − 2, Z(N − 2))] .

(15)

  If now E E 2 {L(N)|Z(N − 1)}|Z(N − 2) is added and subtracted from equation (14), then    VC0 (N − 2, Z(N − 2)) = min Γ (N − 2) + E{E L2 (N)|Z(N − 1) kN−2 , μ (N−2)

+ 2E{L(N)L(N − 1)|Z(N − 2)} − 2E{L(N)|Z(N − 2)}E{L(N − 1)|Z(N − 2)} + E{E 2{L(N)|Z(N − 1)}|Z(N − 2)} − E 2 {L(N)|Z(N − 2)}

 − E 2 {L(N)|Z(N − 1)}|Z(N − 2)} .

(16)

However, since the process under consideration is a multistage decision process, the principle of optimality may be applied to it, and equation (16) then becomes  VC0 (N − 2, Z(N − 2)) = min Γ0 (N − 2) k(N−2), μ (N−2)

+2E{L0(N)L(N − 1)|Z(N − 2)} −2E{L0(N)|Z(N − 2)}E{L(N − 1)|Z(N − 2)} +E{E{L20(N)|Z(N − 1)} −E 2 {L0 (N)|Z(N − 1)}|Z(N − 2)} +E{E 2{L0 (N)|Z(N − 1)}|Z(N − 2)}  −E 2 {L0 (N)|Z(N − 2)} ,

(17)

where

Γ0 (N − 2) = E{L2 (N − 1)|Z(N − 2)} − E 2{L(N − 1)|Z(N − 2)} + 4 μ (N − 2)[E{L0(N) + L(N − 1)|Z(N − 2)} − h(N − 2, Z(N − 2))] , and L0 (N) = L(N, θ0 (N), k0 (N − 1)).

(18)

34

Luis Cosenza, Michael K. Sain, Ronald W. Diersing, and Chang-Hee Won

Furthermore, if equations (17) and (12) are combined, then  VC0 (N − 2, Z(N − 2)) = min Γ0 (N − 2) k(N−2), μ (N−2)

+ 2E{L0(N)L(N − 1)|Z(N − 2)} − 2E{L0(N)|Z(N − 2)}E{L(N − 1)|Z(N − 2)} + E{E 2{L0 (N)|Z(N − 1)}|Z(N − 2)} − E 2 {L0 (N)|Z(N − 2)}

 + E{VC0 (N − 1, Z(N − 1))|Z(N − 2)} .

Proceeding by induction, it follows that 

(19)

   VC0 (i, Z(i)) = min Γ0 (i) + 2E ∑ L0 ( j)L(i + 1)Z(i)  k(i), μ (i) j=i+2    N  − 2E ∑ L0 ( j)Z(i) E{L(i + 1)|Z(i)}  j=i+2       N   2 +E E ∑ L0 ( j)Z(i + 1) Z(i) j=i+2       N   − E2 E ∑ L0 ( j)Z(i + 1) Z(i) j=i+2 

N

+ E{VC0 (i + 1, Z(i + 1))|Z(i)} ,

(20)

where n0 ≤ i ≤ N − 2, VC0 (N − 1, Z(N − 1)) is as given by equation (9) and where

Γ0 (i) = E{L2 (i + 1)|Z(i)} − E 2{L(i + 1)|Z(i)}      N  + 4 μ (i) E ∑ L0 ( j) + L(i + 1)Z(i) − h(i, Z(i)) . j=i+2

(21)

Theorem 1. Consider the nonlinear problem given in (1) and (3). A solution k∗ is the optimal minimum cost variance (MCV) strategy, if there exists a solution VC0 (i, Z(i)) to    (22) VC0 (i, Z(i)) = min E{VC0 (i + 1, Z(i + 1))|Z(i)} + E L2 (i + 1)|Z(i) k(i)



 − E {L(i + 1)|Z(i)} + E E 2

2

N

∑

j=i+2

      L0 ( j)Z(i + 1) Z(i)

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case

35

 

  N   + E2 E ∑ L0 ( j)Z(i + 1) Z(i) j=i+2

  N  + 2E ∑ L0 ( j) L(i + 1)Z(i) j=i+2   N   − 2E ∑ L0 ( j)Z(i) E {L(i + 1)|Z(i)} j=i+2     N  + 4 μ (i) E ∑ L0 ( j) + L(i + 1)Z(i) − M(i, Z(i)) , j=i+2

where γ (k) is a Lagrange multiplier, L0 ( j) = L( j, x( j), k∗ ( j − 1)), and k∗ is the minimizing argument of (22). Proof. From the one-step analysis, we see that the variance cost is minimized. We need to prove by the method of induction that (22) holds. We shall assume that (22) holds for time i + 1. We now will need to show that with this assumption, equation (22) is valid for time i. By the definition of VC0 (i, Z(i)) we have  VC0 (i, Z(i)) = E{J 2 (i, x(i); k)|Z(i)} − E 2 {J(i, x(i); k)|Z(i)} min k(i),··· ,k(N−1)  + 4 μ (i) [E{J(i, x(i); k)|Z(i)} − M(i, Z(i))] , which by substitution gives V (i, Z(i)) =

min

k(i),··· ,k(N−1)

 E{(L(i + 1) + J(i + 1, x(i + 1); k))2|Z(i)}

− E 2{L(i + 1) + J(i + 1, x(i + 1); k)|Z(i)} + 4 μ (i) [E{J(i + 1, x(i + 1); k)|Z(i)} − M(i, Z(i))]  = min E{L2 (i + 1)|Z(i)}



k(i),··· ,k(N−1)

+ 2E{L(i + 1)J(i + 1, x(i + 1); k)|Z(i)} + E{J 2(i + 1, x(i + 1); k)|Z(i)} − E 2{L(i + 1)|Z(i)} − E 2{J(i + 1, x(i + 1); k)|Z(i)} − 2E{L(i + 1)|Z(i)}E{J(i + 1, x(i + 1); k)|Z(i)} + 4 μ (i) [E{J(i + 1, x(i + 1); k)|Z(i)} − M(i, Z(i))]



where J is given in (3) with i in place of 0. Now by using the principle of optimality we have  VC0 (i, Z(i)) = min E{L2 (i + 1)|Z(i)} − E 2{L(i + 1)|Z(i)} k(i)

36

Luis Cosenza, Michael K. Sain, Ronald W. Diersing, and Chang-Hee Won



N

∑

+ 2E

L( j)0 L(i + 1)|Z(i)

j=i+2



N

∑

− 2E



L0 ( j)|Z(i) E {L(i + 1)|Z(i)}

j=i+2

⎫ ⎫ ⎧ ⎧

2 ⎬ ⎬ ⎨ ⎨ N L0 ( j) |Z(i + 1) |Z(i) +E E ∑ ⎭ ⎭ ⎩ ⎩ j=i+2   N

∑

− E E2 

j=i+2

 

j=i+2

 +E E −E

2

N

∑

2

N

E 

∑

L0 ( j)|Z(i + 1) |Z(i)



L0 ( j)|Z(i + 1) |Z(i)



L0 ( j)|Z(i + 1) |Z(i)

j=i+2

+ 4 μ (i) E{

N

∑

 L0 ( j) + L(i + 1)|Z(i)} − M(i, Z(i))

,

j=i+2

where we still only have the mean constraint for time i. But for time i + 1, the mean constraint is satisfied if the optimal solution k∗ (i + 1, x(i + 1)) is played. Therefore equation (22) is satisfied for time i. 2 With this result we can now turn our attention to solving the special case when the system is linear and the cost is quadratic. We apply the nonlinear, nonquadratic cost results and get a recursion equation for this case. We then determine the optimal MCV strategy for full-state feedback information.

4 Linear Quadratic Case Let I again denote a subset of the integers with n0 as its smallest element and introduce Rm×n and Sn×n where Rm×n represents the linear space of m × n real matrices and Sn×n the real linear space of n × n symmetric matrices. Consider then the controllable system described by the following stochastic difference equations: x( j + 1) = A( j)x( j) + B( j)u( j) + w( j) , y( j) = x( j) ,

x(n0 ) = x0 ,

(23) (24)

where A( j) ∈ Rn×n is bounded and nonsingular, and B( j) ∈ Rn×m is bounded, j ∈ I. The actuation noise sequence, w( j), is a sequence of identically distributed, zero mean, independent Gaussian variables with covariance matrix given by E{w( j) >< w( j)} = QW ,

(25)

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case

37

where QW ∈ Sn×n is a time-invariant diagonal matrix, and where · >< · : Rn × Rn → Rn×n is the dyad. The loss function is given by L( j, θ ( j), k( j − 1)) = θ ( j), R( j − 1)θ + k( j − 1), P( j − 1)k( j − 1) ,

j ∈ I,

(26)

where < ·, · >: Rn × Rn → R is the Euclidean inner product and θ ( j) is the unique solution of equation (23) satisfying the initial condition x(n0 ) = x0 . R( j) and P( j) are positive definite,5 bounded, and symmetric for all j, j ∈ I. Similarly, the mean value constraint is given by h(n0 , Z(n0 )) = m(n0 ) + θ (n0 ), M(n0 )θ (n0 ) ,

(27)

where m(n0 ) ∈ R+ and the matrix M(n0 ) ∈ Sn×n must be bounded and positive definite. Both m(n0 ) and M(n0 ) must be selected such that h(n0 , Z(n0 )) > α (n0 , Z(n0 )) ,

(28)

where α (n0 , Z(n0 )) is as given by equation (6). The assumption of linear control laws leads naturally to quadratic optimal costs, that is, for linear control laws it is always possible to write VC0 (i + 1, Z(i + 1)) = v0 (i + 1) + θ (i),V0 (i + 1)θ (i) ,

(29)

where v0 (i) ∈ R+ and V0 (i) ∈ Sn×n is nonnegative definite, and where n0 ≤ i ≤ N − 1. Therefore, E{VC0 (i + 1, Z(i + 1))|Z(i)} = v0 (i + 1) + β (i),V0 (i + 1)β (i) + Tr {V0 (i + 1)QW } .

(30)

If the following definition is introduced, RM (i) = R(i) + M(i + 1) for n0 ≤ i ≤ N − 1, and the terminal conditions are given as m(N) = 0, M(N) = 0, v0 (N) = 0, and V0 (N) = 0, then with some mathematical manipulations we have  VC0 (i, Z(i)) = min 4 β (i), RM (i)QW RM (i)β (i) k(i), μ (i)   + E w(i), RM (i)w(i) 2 − Tr2 {RM (i)QW } + v0 (i + 1)  + β (i),V0 (i + 1)β (i) + Tr{V0 (i + 1)QW } + 4 μ (i) m(i + 1) + k(i), P(i)k(i) + β (i), RM (i)β (i) + Tr {RM (i)QW }  − m(i) − θ (i), M(i)θ (i) , n0 ≤ i ≤ N − 1, 5 The

(31)

assumptions that QW be diagonal and R( j) be positive definite have been made for convenience only.

38

Luis Cosenza, Michael K. Sain, Ronald W. Diersing, and Chang-Hee Won

where β (i) = A(i)θ (i) + B(i)k(i). Performing the minimization with respect to k(i), the optimal MCV controller is given as  −1 T k0 (i) = K0 (i)θ (i) = − BT (i)Λ (i)B(i) + μ (i)P(i) B (i)Λ (i)A(i)θ (i), where

Λ (i) = RM (i)QW RM (i) + V0(i + 1)/4 + μ (i)RM (i)

(32) (33)

for n0 ≤ i ≤ N − 1. Using this optimal controller and performing the minimization in terms of μ (i) we have the mean constraint M(i) = K0T (i)P(i)K0 (i) + AT0 (i)RM (i)A0 (i) m(i) = m(i + 1) + Tr{RM (i)QW }

(34)

and we also have the variance V0 (i) = AT0 (i) [4RM (i)QW RM (i) + V0(i + 1)]A0 (i)   v0 (i) = v0 (i + 1) + Tr{V0 (i + 1)QW } + E w(i), RM (i)w(i) 2 − Tr2 {RM (i)QW } , (35) where A0 (i) = A(i) + B(i)K0 (i) is the closed loop A matrix and n0 ≤ i ≤ N − 1. It is important to understand the differences between the recursion equations of a minimum mean problem and those of a minimum cost variance problem. In a minimum mean problem, the solution of the recursion equations leads to the minimum of the expected value of a performance index and to its corresponding control law. In a minimum cost variance problem, subsequent to the selection of μ (i), n0 ≤ i ≤ N − 1, solution of the recursion equations leads to a mean value of the performance index together with its corresponding minimum cost variance and optimal control law. By properly altering μ (i), n0 ≤ i ≤ N − 1, several such sets of expected values, minimum cost variances, and optimal control laws may be obtained. Clearly then, the amount of information which the optimization procedure herein employed furnishes concerning the performance index far exceeds that supplied by its mean value counterpart. Furthermore, observe that the minimum mean problem is a particular case of the problem herein solved, namely, it is the solution of the recursion equations in the limit as μ (i) approaches infinity, n0 ≤ i ≤ N − 1. Similarly, it may be shown that when it is possible to set μ (i) equal to zero, n0 ≤ i ≤ N − 1, then one obtains the solution of a MCV problem with no constraint on the mean value of the performance index. Generally speaking, such “pure” cost variance minimizations are not available in continuous time. It is of interest to observe that the minimum cost variance corresponding to the smallest mean is finite. More interesting, however, is the fact that under certain conditions there exists a finite mean value whose corresponding V0 (i) is zero, that is, there exists a finite mean value whose corresponding minimum cost variance is independent of the initial conditions. To prove this assertion, suppose QW and B(i) are nonsingular, n0 ≤ i ≤ N − 1. Then, replacing μ (i) by zero in the recursion equations, it follows that

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case

K0 (i) = B−1 (i)A(i) ,

n0 ≤ i ≤ N − 1,

39

(36)

which, from equation (35), implies that V0 (i) is zero, n0 ≤ i ≤ N − 1. In the preceding paragraph it was hinted that it is not always possible to replace μ (i) by zero, the reason being that the solution of the recursion equations is contingent upon the nonsingularity of the matrix BT (i)Λ (i)B(i) + μ (i)P(i), n0 ≤ i ≤ N − 1.

5 Application to First Generation Structural Benchmark for Earthquakes With the theory now well established, the control algorithm discussed is applied to the First Generation Structural Benchmark under seismic excitation. The structure under consideration is a three-story building excited by an earthquake. For control purposes, the building has an active mass driver on the third floor. The benchmark problem has a 28-state evaluation model. In the interest of control, a 10-state design model is used. For more details on the building, models, and the discussion of the performance criteria, the reader is encouraged to refer to [SDD98]. The benchmark control design model is a continuous-time model, so to apply the results in this paper, 0.8 LQG MCV −8.4%

0.7

0.6

0.5

−16.6%

0.4 −32.2% 0.3 −30.0% 0.2

0.1

0

J1

J2

J6

Fig. 1. Building performance.

J7

40

Luis Cosenza, Michael K. Sain, Ronald W. Diersing, and Chang-Hee Won

the model is discretized. Furthermore the state and control weighting matrices, R( j) and P( j), are respectively selected to be 0.1I10 and 50, where I10 is the 10 by 10 identity matrix. For the MCV control, the parameter, μ , is selected to be 1.3 × 106. Simulation results appear in Figure 1 and Figure 2. The results in Figure 1 represent the performance of the building. It is seen that there is a significant reduction for each of these performance criteria. For the rootmean-square criteria, J1 and J2 , there is about a 30% reduction in the MCV case from the LQG controller results. For peak response of the building, there is also a notable decrease in the performance criteria. There is about a 16% reduction for J6 and an 8.4% reduction for J7 . With this reduction in the civil engineering criteria that deal with the building performance, the question becomes: What about the criteria that deal with the control effort? As would seem likely, the increase in performance corresponds with an increase in control effort, as seen in Figure 2. Despite this increase over the LQG case, the control is still within the constraint imposed on the control in the benchmark problem. This suggests that the MCV control makes more efficient use of the control resources available.

1.4 LQG MCV

109.0%

84.4%

29.0%

1.2

1

0.8 57.2%

55.4%

27.8%

0.6

0.4

0.2

0

J3

J4

J5

J8

Fig. 2. Control effort.

J9

J10

Cumulant Control Systems: The Cost-Variance, Discrete-Time Case

41

6 Conclusion A new version of the Bellman recursion equation for optimal cost variance has been obtained for the problem of minimizing the variance of a cost, given a constraint upon the cost mean. Although emphasis has been placed upon linear dynamical systems in discrete time, some of the steps were carried out for nonlinear, nonquadratic cases. A complete solution of the recursion has been obtained for the case of fullstate measurements. However, the general recursion has been derived for the case of noisy measurements, and the next step of the research is to complete the solution for that case. The MCV controller was then applied to the First Generation Benchmark for seismically excited structures. The results were compared to those of the LQG control. The MCV controller showed substantial improvement over the LQG results, while observing the given control constraints.

References [Cos69]

[PSL04]

[PJSSL04]

[PSL02-1]

[PSL02-2]

[PSL02-3]

[Pha04]

[SDD98]

L. Cosenza, On the Minimum Variance Control of Discrete-Time Systems, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, Jan. 1969. K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants , Proceedings American Control Conference, pp. 691–696, Boston, Massachusetts, June 30, 2004. K. D. Pham, G. Jin, M. K. Sain, B. F. Spencer, Jr., and S. R. Liberty, Generalized LQG Techniques for the Wind Benchmark Problem, Special Issue of ASCE Journal of Engineering Mechanics on the Structural Control Benchmark Problem, Vol. 130, No. 4, April 2004. K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: StateFeedback, Finite-Horizon Paradigm with Application to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002. K. D. Pham, M. K. Sain, and S. R. Liberty, Finite Horizon Full-State Feedback kCC Control in Civil Structures Protection, Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences, Proceedings of a Workshop held in Lawrence, Kansas, Edited by B. Pasik-Duncan, Springer-Verlag, Berlin-Heidelberg, Germany, Vol. 280, pp. 369–383, September 2002. K. D. Pham, M. K. Sain, and S. R. Liberty, Robust Cost-Cumulants Based Algorithm for Second and Third Generation Structural Control Benchmarks, Proceedings American Control Conference, pp. 3070–3075, Anchorage, Alaska, May 8– 10, 2002. K. D. Pham, Statistical Control Paradigms for Structural Vibration Suppression, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, May 2004. B. F. Spencer Jr., S. J. Dyke, and H. S. Deoskar, Benchmark Problems in Structural Control - Part I: Active Mass Driver System, Earthquake Engineering and Structural Dynamics, Vol. 27, pp. 1127–1139, 1998.

Statistical Control of Stochastic Systems Incorporating Integral Feedback: Performance Robustness Analysis Khanh D. Pham Space Vehicles Directorate, Air Force Research Laboratory, Kirtland AFB, NM 87117, USA

Summary. An innovative paradigm for statistical approximation is presented to evaluate performance measure statistics of a class of stochastic systems with integral control. This methodology, which makes use of both compactness from the logic of the state-space model description and quantitativity from the probabilistic knowledge of stochastic disturbances, now allows us to predict more accurately the effect of a chi-squared random variable on the performance uncertainty of the stochastic system with both state feedback and integral output feedback. It is shown that the computational method detailed herein is able to calculate the exact statistics of the performance measure of any orders which are then utilized in the design of an optimal statistical control solution to effectively address the unresolved challenge of closed-loop performance robustness without massive Monte Carlo simulations.

1 Introduction Previously considered statistical controllers in [PSL02a] and [PhR07] for vibration suppression purposes all have one deficiency in that they do not improve the type of the systems. As a result, these state- and output-feedback control paradigms are generally useful only for stochastic regulator systems for which the systems do not track inputs. Since most control systems must often track inputs, a more complex task with a required tracking performance must be part of the statistical control structure in addition to vibration suppression properties. One configuration solution to this kind of controller is to introduce integral control, just as with the proportionalintegral controller, which then assures zero steady-state tracking error. Investigations by [FMT80] and [Joh68] are good examples of adding an integral of the plant position to the plant state-space representation. When dealing with stochastic uncertainties that propagate through a time-varying system and a quadratic performance measure, there is a growing interest in addressing the uncertainty and robustness of system performance with respect to all the realizations of the underlying stochastic process. The innovative work proposed herein extends early deterministic results in [FMT80] and [Joh68] as well as recent stochastic developments in [PSL02a]– [PhR07] for a general performance index using the concept of statistical optimality C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 3, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

44

Khanh D. Pham

where the accuracy of both state regulation and tracking error are now assessed with a rich set of performance measure statistics. This chapter contains 1) some discussion on the formulation of a class of linear time-varying stochastic systems involving stochastic uncertainty and integralquadratic performance measure statistics representing a combination of transient performance and operating costs; 2) detailed discussions on statistical control statements and optimal solution concepts for the problems with state and integral output feedback.

2 Performance Measure Statistics Consider a class of stochastic systems with integral output feedback modeled on [t0 ,t f ] and governed by dxr (t) = (Ar (t)xr (t) + Br (t)u(t))dt + Er (t)dw(t), yr (t) = Cr (t)xr (t) + Dr (t)u(t), dxt (t) = (r(t) − yr (t))dt,

xr (t0 )

(1) (2)

xt (t0 ),

(3)

where the process noise w(t)  w(t, ω ) : [t0 ,t f ] × Ω → R p is a stationary Wiener process starting from t0 , independent of the initial by know system states xr (t0 )  x0r and xt (t0 )  xt0 , and defined with {Ft }t≥0 being its filtration on a complete filtered probability space (Ω , F the correlation  , {Ft }t≥0 , P) over [t0 ,t f ] with  of independent increments E [w(τ ) − w(ξ )][w(τ ) − w(ξ )]T = W |τ − ξ | and W > 0. Moreover, xr (t)  xr (t, ω ) : [t0 ,t f ] × Ω → Rnr and xt (t)  xt (t, ω ) : [t0 ,t f ] × Ω → Rnt respectively are the regulating and tracking states in the Hilbert spaces L2Ft (Ω ; C ([t0 ,t f ]; Rnr )) and L2Ft (Ω ; C ([t0 ,t f ]; Rnt )) of Rnr - and Rnt -valued, square integrable processes  are adapted to the sigma field Ft gen  on [t0 ,t f ] that tf

erated by w(t) with E t0

xTr (τ )xr (τ )d τ

< ∞; y(t)  y(t, ω ) : [t0 ,t f ] × Ω → Rno

and r(t) ∈ L2 (C ([t0 ,t f ]; Rno )) are the output and reference input. System coefficients Ar ∈ C ([t0 ,t f ]; Rnr ×nr ), Br ∈ C ([t0 ,t f ]; Rnr ×m ), Cr ∈ C ([t0 ,t f ]; Rno ×nr ), Dr ∈ C ([t0 ,t f ]; Rno ×m ) and Er ∈ C ([t0 ,t f ]; Rnr ×p ) are deterministic, bounded matrixvalued functions. The set of admissible controls u ∈ L2Ft (Ω ; C ([t0 ,t f ]; Rm )) belongs to the Hilbert space of Rm -valued square integrable processes on [t0 ,t f ] that are adapted to the sigma field Ft generated by w(t). In addition, the stochastic system (1) in the absence of noisy disturbances is supposed to be uniformly exponentially stable. That is, there exist positive constants η1 and η2 such that the pointwise matrix norm of the state transition matrix associated with the closed-loop state transition matrix satisfies the inequality ||Φ (t, τ )|| ≤ η1 e−η2 (t−τ )

∀ t ≥ τ ≥ t0 .

The pair (A(t), B(t)) is stabilizable if there exists a bounded matrix-valued function K(t) such that dx(t) = (A(t) + B(t)K(t))x(t)dt is uniformly exponentially stable.

Stochastic Systems with Integral Action

45

Similarly, the pair (C(t), A(t)) is detectable if there must exist a bounded matrixvalued function L(t) so that dx(t) = (A(t) − L(t)C(t)) x(t)dt is also uniformly exponentially stable. Associated with 4-tuple (t0 , x0r , xt0 ; u) ∈ [t0 ,t f ] × Rn × L2Ft (Ω ; C ([t0 ,t f ]; Rm )) is an integral-quadratic form (IQF) performance measure J : [t0 ,t f ] × Rnr × Rnt × L2Ft (Ω ; C ([t0 ,t f ]; Rm )) → R+ such that    f  xr (t f ) 0 Q r 0 0 T T J(t0 , xr , xt ; u) = xr (t f ) xt (t f ) xt (t f ) 0 Qtf     tf  T  Qr (τ ) 0 xr (τ ) T T xr (τ ) xt (τ ) + u (τ )R(τ )u(τ ) d τ , (4) + 0 Qt (τ ) xt (τ ) t0 



Qrf 0 f 0 Qt



∈ Rn×n and n  nr + nt , where the terminal penalty weighting Q f    Qr 0 the performance weighting Q  ∈ C ([t0 ,t f ]; Rn×n ), and the control effort 0 Qt weighting R ∈ C ([t0 ,t f ]; Rm×m ) are deterministic, bounded, matrix-valued and positive semidefinite with R(t) invertible. The class of stochastic systems with integral action is represented as shown in Figure 1. Furthermore, as shown in [LiH76], under linear, state feedback control and a linear-quadratic system, all cumulants of the integral-quadratic form (IQF) cost have the quadratic affine functional form. This common form of the cumulants facilitates the definition of cumulant-based performance index and the corresponding formulation of the optimization problem. Therefore, it is reasonable to restrict the search for optimal control laws to linear timevarying feedback laws generated from the states of regulating xr (t) and tracking xt (t) together with an affine input l f (t) by

Dr w

Er r

+

∫ −

xt

Kt

u

+

Br

+

+

∫

+

−

xr

Cr

+

+

Ar Kr

Fig. 1. Diagram of stochastic system with state and integral output feedbacks.

y

46

Khanh D. Pham

   xr (t) + l f (t)  K(t)x(t) + l f (t), u(t) = Kr (t) Kt (t) xt (t) 

∀t ∈ [t0 ,t f ],

(5)

where the admissible vector-valued affine input l f and matrix-valued feedback gain K ∈ C ([t0 ,t f ]; Rm×n ) are yet to be determined. Hence, for the given initial condition (t0 , x0 ) ∈ [t0 ,t f ] × Rn and subject to the control policy (5), the dynamics of the stochastic system incorporating integral action are rewritten as follows: dx(t) = (A(t)x(t) + B(t)u(t) + S(t)r(t))dt + G(t)dw(t),

x(t0 ) = x0

(6)

together with its IQF performance measure tf  T  T J(t0 , x0 ; u) = x (t f )Q f x(t f ) + x (τ )Q(τ )x(τ ) + uT (τ )R(τ )u(τ ) d τ

(7)

t0

 Ar (t) 0 ; B(t)  provided that the augmented parameters are given by A(t)  −Cr (t) 0        0 Er (t) Br (t) 0 x ; G(t)  ; and the initial system condition x0  r0 . ; S(t)  I −Dr (t) 0 xt Clearly then, the performance measure (7) is now a random variable of chisquared type. Hence, the uncertainty and robustness of the performance distribution must be addressed via a complete set of higher-order statistics beyond statistical averaging. It is therefore necessary to generate some higher-order statistics associated with the performance measure (7). In general, it is suggested that the initial condition (t0 , x0 ) should be replaced by any arbitrary pair (α , xα ). Then, for the given, admissible affine input l f and feedback gain K, the “performance-to-come” of the performance measure (7) is introduced as follows: tf  T  J(α , xα )  xT (t f )Q f x(t f ) + x (τ )Q(τ )x(τ ) + uT (τ )R(τ )u(τ ) d τ (8) 

α

The moment-generating function of (8) is defined by

ϕ (α , xα ; θ )  E {exp (θ J (α , xα ))} ,

(9)

for all small parameters θ in an open interval about 0. Thus, the cumulant-generating function immediately follows:

ψ (α , xα ; θ )  ln {ϕ (α , xα ; θ )} ,

(10)

for all θ in some (possibly smaller) open interval about 0 while n{·} denotes the natural logarithmic transformation. Theorem 1 (Cumulant-Generating tive parameter, α ∈ [t0 ,t f ] is a time   T xαϒ (α ; θ )xα + 2xTα η (α ; θ ) , and assumptions of (A, B) and (C, A)

Function). Suppose that θ is a small posirunning variable, ϕ (α , xα ; θ )  ρ (α ; θ ) exp υ (α ; θ )  ln{ρ (α ; θ )}. Under the system uniformly stabilizable and detectable, the

Stochastic Systems with Integral Action

47

cumulant-generating function that compactly and robustly represents the uncertainty of the performance distribution is given by

ψ (α , xα ; θ ) = xTα ϒ (α ; θ )xα + 2xTα η (α ; θ ) + υ (α ; θ ) ,

(11)

where the cumulant-generating components ϒ (α ; θ ), η (α ; θ ), and υ (α ; θ ) solve the time-backward differential equations d ϒ (α ; θ ) = −[A(α ) + B(α )K(α )]T ϒ (α ; θ ) − ϒ (α ; θ )[A(α ) + B(α )K(α )] dα   − 2ϒ (α ; θ )G(α )W GT (α )ϒ (α ; θ ) − θ Q(α ) + K T (α )R(α )K(α ) , (12)   d η (α ; θ ) = −[A(α ) + B(α )K(α )]T η (α ; θ ) − ϒ (α ; θ ) B(α )l f (α ) + S(α )r(α ) dα (13) − θ K T (α )R(α )l f (α ) ,   d υ (α ; θ ) = −Tr ϒ (α ; θ )G (α )W GT (α ) dα   − 2η T (α ; θ ) B(α )l f (α ) + S(α )r(α ) − θ l Tf (α )R(α )l f (α )

(14)

! together! with the terminal-value conditions ϒ (t f ; θ ) = θ Q f ; η t f ; θ = 0; and υ t f ; θ = 0. Proof. For notational simplicity, additional shorthand notation is useful such as ϖ (α , xα ; θ )  exp {θ J (α , xα )} and ϕ (α , xα ; θ )  E {ϖ (α , xα ; θ )} together with the time derivative of  d ϕ (α , xα ; θ ) = −θ xTα [Q(α ) + K T (α )R(α )K(α )]xα dα

 + 2xTα K T (α )R(α )l f (α ) + l Tf (α )R(α )l f (α ) ϕ (α , xα ; θ ) . (15)

Using the standard Ito formula, it yields d ϕ (α , xα ; θ ) = E {d ϖ (α , xα ; θ )} , = ϕxα (α , xα ; θ )[A(α )+B(α )K(α )] xα d α + ϕα (α , xα ; θ ) d α   + ϕxα (α , xα ; θ ) B(α )l f (α ) + S(α )r(α ) d α  1  + Tr ϕxα xα (α , xα ; θ ) G (α )W GT (α ) d α , 2   which under definition ϕ (α , xα ; θ )  ρ (α ; θ ) exp xTα ϒ (α ; θ )xα + 2xTα η (α ; θ ) and its partial derivatives, leads to the following result:

48

Khanh D. Pham

d ϕ ( α , xα ; θ ) = dα



 d d + xTα ϒ (α ; θ )xα + 2xTα η (α ; θ ) ρ (α , θ ) dα dα

d d α ρ (α ; θ )

+ xTα [A(α ) + B(α )K(α )]T ϒ (α ; θ )xα + xTαϒ (α ; θ ) [A(α ) + B(α )K(α )] xα   + 2xTα [A(α ) + B(α )K(α )]T η (α ; θ ) + 2xTαϒ (α ; θ ) B(α )l f (α ) + S(α )r(α )     + 2η T (α ; θ ) B(α )l f (α ) + S(α )r(α ) + Tr ϒ (α ; θ )G(α )W GT (α ) + 2xTαϒ (α ; θ )G(α )W GT (α )ϒ (α ; θ )xα ϕ (α , xα ; θ ) . (16) Substituting (15) into (16) and having both linear and quadratic terms independent of xα , we obtain d ϒ (α ; θ ) = −[A(α ) + B(α )K(α )]T ϒ (α ; θ ) − ϒ (α ; θ )[A(α ) + B(α )K(α )] dα   − 2ϒ (α ; θ )G(α )W GT (α )ϒ (α ; θ ) − θ Q(α ) + K T (α )R(α )K(α ) ,   d η (α ; θ ) = −[A(α ) + B(α )K(α )]T η (α ; θ ) − ϒ (α ; θ ) B(α )l f (α ) + S(α )r(α ) dα − θ K T (α )R(α )l f (α ) ,   d υ (α ; θ ) = −Tr ϒ (α ; θ )G (α )W GT (α ) dα   − 2η T (α ; θ ) B(α )l f (α ) + S(α )r(α ) − θ l Tf (α )R(α )l f (α ) . At the final time α = t f , it follows that   ϕ (t f , x(t f ); θ ) = ρ (t f ; θ ) exp xT (t f )ϒ (t f ; θ )x(t f ) + 2xT (t f )η (t f ; θ )    = E exp θ [xT (t f )Q f x(t f )] , which in turn yields the terminal-value conditions as ϒ (t f ; θ ) = θ Q f ; η (t f ; θ ) = 0; ρ (t f ; θ ) = 1; and υ (t f ; θ ) = 0. 2 It is important to note that the expression for the cumulant-generating function (11) for the performance measure (7) indicates that the additional affine term takes into account the mismatched dynamics in the transient responses. By definition, higher-order statistics that encapsulate the uncertain nature of both regulating and tracking performances can now be generated via a Maclaurin series of the cumulantgenerating function (11) ∞

ψ (α , xα ; θ )  ∑ κi (α , xα ) i=1 ∞

=∑

i=1

θi i!

  ∂ (i)  ψ ( α , x ; θ )  α  ∂ θ (i)

(17)

θ =0

θi , i!

Stochastic Systems with Integral Action

49

in which κi (α , xα ) are called the cost cumulants, or equivalently, the performance measure statistics. Moreover, the series expansion coefficients are computed by using the cumulant-generating function (11)     (i) ∂ (i)   T ∂ ψ ( α , x ; θ ) = x ϒ ( α ; θ ) xα (18)   α α   ∂ θ (i) ∂ θ (i) θ =0 θ =0     (i) ∂ ∂ (i)   T η ( α ; θ ) + υ ( α ; θ ) . (19) + 2xα     ∂ θ (i) ∂ θ (i) θ =0

θ =0

In view of the results (17) and (19), the ith performance measure statistic for the problem with integral output feedback therefore follows   (i)  T ∂ κi (α , xα ) = xα ϒ ( α ; θ ) xα   ∂ θ (i) θ =0     (i) ∂ ∂ (i)   T + 2xα η ( α ; θ ) + υ ( α ; θ ) , (20)     ∂ θ (i) ∂ θ (i) θ =0

θ =0

for any finite 1 ≤ i < ∞. For notational convenience, the change of notation   ∂ (i)  ϒ ( α ; θ ) Hi (α )    ∂ θ (i) θ =0  (i) ∂  η ( α ; θ ) D˘ i (α )   (i)  ∂θ  θ =0  ∂ (i)  υ ( α ; θ ) Di (α )    ∂ θ (i)

(21)

(22)

(23)

θ =0

is introduced so that the next theorem yields a tractable and computational method of generating performance measure statistics in the time domain. This numerical procedure is preferred to that of (20) for the reason that the resulting cumulant-generating equations now allow the incorporation of classes of linear feedback controllers in statistical control problems. Theorem 2 (Performance Measure Statistics). Assume that a regulating and tracking stochastic system is described by (6)–(7) in which the pairs (A, B) and (C, A) are uniformly stabilizable and uniformly detectable. For k ∈ Z+ fixed, the kth cumulant of performance (7) is given by

κk (t0 , x0 ) = xT0 Hk (t0 )x0 + 2xT0 D˘ k (t0 ) + Dk (t0 ),

(24)

where the cumulant-generating solutions {Hi (α )}ki=1 , {D˘ i (α )}ki=1 , {Di (α )}ki=1 evaluated at α = t0 satisfy the differential equations (with the dependence of Hi (α ), D˘ i (α ) and Di (α ) upon l f and K suppressed)

50

Khanh D. Pham

d Hi (α ) = − [A(α ) + B(α )K(α )]T H1 (α ) − H1 (α ) [A(α ) + B(α )K(α )] dα − Q(α ) − K T (α )R(α )K(α ) d Hi (α ) = − [A(α ) + B(α )K(α )]T Hi (α ) − Hi (α ) [A(α ) + B(α )K(α )] dα i−1 2i! H j (α )G(α )W GT (α )Hi− j (α ) −∑ j=1 j!(i − j)!

(25)

(26)

  d ˘ D1 (α ) = − [A(α ) + B(α )K(α )]T D˘ 1 (α ) − H1 (α ) B(α )l f (α ) + S(α )r(α ) dα (27) − K T (α )R(α )l f (α ) d ˘ Di (α ) = − [A(α ) + B(α )K(α )]T D˘ i (α ) dα   − Hi (α ) B(α )l f (α ) + S(α )r(α ) (28)     d D1 (α ) = −Tr H1 (α )G(α )W GT (α ) − 2D˘ T1 (α ) B(α )l f (α ) + S(α )r(α ) dα (29) − l Tf (α )R(α )l f (α )   d Di (α ) = −Tr Hi (α )G(α )W GT (α ) dα   ˘ Ti (α ) B(α )l f (α ) + S(α )r(α ) , (30) − 2D with terminal-value conditions H1 (t f ) = Q f , Hi (t f ) = 0 for 2 ≤ i ≤ k; D˘ i (t f ) = 0 for 1 ≤ i ≤ k; and Di (t f ) = 0 for 1 ≤ i ≤ k. Proof. The expression of performance measure statistics described in (24) is readily justified by using the result (20) and the definitions (21)–(23). What remains is to show that the solutions Hi (α ), D˘ i (α ) and Di (α ) for 1 ≤ i ≤ k indeed satisfy the dynamical equations (25)–(30). Note that equations (25)–(30) are satisfied by the solutions Hi (α ), D˘ i (α ) and Di (α ) and can be obtained by successively taking time derivatives with respect to θ of the supporting equations (12)–(14)  together with the  assumption of A(α ) + B(α )K(α ) uniformly stabilizable on t0 ,t f . 2

3 Problem Statements The statistical control research discussed here has a performance index reflecting the intrinsic performance variability introduced by process noise stochasticity. Also, remember that all the cumulant values (24) depend in part of the known initial condition x(t0 ). Although different states x(t) will result in different values for the “performance-to-come” (8), the cumulant values are however functions of time-backward evolutions of the cumulant-generating variables Hi (α ), D˘ i (α ) and Di (α ) that totally ignore all the intermediate values x(t). This fact therefore makes the new optimization problem considered in statistical control particularly unique compared with the more traditional dynamic programming class of investigations.

Stochastic Systems with Integral Action

51

In other words, the time-backward equations (25)–(30) are now considered as the “new” dynamical equations for statistical control. With this evidence, the resulting Mayer optimization and its associated value function in the framework of dynamic programming [FlR75] therefore depend on the “new” state variables Hi (α ), D˘ i (α ) and Di (α ), not the classical states x(t), as people may often expect. As for statements of statistical control, it is convenient to introduce k-tuple ! ˘  D˘ 1 (·), . . . , D˘ k (·) , variables H , D˘ and D as H (·)  (H1 (·), . . . , Hk (·)), D(·) D(·)  (D1 (·), . . . , Dk (·)) for each element Hi ∈ C 1 ([t0 ,t f ]; Rn×n ) of H , D˘ i ∈ C 1 ([t0 ,t f ]; Rn ) of D˘ and Di ∈ C 1 ([t0 ,t f ]; R) of D having the representations Hi (·)  Hi (·), D˘ i (·)  D˘ i (·) and Di (·)  Di (·) with the right members satisfying the dynamic equations (25)–(30) on the horizon [t0 ,t f ]. The problem formulation can be considerably simplified if the following convenient mappings are introduced: Fi : [t0 ,t f ] × (Rn×n)k × Rm×n → Rn×n G˘i : [t0 ,t f ] × (Rn×n)k × (Rn )k × Rm×n × Rm → Rn Gi : [t0 ,t f ] × (Rn×n)k × (Rn )k × Rm → R, where the rules of action are given by F1 (α , H , K)  − [A(α ) + B(α )K(α )]T H1 (α ) − H1 (α ) [A(α ) + B(α )K(α )] − Q(α ) − K T (α )R(α )K(α ) Fi (α , H , K)  − [A(α ) + B(α )K(α )]T Hi (α ) − Hi (α ) [A(α ) + B(α )K(α )] i−1

−∑

j=1

2i! H j (α )G(α )W GT (α )Hi− j (α ) j!(i − j)!

! ˘ K, l f  − [A(α ) + B(α )K(α )]T D˘ 1 (α ) G˘1 α , H , D,   − H1 (α ) B(α )l f (α ) + S(α )r(α ) − K T (α )R(α )l f (α ) ! ˘ K, l f  − [A(α ) + B(α )K(α )]T D˘ i (α ) G˘i α , H , D,   − Hi (α ) B(α )l f (α ) + S(α )r(α ) !   G1 α , H , D˘ , l f  −Tr H1 (α )G(α )W GT (α )   − 2D˘ 1T (α ) B(α )l f (α ) + S(α )r(α ) − l Tf (α )R(α )l f (α ) !   Gi α , H , D˘ , l f  −Tr Hi (α )G(α )W GT (α )   − 2D˘ iT (α ) B(α )l f (α ) + S(α )r(α ) . The product mappings can be shown to be important in a compact formulation F1 × · · · × Fk : [t0 ,t f ] × (Rn×n)k × Rm×n → (Rn×n )k

52

Khanh D. Pham

G˘1 × · · · × G˘k : [t0 ,t f ] × (Rn×n)k × (Rn )k × Rm×n × Rm → (Rn )k G1 × · · · × Gk : [t0 ,t f ] × (Rn×n)k × (Rn )k × Rm → Rk along with the corresponding notation F  F1 × · · · × Fk , G˘  G˘1 × · · · × G˘k and G  G1 × · · · × Gk . Thus, the dynamic equations of motion (25)–(30) can be rewritten as d H (α ) = F (α , H (α ), K(α )), H (t f ) ≡ H f dα ! d ˘ ˘ f ) ≡ D˘ f D(α ) = G˘ α , H (α ), D˘ (α ), K(α ), l f (α ) , D(t dα ! d D(α ) = G α , H (α ), D˘ (α ), l f (α ) , D(t f ) ≡ D f , dα ! with k-tuple values H f  Q f , 0, . . . , 0 , D˘ f = (0, . . . , 0) and D f = (0, . . . , 0). Note that the product system uniquely determines H , D˘ and D once the admissible affine signal l f and feedback gain K are specified. Hence, they are considered ˘ K, l f ) and D ≡ D(·, K, l f ). The performance index in as H ≡ H (·, K), D˘ ≡ D(·, statistical control problems can now be formulated in l f and K. Definition 1 (Performance Index). Fix k ∈ Z+ and the sequence of scalar coefficients μ = {μi ≥ 0}ki=1 with μ1 > 0. Then for the given (t0 , x0 ), the performance index φ0 : {t0 } × (Rn×n)k × (Rn )k × Rk → R+ in statistical control for the stochastic regulator problem with an integral tracking accommodation over [t0 ,t f ] is defined as ! ˘ 0 , K, l f ), D(t0 , K, l f )  φ0 t0 , H (t0 , K), D(t k

∑ μi



 xT0 Hi (t0 , K)x0 + 2xT0 D˘ i (t0 , K, l f ) + Di (t0 , K, l f ) , (31)

i=1

where the real constants μi represent different degrees of freedom of shaping the performance distribution and the cumulant-generating components {Hi (t0 , K)}ki=1 , k  D˘ i (t0 , K, l f ) i=1 and {Di (t0 , K, l f )}ki=1 evaluated at α = t0 satisfy the dynamic equations d H (α ) = F (α , H (α ), K(α )), H (t f ) ≡ H f dα ! d ˘ ˘ f ) ≡ D˘ f D(α ) = G˘ α , H (α ), D˘ (α ), K(α ), l f (α ) , D(t dα ! d D(α ) = G α , H (α ), D˘ (α ), l f (α ) , D(t f ) ≡ D f . dα

Stochastic Systems with Integral Action

53

For the given terminal data (t f , H f , D˘ f , D f ), the classes of admissible affine input and feedback gain are then defined. Definition 2 (Admissible Affine Inputs and Feedback Gains). Let compact subsets L ⊂ Rm and K ⊂ Rm×n be the sets of allowable linear affine inputs and gain values. For the given k ∈ Z+ and the sequence μ = {μi ≥ 0}ki=1 with μ1 > 0, the set of admissible linear inputs Lt f ,H f ,D˘ f ,D f ;μ and feedback gains Kt f ,H f ,D˘ f ,D f ;μ are

respectively assumed to be the classes of C ([t0 ,t f ]; Rm ) and C ([t0 ,t f ]; Rm×n ) with values l f (·) ∈ L and K(·) ∈ K for which solutions to the dynamic equations with the ˘ f ) = D˘ f and D(t f ) = D f terminal-value conditions H (t f ) = H f , D(t d H (α ) = F (α , H (α ), K(α )) , dα ! d ˘ D(α ) = G˘ α , H (α ), D˘ (α ), K(α ), l f (α ) , dα ! d D(α ) = G α , H (α ), D˘ (α ), l f (α ) dα

(32) (33) (34)

exist on the interval of optimization [t0 ,t f ]. Next the optimization statements for the statistical control of the stochastic regulator problem incorporating integral action over a finite horizon are stated. Definition 3 (Optimization Problem). Suppose that k ∈ Z+ and the sequence μ = { μi ≥ 0}ki=1 with μ1 > 0 are fixed. Then the optimization problem of statistical control over [t0 ,t f ] is given by the minimization of (31) over l f (·) ∈ Lt f ,H f ,D˘ f ,D f ;μ , K(·) ∈ Kt f ,H f ,D˘ f ,D f ;μ subject to the dynamic equations of motion (32)–(34) for α ∈ [t0 ,t f ]. The subsequent results will then illustrate a construction of scalar-valued functions which are potential candidates for the value function.  Definition 4 (Reachable Set). Let reachable set Q be defined as Q  (ε , Y , Z˘ ,  Z ) ∈ [t0 ,t f ] × (Rn×n)k × (Rn )k × Rk such that Lε ,Y ,Z˘ ,Z ;μ = 0 and Kε ,Y ,Z˘ ,Z ;μ = 0. By adapting to the initial cost problem and the terminologies present in the statistical control, the Hamilton–Jacobi–Bellman (HJB) equation satisfied by the value function ! V ε , Y , Z˘ , Z is then given as follows. ! Theorem 3 (HJB Equation–Mayer Problem). Let ε , Y , Z˘ , Z be any interior ! point of the reachable set Q at which the value function V ε , Y , Z˘ , Z is differentiable. If there exist optimal affine signal l ∗f ∈ Lε ,Y ,Z˘ ,Z ;μ and feedback gain K ∗ ∈ Kε ,Y ,Z˘ ,Z ;μ , then the partial differential equation of dynamic programming  0=

min

l f ∈L, K∈K

! ! ∂ ∂ V ε , Y , Z˘ , Z vec(F (ε , Y , K)) V ε , Y , Z˘ , Z + ∂ε ∂ vec(Y )

54

Khanh D. Pham

+

!! ! ∂ ! V ε , Y , Z˘ , Z vec G˘ ε , Y , Z˘ , K, l f ˘ ∂ vec Z !! ! ∂ V ε , Y , Z˘ , Z vec G ε , Y , Z˘ , l f + ∂ vec(Z )

(35)

! ! is satisfied together with the boundary V t0 , H0 , D˘ 0 , D0 = φ0 t0 , H0 , D˘ 0 , D0 . ! Theorem 4 (Verification Theorem). Fix k ∈ Z+ and let W ε , Y , Z˘ , Z be a continuously differentiable solution of the HJB equation (35) which satisfies the boundary condition ! ! W t0 , H0 , D˘ 0 , D0 = φ0 t0 , H0 , D˘ 0 , D0 . (36) Let (t f , H f , D˘ f , D f ) be in Q; (l f , K) in Lt f ,H f ,D˘ f ,D f ;μ × Kt f ,H f ,D˘ f ,D f ;μ ; H , D˘ and D the solutions of (32)–(34). Then W (α , H (α ), D˘ (α ), D(α )) is time-backward increasing. If (l ∗f , K ∗ ) is in Lt f ,H f ,D˘ f ,D f ;μ × Kt f ,H f ,D˘ f ,D f ;μ defined on [t0 ,t f ] with corresponding solutions, H ∗ , D˘ ∗ , and D ∗ of (32)–(34) such that for α ∈ [t0 ,t f ] 0=

! ∂ W α , H ∗ (α ), D˘ ∗ (α ), D ∗ (α ) ∂ε ! ∂ W α , H ∗ (α ), D˘ ∗ (α ), D ∗ (α ) vec(F (α , H ∗ (α ), K ∗ (α ))) + ∂ vec(Y ) ! ∂ + W α , H ∗ (α ), D˘ ∗ (α ), D ∗ (α ) vec G˘(α , H ∗ (α ), ˘ ∂ vec(Z ) !! ! ∂ W α , H ∗ (α ), D˘ ∗ (α ), D ∗ (α ) × D˘ ∗ (α ), K ∗ (α ), l ∗f (α ) + ∂ vec(Z ) !! ×vec G α , H ∗ (α ), D˘ ∗ (α ), l ∗f (α ) (37)

then l ∗f and K ∗ are optimal. Moreover, ! ! W ε , Y , Z˘ , Z = V ε , Y , Z˘ , Z ,

(38)

! where V ε , Y , Z˘ , Z is the value function.

4 Multi-Cumulant Control Solution The HJB approach for obtaining a feedback solution for the statistical control problem over the finite horizon of optimization requires one to parameterize the ter! minal time and states of the dynamical equations as ε , Y , Z˘ , Z rather than ! t f , H f , D˘ f , D f . That is, for ε ∈ [t0 ,t f ] and 1 ≤ i ≤ k, the states of the system (32)– (34) defined on the interval [t0 , ε ] have the terminal values denoted by H (ε ) ≡ Y , ˘ ε ) ≡ Z˘ , and D(ε ) ≡ Z . The cues for estimating a candidate solution to the HJB D(

Stochastic Systems with Integral Action

55

equation (35) reside in the fact that the performance index (31) is quadratic affine in terms of the arbitrarily fixed x0 : ! W ε , Y , Z˘ , Z = xT0

k

∑ μi (Yi + Ei(ε )) x0

i=1

k ! k + 2xT0 ∑ μi Z˘i + T˘i (ε ) + ∑ μi (Zi + Ti (ε )) , (39) i=1

i=1

where the time-varying parametric functions Ei ∈ C 1 ([t0 ,t f ]; Rn×n ), T˘i ∈ C 1 ([t0 ,t f ]; Rn ) and Ti ∈ C 1 ([t0 ,t f ]; R) are yet to be determined. Using the isomorphic vec mapping, the next result clearly follows. ! Theorem 5 (Candidates for Value Function). Fix k ∈ Z+ and let ε , Y , Z˘ , Z be any interior point! of the reachable set Q at which the real-valued function W ε , Y , Z˘ , Z of the form (39) is differentiable. The derivative of ! W ε , Y , Z˘ , Z with respect to ε is given by   k ! d d W ε , Y , Z˘ , Z = xT0 ∑ μi Fi (ε , Y , K)+ Ei (ε ) x0 dε dε i=1   k ! d + 2xT0 ∑ μi G˘i ε , Y , Z˘ , K, l f + T˘i (ε ) dε i=1   k ! d + ∑ μi Gi ε , Y , Z˘ , l f + Ti (ε ) , dε i=1

(40)

provided l f ∈ L and K ∈ K. Trying the guess (39) in the HJB equation (35), it follows that  0≡

min

l f ∈L, K∈K

  d F μ ( ε , Y , K) + E ( ε ) x0 i i i ∑ dε i=1   k ! d + 2xT0 ∑ μi G˘i ε , Y , Z˘ , K, l f + T˘i (ε ) dε i=1   k ! d + ∑ μi Gi ε , Y , Z˘ , l f + Ti (ε ) . (41) dε i=1

xT0

k

Note that k

k

k

∑ μi Fi (ε , Y , K) = − [A(ε ) + B(ε )K]T ∑ μi Yi − ∑ μi Yi [A(ε ) + B(ε )K]

i=1

i=1

i=1

k

i−1

i=2

j=1

− μ1 Q(ε ) − μ1 K T R(ε )K − ∑ μi ∑

2! Y j G(ε )W GT (ε )Yi− j , j!(i − j)!

56

Khanh D. Pham k

∑ μi G˘i

! ε , Y , Z˘ , K, l f = − [A(ε ) + B(ε )K]T

i=1

k

∑ μi Z˘i

i=1

  − ∑ μi Yi B(ε )l f + S(ε )r(ε ) − μ1K T R(ε )l f , k

i=1

k

∑ μi G i

i=1

k !   ε , Y , Z˘ , l f = − ∑ μi Tr Yi G(ε )W GT (ε ) i=1

k   − 2 ∑ μi Z˘i T B(ε )l f + S(ε )r(ε ) − μ1 l Tf R(ε )l f . i=1

Differentiating the expression within the curly brackets of (41) with respect to K and l f yields the necessary conditions for an extremum of (31) on [t0 , ε ] 



k





k

−2B (ε ) ∑ μi Yi − 2 μ1 R(ε )K M0 + −2B (ε ) ∑ μi Z˘i − 2 μ1 R(ε )l f (x0 )T ≡ 0 T

T

i=1



i=1



k

−2B (ε ) ∑ μi Yi − 2 μ1R(ε )K x0 + T

i=1





k

−2B (ε ) ∑ μi Z˘i − 2 μ1 R(ε )l f T

≡ 0.

i=1

Since both x0 and M0  x0 xT0 are arbitrary vector and rank-one matrix, these necessary conditions for an extremum of (31) on [t0 , ε ] therefore imply that k

"r Z˘r l f (ε , Z˘ ) = −R−1 (ε )BT (ε ) ∑ μ

(42)

"r Yr , K(ε , Y ) = −R−1 (ε )BT (ε ) ∑ μ

(43)

r=1 k r=1

"r  μi /μ1 and μ1 > 0. Substituting (42) and (43) into (41) leads to the value where μ function  k k k d T x0 ∑ μi Ei (ε ) − AT (ε ) ∑ μi Yi − ∑ μi Yi A(ε ) − μ1 Q(ε ) i=1 d ε i=1 i=1 k

k

k

i=1

i=1

k

"r Yr B(ε )R−1 (ε )BT (ε ) ∑ μi Yi + ∑ μi Yi (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ "s Ys +∑μ r=1

s=1

k

k

r=1

s=1

"r Yr B(ε )R−1 (ε )BT (ε ) ∑ μ "s Ys − μ1 ∑ μ

Stochastic Systems with Integral Action k

i−1

i=2

j=1

− ∑ μi ∑  + 2xT0

k

d

57



2i! Y j G(ε )W GT (ε )Yi− j x0 j!(i − j)! k

k

k

i=1

r=1

∑ μi d ε T˘i (ε ) − AT (ε ) ∑ μi Z˘i + ∑ μr Yr B(ε )R−1 (ε )BT (ε ) ∑ μi Z˘i

i=1



k



k

i=1

"r Z˘r + S(ε )r(ε ) − ∑ μi Yi −B(ε )R−1 (ε )BT (ε ) ∑ μ i=1

r=1

k

"r Yr B(ε )R − μ1 ∑ μ

−1

r=1

k



"s Z˘s (ε )B (ε ) ∑ μ T

s=1

  d + ∑ μi Ti (ε ) − ∑ μi Tr Yi G(ε )W GT (ε ) i=1 d ε i=1   k k "r Z˘r + S(ε )r(ε ) − 2 ∑ μi Z˘i T −B(ε )R−1 (ε )BT (ε ) ∑ μ k

k

i=1

r=1

k

k

r=1

s=1

"r Z˘rT B(ε )R−1 (ε )BT (ε ) ∑ μ "s Z˘s . (44) − μ1 ∑ μ  k The remaining task is to display parametric functions {Ei (·)}ki=1 , T˘i (·) i=1 and {Ti (·)}ki=1 , which yield a sufficient condition to have the left-hand side of (44) be  k zero for any ε ∈ [t0 ,t f ], when {Yi }ki=1 and Z˘i i=1 are evaluated along solutions to the cumulant-generating equations. A careful observation of (44) suggests that  k {Ei (·)}ki=1 , T˘i (·) i=1 and {Ti (·)}ki=1 may be chosen to satisfy the Riccati-type differential equations k d "s Hs (ε ) E1 (ε ) = AT (ε )H1 (ε ) + H1 (ε )A(ε ) − H1 (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ dε s=1 k

"r Hr (ε )B(ε )R−1 (ε )BT (ε )H1 (ε ) + Q(ε ) − ∑ μ r=1

k

k

r=1

s=1

"r Hr (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ "s Hs (ε ) +∑μ

(45)

k d "s Hs (ε ) Ei (ε ) = AT (ε )Hi (ε ) + Hi (ε )A(ε ) − Hi (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ dε s=1 k

"r Hr (ε )B(ε )R−1 (ε )BT (ε )Hi (ε ) −∑μ r=1 i−1

+∑

j=1

2i! H j (ε )G(ε )W GT (ε )Hi− j (ε ) j!(i − j)!

(46)

58

Khanh D. Pham k d ˘ "r Hr (ε )B(ε )R−1 (ε )BT (ε )D˘ 1 (ε ) T1 (ε ) = AT (ε )D˘ 1 (ε ) − ∑ μ dε r=1   k "r D˘ r (ε ) + S(ε )r(ε ) + H1 (ε ) −B(ε )R−1 (ε )BT (ε ) ∑ μ r=1

k

k

r=1

s=1

"r Hr (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ "s D˘ s (ε ) +∑μ

(47)

k d ˘ "r Hr (ε )B(ε )R−1 (ε )BT (ε )D˘ i (ε ) Ti (ε ) = AT (ε )D˘ i (ε ) − ∑ μ dε r=1   k −1 T ˘ "r Dr (ε ) + S(ε )r(ε ) + Hi (ε ) −B(ε )R (ε )B (ε ) ∑ μ

(48)

r=1

  d T1 (ε ) = Tr H1 (ε )G(ε )W GT (ε ) dε 

 k T −1 T ˘ ˘ "r Dr (ε ) + S(ε )r(ε ) + 2D1 (ε ) −B(ε )R (ε )B (ε ) ∑ μ r=1

k

k

r=1

s=1

"r D˘ rT (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ "s D˘ s (ε ) +∑μ   d Ti (ε ) = Tr Hi (ε )G(ε )W GT (ε ) dε 

k

(49)



"r D˘ r (ε ) + S(ε )r(ε ) . + 2D˘ iT (ε ) −B(ε )R−1 (ε )BT (ε ) ∑ μ

(50)

r=1

The linear control input and feedback gain specified in (42) and (43) are now applied along the solution trajectories of the Riccati-type equations (32)–(34) k d "s Hs (ε ) H1 (ε ) = −AT (ε )H1 (ε ) − H1 (ε )A(ε ) + H1 (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ dε s=1 k

"r Hr (ε )B(ε )R−1 (ε )BT (ε )H1 (ε ) − Q(ε ) + ∑ μ r=1

k

k

r=1

s=1

"r Hr (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ "s Hs (ε ) −∑μ

(51)

Stochastic Systems with Integral Action

59

k d "s Hs (ε ) Hi (ε ) = −AT (ε )Hi (ε ) − Hi (ε )A(ε ) + Hi (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ dε s=1 k

"r Hr (ε )B(ε )R−1 (ε )BT (ε )Hi (ε ) +∑μ r=1 i−1

−∑

j=1

2i! H j (ε )G(ε )W GT (ε )Hi− j (ε ) j!(i − j)!

(52)

k d ˘ "r Hr (ε )B(ε )R−1 (ε )BT (ε )D˘ 1 (ε ) D1 (ε ) = −AT (ε )D˘ 1 (ε ) + ∑ μ dε r=1   k −1 T "r D˘ r (ε ) + S(ε )r(ε ) − H1 (ε ) −B(ε )R (ε )B (ε ) ∑ μ r=1

k

k

r=1

s=1

"r Hr (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ "s D˘ s (ε ) −∑μ

(53)

k d ˘ "r Hr (ε )B(ε )R−1 (ε )BT (ε )D˘ i (ε ) Di (ε ) = −AT (ε )D˘ i (ε ) + ∑ μ dε r=1   k −1 T ˘ "r Dr (ε ) + S(ε )r(ε ) − Hi (ε ) −B(ε )R (ε )B (ε ) ∑ μ

(54)

r=1

  d D1 (ε ) = −Tr H1 (ε )G(ε )W GT (ε ) dε 

 k T −1 T ˘ ˘ "r Dr (ε ) + S(ε )r(ε ) − 2D1 (ε ) −B(ε )R (ε )B (ε ) ∑ μ r=1

k

k

r=1

s=1

"r D˘ rT (ε )B(ε )R−1 (ε )BT (ε ) ∑ μ "s D˘ s (ε ) −∑μ   d Di (ε ) = −Tr Hi (ε )G(ε )W GT (ε ) dε 

k

(55)



"r D˘ r (ε ) + S(ε )r(ε ) , − 2D˘ iT (ε ) −B(ε )R−1 (ε )BT (ε ) ∑ μ

(56)

r=1

with the terminal-value conditions H1 (t f ) = Q f , Hi (t f ) = 0 for 2 ≤ i ≤ k; D˘ i (t f ) = 0 for 1 ≤ i ≤ k; and Di (t f ) = 0 for 1 ≤ i ≤ k. The boundary condition of W (ε , Y , Z˘ , Z ) implies that

60

Khanh D. Pham

xT0

k

k

i=1

i=1

∑ μi (Hi0 + Ei(t0 )) x0 + 2xT0 ∑ μi

! k D˘ i0 + T˘i (t0 ) + ∑ μi (Di0 + Ti (t0 ))

= xT0

i=1

k

k

k

i=1

i=1

i=1

∑ μi Hi0 x0 + 2xT0 ∑ μi D˘ i0 + ∑ μi Di0 .

The initial conditions for the equations (45)–(50) follow Ei (t0 ) = 0, T˘i (t0 ) = 0 and Ti (t0 ) = 0. Therefore, the linear affine input (42) and feedback gain (43) minimizing the performance index (31) become optimal: k

"r D˘ r∗ (ε ) , l ∗f (ε ) = −R−1 (ε )BT (ε ) ∑ μ r=1 k

"r Hr∗ (ε ) . K ∗ (ε ) = −R−1 (ε )BT (ε ) ∑ μ r=1

Finally, the main results are summarized as follows: 1) the feedback controller (58) provides stability augmentation and 2) the feedforward controller (59) furnishes the tracking capability. Theorem 6 (Statistical Control with Integral Feedback Action). Suppose that the pair (A, B) is uniformly stabilizable and the pair (C, A) is uniformly detectable. When k ∈ Z+ and the sequence μ = {μi ≥ 0}ki=1 with μ1 > 0 fixed, the optimal statistical controller for the stochastic system with integral feedback action is implemented as follows: u∗ (t) = K ∗ (t)x∗ (t) + l ∗f (t)

(57)

k

"r Hr∗ (α ) K ∗ (α ) = −R−1 (α )BT (α ) ∑ μ

(58)

"r D˘ r∗ (α ), l ∗f (α ) = −R−1 (α )BT (α ) ∑ μ

(59)

r=1 k r=1

"r  μi /μ1 emphasize different degrees of shaping the probability denwhere μ sity function of the performance measure (7). Optimal solutions {Hr∗ (α )}kr=1 and k  ∗ D˘ r (α ) r=1 representing the cumulant-generating components satisfy the timebackward matrix-valued differential equations d H ∗ (α ) = − [A(α ) + B(α )K ∗ (α )]T H1∗ (α ) − H1∗ (α ) [A(α ) + B(α )K ∗ (α )] dα 1 (60) − Q(α ) − K ∗T (α )R(α )K ∗ (α ) , d H ∗ (α ) = − [A(α ) + B(α )K ∗ (α )]T Hr∗ (α ) − Hr∗ (α ) [A(α ) + B(α )K ∗ (α )] dα r r−1 2r! ∗ Hs∗ (α )G(α )W GT (α )Hr−s −∑ (α ) (61) s!(r − s)! s=1

Stochastic Systems with Integral Action

61

and the time-backward vector-valued differential equations d ˘∗ D (α ) = − [A(α ) + B(α )K ∗ (α )]T D˘ 1∗ (α ) − K ∗T (α )R(α )l ∗f (α ) dα 1   − H1 (α ) B(α )l ∗f (α ) + S(α )r(α ) , d ˘∗ D (α ) = − [A(α ) + B(α )K ∗ (α )]T D˘ r∗ (α ) dα r   − Hr (α ) B(α )l ∗f (α ) + S(α )r(α )

(62) (63)

(64)

with the terminal-value conditions H1∗ (t f ) = Q f , Hr∗ (t f ) = 0 for 2 ≤ r ≤ k and D˘ r∗ (t f ) = 0 for 1 ≤ r ≤ k. It is worth observing that the optimal feedback gain (58) and affine control input (59) operate dynamically on the time-backward histories of the linear combination of performance measure statistics satisfying the equations of dynamics (60)–(61) and (63)–(64) from the final to the current time as illustrated in Figure 2. Moreover, it is obvious that these performance measure statistics effectively depict multiple ⎡ x (t )⎤ u * (t ) = K * (t ) x (t ) + l *f (t ) = ⎡⎣ K r* (t ) K t* (t )⎤⎦ ⎢ r ⎥ + l *f (t ) ⎣ xt (t )⎦ k

K * (α ) = − R −1 (α ) BT (α )∑ μˆ r H r* (α ) = ⎡⎣ K r* (α ) K t* (α )⎤⎦ r =1

l (α ) = − R * f

−1

k



(α ) B (α )∑ μˆ r Dr* (α ) T

r =1

 η = α , H (α ), D (α )

(

)

T d H1* (α ) = − ⎡⎣ A (α ) + B (α ) K * (α )⎤⎦ H1* (α ) − H1* (α ) ⎡⎣ A (α ) + B (α ) K * (α )⎤⎦ dα

− (K * (α )) R (α ) K * (α ) − Q (α ); T

H1* (t f )= Q f

T d H r* (α ) = − ⎡⎣ A (s ) + B (α ) K * (α )⎤⎦ H r* (α ) − H r* (α ) ⎡⎣ A (α ) + B (α ) K * (α )⎤⎦ dα r −1 2r ! H * (α )G (α )WG T (α ) H r*− s (α ); H r* (t f )= 0; 2≤r ≤k −∑ s ! (r − s )! s s =1 T T  d * D1 (α ) = − ⎡⎣ A (α ) + B (α ) K * (α )⎤⎦ D1* (α ) − (K * (α )) R (α )l *f (α ) dα  D1* (t f )= 0 − H1* (α ) ⎡⎣ B (α )l *f (α ) + S (α )r (α )⎤⎦ ; T  d * Dr (α ) = − ⎡⎣ A (α ) + B (α ) K * (α )⎤⎦ Dr* (α ) dα  2≤r ≤k − H r* (α ) ⎡⎣ B (α )l *f (α ) + S (α )r (α )⎤⎦ ; Dr* (t f )= 0;

Fig. 2. Optimal control solution for stochastic system with state- and integral outputfeedbacks.

62

Khanh D. Pham

resolutions of the performance uncertainty. Such useful statistics also depend upon the noise process characteristics. Therefore, the statistical control paradigm consisting of optimal feedback gain (58) and affine input (59) has intentionally traded the property of the certainty equivalence principle that would be obtained from the special case of LQG control, for the adaptability of dealing with highly dynamic uncertainty and multi-performance objectives.

Acknowledgments This material is based upon work supported in part by the U.S. Air Force Research Laboratory-Space Vehicles Directorate and the U.S. Air Force Office of Scientific Research. Much appreciation from the author goes to Dr. Donna C. Senft, the branch chief of Spacecraft Components Technology, for serving as the reader of this work and providing helpful criticism.

References [FlR75]

W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control. New York: Springer-Verlag, 1975. [FMT80] S. Fukata, A. Mohri, and M. Takata, On the Determination of the Optimal Feedback Gains for Multivariable Linear Systems Incorporating Integral Action, International Journal of Control, Vol. 31, pp. 1027–1040, 1980. [Joh68] C. D. Johnson, Optimal Control of the Linear Regulator with Constant Disturbances, IEEE Transactions on Automatic Control, Vol. 13, pp. 416–421, 1968. [LiH76] S. R. Liberty and R. C. Hartwig, On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants, Information and Control, Vol. 32, No. 3, pp. 276–305, 1976. [PSL02a] K. D. Pham, M. K. Sain, and S. R. Liberty, Robust Cost-Cumulants Based Algorithm for Second and Third Generation Structural Control Benchmarks, Proceedings of the American Control Conference, pp. 3070–3075, Anchorage, Alaska, May 08–10, 2002. [PSL02b] K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: StateFeedback, Finite-Horizon Paradigm with Application to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002. [PSL04] K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants, Proceedings of the American Control Conference, pp. 691–696, Boston, Massachusetts, June 30, 2004. [PSL05] K. D. Pham, M. K. Sain, and S. R. Liberty, Statistical Control for Smart BaseIsolated Buildings via Cost Cumulants and Output Feedback Paradigm, Proceedings of the American Control Conference, pp. 3090–3095, Portland, Oregon, June 8–10, 2005.

Stochastic Systems with Integral Action [Pha05]

[PhR07]

63

K. D. Pham, Minimax Design of Statistics-Based Control with Noise Uncertainty for Highway Bridges, Proceedings of DETC 2005/2005 ASME 20th Biennial Conference on Mechanical Vibration and Noise: Active Control of Vibration and Acoustics I, DETC2005-84593, Long Beach, California, September 24–28, 2005. K. D. Pham and L. Robertson, Statistical Control Paradigm for Aerospace Structures Under Impulsive Disturbances, The 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA 2007-1755, Honolulu, Hawaii, April 23–26, 2007.

Multi-Cumulant and Pareto Solutions for Tactics Change Prediction and Performance Analysis in Stochastic Multi-Team Noncooperative Games Khanh D. Pham,1 Stanley R. Liberty,2 and Gang Jin3 1 2 3

Space Vehicles Directorate, Air Force Research Laboratory, Kirtland AFB, NM 87117, USA Office of President, Kettering University, Flint, MI 48504, USA. [email protected] Electronics & Electrical Engineering, Ford Motor Company, Dearborn, MI 48124, USA. [email protected]

Summary. New solution concepts, called the multi-cumulant Pareto Nash and minimax strategies, are proposed for quadratic decision problems where multiple teams of decision makers are interested in strategies that not only ensure cooperation within each team and competition among different teams but also provide noncooperative teams the capability of assessing team performance and predicting tactics via complete statistical descriptions. Analytical expressions for higher-order statistics associated with strategy selection and performance assessment as well as closed-form feedback Nash equilibrium and minimax solutions to the special linear-quadratic class of stochastic multi-team games are also presented. The capability to shape the probability density function of the team performance measure is possible because not only is the first performance statistic considered as in the special case of the linear-quadratic-gaussian problem, but also some finite linear combinations of other performance statistics are included. In all decision strategies developed here, the decision feedback gains are explicitly dependent upon the “information” statistics which are then used to directly target the uncertainty of team performance and decision laws. It is concluded that the need to account for the reduction of performance uncertainty gives rise to the interaction between dual decision control functions: reducing uncertainty and exercising control. As the result, the certainty equivalence property is no longer available for the class of statistical control problems considered here.

1 Introduction While dealing with the problem of analysis and control of large-scale distributed systems and making robust decisions under different degrees of adversary uncertainty, several solution concepts have been proposed in the framework of team and game theories. Many applications, including networks security, threat prediction, and terrorism attack prevention can be modeled as stochastic, noncooperative, and multiteam games. In such a game, more than two teams of decision makers must make C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 4, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

66

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

optimal collective strategies to reach payoff equilibrium in the presence of others attempting to improve their outcomes. Within each team, the members think jointly for the collective benefits, but outside the team the members compete with other members of competing teams. In the literature, most studies on deterministic noncooperative games have concentrated on the strategy selection of a Nash equilibrium. This equilibrium ensures that no team has incentives to unilaterally deviate in order to improve its overall team performance. Very little work, if any, has been published on the subject of the intrinsic performance variability introduced by process noise stochasticity that involves (1) determination of how Nash-based strategies behave and affect performance distribution of each team and (2) environmental disturbances which ultimately decide the sample path performance by drawing a realization of the underlying process. Until recently, feedback solutions in the context of cost statistics beyond the cost mean have just surfaced and gained much attention in the stochastic control and differential game communities. Interested readers should refer to ongoing developments for optimal stochastic regulator and/or servo problems [PLSS99]– [Pha08b], [DSPW07] as well as for stochastic cooperative and noncooperative games [DiS05], [DiS06], [Pha07b]– [Pha08b] using any finite number of cost statistics for cooperative and noncooperative games. The present work provides an innovative and novel approach for dynamically predicting adversary tactics of possible rival teams utilizing the tenets of statistical control and game theories. That is to say, the complete information of each possible course of actions and team performance will entirely be characterized by a probability density function of chi-squared performance measure. There are two levels of coordination and control for multi-team noncooperative games: lower (cooperation within each team) level and higher (noncooperation between teams) level. This hierarchical structure is important to model surprise attacks as well as to predict tactic pattern changes. The lower-level Pareto coordination is used by each team to determine the cooperative actions for each team member based on available information, including team objectives, past data, and possible choices at the current game state. As an effective and ideal mechanism, a Nash game-theoretic framework is utilized at a higher level among competing teams to capture the nature of team conflicts so that the determination of one team’s strategies is tightly coupled to those of the other teams. Via statistical control and game theories, it is therefore anticipated that future behavioral changes will lose the element of surprise due to the properties of selfenforcing Nash solutions that are capable of reshaping probability distributions of collective courses of action and team performance. When the simultaneous order of decision making is not possible and other teams seek to do maximal harm to one or more remaining teams, the subject teams then play defensively and adopt team minimax strategies. In addition to the contribution to the theory of behavioral change prediction in hostile environments, these results are also expected to enable automated responses during active conflicts and post-conflict mission assessments as well. This research article is organized as follows. First, a general discussion on the formulation of multi-team decision problems in stochastic systems involving uncertainty, informational decentralization, and possible conflicts of interest is provided. Second, the statistical control statements and solution concepts for stochastic multi-

Tactics Change Prediction and Performance Analysis

67

team Nash games are treated in detail. Third, the results are also extended to pessimistic situations such as multi-team minimax games. Finally, some conclusions are included.

2 Problem Formulation More recently, people have begun applying game theory to deal with prediction and prevention under intelligent tactical situations [Cru01] and [LSC03]. Uncertainty analysis in prediction and/or prevention of adversary tactics, which assesses the impact of the uncertainty of tactical characteristics and patterns, is now widely recognized as an important and indispensable component in robust threat prediction and decision making. A statistical theoretic approach is effective because it takes into account the fact that adversaries are intelligent and irregular. The inherent capability of completely characterizing each possible course of action and surprise attack in probability density space as offered by statistical control theory makes it suitable for situations in which the uncertainty of patterns and strategies is unavoidable. As an illustration, it is of interest to consider a multi-team nonzero-sum differential game with multiple teams denoted by the set χ  {1, 2, . . ., N} where each team χ has mχ members identified by the set mχ  m1 , . . . , mχ . The length of horizon on which the dynamics of the decision process evolves is specified by the  interval t0 ,t f . The state-space description of the interaction of the decision teams together with environmental uncertainty is governed by a stochastic differential equa  tion modeled on t0 ,t f ,

m N

dx(t) =

A(t)x(t) +

χ

χ

χ

∑ ∑ Bi (t)ui (t)

dt + G(t)dw(t) ,

χ =1 i=1

x(t0 ) = x0 ,

(1)

uncertainty affecting the decision process (1) w(t)  w(t, ω ) : where the additive t0 ,t f × Ω → R p is a p-dimensional stationary Wiener process defined with {Ft }t≥0 beingits filtration on a complete filtered probability space (Ω , F , {Ft }t≥0 , P) over t0 ,t f with the correlation of independent increments   E [w(τ ) − w(ξ )][w(τ ) − w(ξ )]T = W |τ − ξ |. It is presumed throughout the development that Ω is a complete separable metric space, F is the sigma field of Borel sets of Ω , and P is now a known probability measure on F . When an elementary event ω ∈ Ω is the outcome of an experiment, states  ω is the sample path of the decision process. Decision   !x(t)  x(t, ω ) : t0 ,t f × Ω → Rn belong to the Hilbert space L2Ft Ω ; C ( t0 ,t f ; Rn ) of Rn -valued,   square integrable processes on t0 ,t f that are adapted to the sigma field Ft gener

 t f T ated by w(t) with E x (τ )x(τ )d τ < ∞ and where the initial state x0 is known t0  !  to all teams; and the continuous-time decision coefficients A ∈ C t0 ,t f ; Rn×n ,

68

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

     ! χ χ Bi ∈ C t0 ,t f ; Rn×mi , and G ∈ C t0 ,t f ; Rn×p are deterministic, bounded, matrix-valued permissible decision values for member i of team functions. A set of  χ χ χ by U χ at t ∈ t0 ,t f isdenoted , with ui ∈ Ui being a typical element. In this seti    χ χ χ belongs to the Hilbert space of Rmi -valued, ting, Ui ∈ L2Ft Ω ; C t0 ,t f ; Rmi   square integrable processes on t0 ,t f that are also adapted to the sigma field Ft generated by w(t). Furthermore, each member i of team χ has available to him current values of χ the decision states via his information structure ηi  {t, x(t)} which then generχ ates the corresponding information space Zi . Admissible decision laws or strateχ gies for each member i of team χ are defined by appropriate mappings γi from χ χ χ χ χ his information space Zi into his decision space Ui where γi : Zi → Ui are all Borel measurable functions. The class of admissible strategies associated with χ member i of team χ is then denoted by Γi . For each fixed admissible strategy of the remaining decision makers, there supposedly exists a real-valued, Borel mea! surable function on S ; Ω × U 1 × · · · × U N where S is the sigma field gener! 1 × · · · × U N by the sets belonging to F and Ω × U ated on the product space ! ! the Borel sets B U 1 , . . . , B U N of U 1 , . . . , U N , for each i ∈ mχ , of which χ χ member i of team χ strives to optimize his choice of possible decisions ui ∈ Ui via a standard finite-horizon integral-quadratic form (IQF) performance measure  χ Ji : t0 ,t f × Rn × U 1 × · · · × U N → R+ , ! χ χ Ji t0 , x0 ; u1 , . . . , uN = xT (t f )Qi f x(t f )  tf χ xT (τ )Qi (τ )x(τ ) + + t0

N

∑

  T χϑ ∑ uϑj (τ )Ri j (τ )uϑj (τ ) d τ , (2)

mϑ

ϑ =1 j=1

 χ χ  where the mχ -tuple of permissible decision uχ  u1 , . . . , umχ is a typical elemχ χ χ χ ment of U χ with U χ  ×i=1 Ui ; the state weightings Qi f ∈ Rn×n and Qi ∈   n×n ! C t0 ,t f ; R are deterministic, bounded, matrix-valued functions with properties of symmetry positive semi-definiteness; and the control input weight and χ χ  χϑ ings Ri j ∈ C t0 ,t f ; Rm j ×m j are deterministic, bounded, matrix-valued functions with properties of symmetry and positive definiteness. It is important to note that the individual objective functional (2) cannot be optimized independently of the decisions of other members and remaining rival teams. Hence, the order of decision making between cooperative team members and among adversarial teams χ ∈ χ is assumed to be simultaneous, which therefore completes the formulation of the subject multi-team decision problem. As can be seen from cooperative team χ , members i for i ∈ mχ do not all have the same objective functionals but they still decide to act cooperatively. An equilibrium χ concept for a negotiating solution is proposed via a Pareto subset UP of the class of χ permissible decisions U . This solution is particularly attractive with the property

Tactics Change Prediction and Performance Analysis

69

that if any other solution is used at least one of the team members is penalized in the sense that his performance is worse, or all the members do the same. χ

χ

Definition 1 (Team Pareto Decisions). The mχ -tuple uP ∈ UP is a nondominated decision for team χ if, for any other decision mχ -tuple uχ ∈ U χ ,    χ χ −1 χ +1 Ji t0 , x0 ; u1P , . . . , uP , u χ , uP , . . . , uNP   χ χ −1 χ χ +1 ≤ Ji t0 , x0 ; u1P , . . . , uP , uP , uP , . . . , uNP only if    χ χ −1 χ +1 Ji t0 , x0 ; u1P , . . . , uP , u χ , uP , . . . , uNP   χ χ −1 χ χ +1 = Ji t0 , x0 ; u1P , . . . , uP , uP , uP , . . . , uNP , for all i ∈ mχ and χ ∈ χ . Since the IQF performance measures (2) are convex functions on a convex set U 1 × · · · × U N , the problem of solving for a subset of permissible Pareto decisions within each team χ with a vector performance-measure criterion is equivalent to the problem of solving an mχ − 1 parameter family of optimal control problems with scalar performance-measure criteria [Kli64] and [Zad63]. This subset of mχ -tuples of permissible Pareto decisions can be obtained by considering a convex combination ! mχ χ χ ! J χ t0 , x0 ; u1 , . . . , uN ; ξ χ  ∑ ξi Ji t0 , x0 ; u1 , . . . , uN

(3)

i=1

! and minimizing J χ t0 , x0 ; u1 , . . . , u χ −1, u χ , u χ +1, . . . , uN ; ξ χ over U χ and for each fixed admissible decision of the remaining decision makers. Thus, the subset of permissible Pareto decisions for team χ is generated by a Pareto parameterization ξ χ , each of which then belongs to a set of team cooperative profiles ξ χ ∈ W χ  m Wχ 

ξ χ ∈ Rm χ :

χ

χ

∑ ξi

χ

= 1; 0 ≤ ξi ≤ 1

(4)

.

i=1

χ

χ

χ

χ

χ

χϑ

χ χϑ

χ χ χ ξi Qi f , Qχ  ∑i=1 ξi Qi , and R j  ∑i=1 ξi Ri j , the negotiBy letting Q f  ∑i=1 ating performance measure (2) can be rewritten as follows:

m

m

m

! χ J χ t0 , x0 ; u1 , . . . , uN ; ξ χ = xT (t f )Q f x(t f )  tf  N mϑ  T χϑ T χ ϑ ϑ x (τ )Q (τ )x(τ ) + ∑ ∑ u j (τ )R j (τ )u j (τ ) d τ . (5) + t0

ϑ =1 j=1

70

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

    χϑ χϑ Furthermore, if Rχϑ  diag R1 , . . . , Rmχ and Bϑ  Bϑ1 , . . . , Bϑmϑ , the negotiating performance measure (5) associated with team χ becomes ! χ J χ t0 , x0 ; u1 , . . . , uN ; ξ χ = xT (t f )Q f x(t f )  tf xT (τ )Qχ (τ )x(τ ) + + t0

N

∑

ϑ =1

 u

ϑ

T

 (τ )R

χϑ

ϑ

(τ )u (τ ) d τ

(6)

x(t0 ) = x0 .

(7)

subject to the corresponding decision dynamics

N

dx(t) =

A(t)x(t) +

∑ Bχ (t)uχ (t)

χ =1

dt + G(t)dw(t) ,

The linear time-varying decision process (7) in the absence of noisy disturbances is called uniformly exponentially stable if there exist positive constants η1 and η2 such that the pointwise matrix norm of the closed-loop state transition matrix satisfies the inequality ||Φ (t, τ )|| ≤ η1 e−η2 (t−τ ) ∀ t ≥ τ ≥ t0 . The pairs (A(t), Bχ (t)) are  stabilizable if there exist bounded matrix-valued functions

K χ (t) such that dx(t) = A(t) + ∑Nχ =1 Bχ (t)K χ (t) x(t)dt is uniformly exponentially stable.

3 Problem Statements Before the game, each team χ adopts a specific cooperative profile ξ χ ∈ W χ by a mutual agreement among the team members and also considers a Pareto strategy mχ χ χ subset ΓP ⊂ Γ χ with Γ χ  ×i=1 Γi . The Pareto set of permissible sub-strategies is  χ χ then defined as such ΓP  γP . Note that for the given initial condition (t0 , x0 ) ∈  !  t0 ,t f × Rn , the decision states x(t) are uniquely determined from γP1 , . . . , γPN ∈ χ ×Nχ =1ΓP . Thus, the negotiating performance measure (6) for team χ can now be ! considered to be dependent on γP1 , . . . , γPN via the Borel measurable mapping J χ : ΓP1 × · · · × ΓPN → R+ . In the case of no observation noise, the decision states at any instant of time are what teams χ need for maintaining a fair degree of accuracy on the future status of the decision process with different permissible decision values applied. Thus, permissible Pareto decisions from teams χ at time t must be functions of decision states at time t and of the time t, with the definition given by   χ   χ χ χ χ χ UP  uP : ∃ γP ∈ ΓP  uP = γP (t, x(t)) , χ∈χ ; t ∈ t0 ,t f . Moreover, as shown in [LiH76], under linear, state-feedback control together with the linear-quadratic structure of the system considered therein, all cumulants of the IQF performance-measure have the same quadratic affine functional form. This common

Tactics Change Prediction and Performance Analysis

71

form of the cumulants facilitates the definition of a cumulant-based performance index and the formulation of an optimization problem involving a finite number of the cumulants of J χ . Therefore, it is reasonable to restrict the search for optimal decision laws to linear time-varying feedback laws generated from the decision states x(t) by   χ uP (t)  K χ (t)x(t) , (8) t ∈ t0 ,t f   mχ χ  where K χ ∈ C t0 ,t f ; R∑i=1 mi ×n for each team χ is any deterministic, bounded, matrix-valued function which is in the class of admissible feedback gains defined in the sequel. The associated performance measure (6) becomes ! χ J χ K 1 , . . . , K N = xT (t f )Q f x(t f )  tf + xT (τ ) Qχ (τ ) + t0

N

∑

ϑ =1



Kϑ

T

 (τ )Rχϑ (τ )K ϑ (τ ) x(τ )d τ . (9)

Due to the linear-quadratic property of (7) and (9), the performance measure (9) associated with team χ is indeed a random variable of chi-squared type. Therefore, it is necessary to develop a procedure for generating performance statistics of the multiteam nonzero-sum differential game by adapting the parametric method in [Jac73] to characterize both moment- and cumulant-generating functions. Such performance statistics are then used to form different performance indices associated with teams χ for χ ∈ χ in the statistical control optimization. This approach begins with a replacement of the initial condition (t0 , x0 ) by any arbitrary pair (α , xα ). Thus, for the given admissible feedback gains K1 , . . . , KN , the performance measure (9) is seen as the “performance-to-come” χ

J χ (α , xα )  xT (t f )Q f x(t f )  tf + xT (τ ) Qχ (τ ) + α

N

∑

ϑ =1

  T K ϑ (τ )Rχϑ (τ )K ϑ (τ ) x(τ )d τ . (10)

Associated with each team χ , the first characteristic function or the momentgenerating function of the “performance-to-come” (10) is defined by

ϕ χ (α , xα ; θ χ )  E {exp (θ χ J χ (α , xα ))}

(11)

for all θ χ in an open interval about 0. What follows next is the second characteristic function or the cumulant-generating function

ψ χ (α , xα ; θ χ )  ln {ϕ χ (α , xα ; θ χ )}

(12)

for all θ χ in some (possibly smaller) open interval about 0 while ln{·} denotes the natural logarithmic transformation of the first characteristic function. Note that recent work is mainly concerned with the cost on large values of the decision states and decision laws; the choices of θ χ are thus restricted to some open interval (0, δ ) for small positive constants δ .

72

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

Theorem 1 (Performance Measure Cumulant-Generating Function). Suppose θ χ ∈ (0, δ ) and the first characteristic function takes the form of ! (13) ϕ χ (α , xα ; θ χ )  ρ (α ; θ χ ) exp xTα ϒ χ (α ; θ χ )xα   χ χ χ χ (14) υ (α ; θ )  ln{ρ (α ; θ )} , ∀ α ∈ t0 ,t f . Then, the cumulant-generating function associated with team χ is given by

ψ χ (α , xα ; θ χ ) = xTα ϒ χ (α ; θ χ )xα + υ χ (α ; θ χ ) ,

(15)

where the scalar solution υ χ (α ; θ χ ) solves !the time-backward differential equation with the terminal-value condition υ χ t f ; θ χ = 0   d χ υ (α ; θ χ ) = −Tr ϒ χ (α ; θ χ )G (α )W GT (α ) , dα

(16)

and the matrix-valued solution ϒ χ (α ; θ χ ) satisfies the time-backward differential χ equation together with its terminal-value condition ϒ χ (t f ; θ χ ) = θ χ Q f  T N d χ ϒ (α ; θ χ ) = − A(α ) + ∑ Bϑ (α )K ϑ (α ) ϒ χ (α ; θ χ ) dα ϑ =1   − ϒ χ (α ; θ χ ) A(α ) + χ

χ

N

∑ Bϑ (α )K ϑ (α )

ϑ =1 T

− 2ϒ (α ; θ )G(α )W G (α )ϒ χ (α ; θ χ )   T N  χ χ ϑ χϑ ϑ − θ Q (α ) + ∑ K (α )R (α )K (α ) . (17) ϑ =1

In addition, the auxiliary solution ρ χ (α ; θ χ ) is satisfying the ! time-backward differential equation with the terminal-value condition ρ χ t f ; θ χ = 1   d χ ρ (α ; θ χ ) = −ρ χ (α ; θ χ ) Tr ϒ χ (α ; θ χ )G (α )W GT (α ) . dα

(18)

Proof. For the given θ χ and χ ∈ χ , a shorthand notation associated with the first characteristic function is denoted by ϖ χ (α , xα ; θ χ )  exp (θ χ J χ (α , xα )). The moment-generating function becomes ϕ χ (α , xα ; θ χ ) = E {ϖ χ (α , xα ; θ χ )} with a time derivative of d χ ϕ (α , xα ; θ χ ) = dα  − θ χ xTα

χ

Q (α ) +

N

∑ (K

ϑ =1

 ϑ T

) (α )R

χϑ

ϑ

(α )K (α ) xα ϕ χ (α , xα ; θ χ ) .

Tactics Change Prediction and Performance Analysis

73

Using the standard Ito formula in [Dav77], one gets d ϕ χ (α , xα ; θ χ ) = E {d ϖ χ (α , xα ; θ χ )}  χ

= E ϖα (α , xα ; θ χ ) d α + ϖxχα (α , xα ; θ χ ) dxα  1  + Tr ϖxχα xα (α , xα ; θ χ )G(α )W GT (α ) d α 2   χ

= ϕα (α , xα ; θ χ ) d α + ϕxχα (α , xα ; θ χ ) A(α ) +

N

∑ Bϑ (α )K ϑ (α )

ϑ =1

xα d α

 1  + Tr ϕxχα xα (α , xα ; θ χ ) G (α )W GT (α ) d α , 2 which, when combined with (13), leads to  − θ χ xTα  =

χ

Q (α ) +

d χ χ d α ρ (α ; θ ) ρ χ (α ; θ χ )

N

∑

ϑ =1

+ xTα 

  T ϑ χϑ ϑ K (α )R (α )K (α ) xα ϕ χ (α , xα ; θ χ )

d χ ϒ (α ; θ χ )xα dα

+ xTα A(α ) +

T

N

∑ Bϑ (α )K ϑ (α )

ϑ =1



+ xTαϒ χ (α ; θ χ )

A(α ) +

N

∑B

ϒ χ (α ; θ χ )xα 

ϑ

ϑ =1 T

ϑ

(α )K (α ) xα

+ 2xTαϒ χ (α ; θ χ )G(α )W G (α )ϒ χ (α ; θ χ )xα  χ  + Tr ϒ (α ; θ χ )G(α )W GT (α ) ϕ χ (α , xα ; θ χ ) . To have constant and quadratic terms independent of xα , it is required that  T N d χ χ ϑ ϑ ϒ (α ; θ ) = − A(α ) + ∑ B (α )K (α ) ϒ χ (α ; θ χ ) dα ϑ =1   − ϒ χ (α ; θ χ ) A(α ) + χ

χ

N

∑ Bϑ (α )K ϑ (α )

ϑ =1 T

− 2ϒ (α ; θ )G (α )W G (α ) ϒ χ (α ; θ χ )  −θ

χ

χ

Q (α ) +

N

∑ (K

ϑ =1

ϑ T

) (α )R

 χϑ

ϑ

(α )K (α )



74

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

and

  d χ ρ (α ; θ χ ) = −ρ χ (α ; θ χ ) Tr ϒ χ (α ; θ χ )G (α )W GT (α ) dα ! χ with the terminal-value conditions ϒ χ (t f ; θ χ ) = θ χ Q f and ρ χ t f ; θ χ = 1. Finally, the time-backward differential equation satisfied by υ χ (α ; θ χ ) follows: ! υχ tf ;θ χ = 0 .

  d χ υ (α ; θ χ ) = −Tr ϒ χ (α ; θ χ )G (α )W GT (α ) , dα

2 Now higher-order statistics can be generated for the multi-team stochastic differential game by looking at a Maclaurin series expansion of the cumulant-generating function ∞

χ

ψ χ (α , xα ; θ χ )  ∑ κi (α , xα ) i=1 ∞

=∑

i=1

(θ χ )i i!

  ∂ (i) χ χ  ψ ( α , x ; θ )  α  ∂ (θ χ )(i)

θ χ =0

(θ χ )i i!

(19)

χ

in which the κi (α , xα )’s are called the performance cumulants associated with team χ for χ ∈ χ . Note that the series coefficients can be computed using (15),   ∂ (i) χ χ  ψ ( α , x ; θ )  α  ∂ (θ χ )(i) =

xTα

θ χ =0

  ∂ (i) χ χ  ϒ (α ; θ ) χ (i)  ∂ (θ )

θ χ =0

  ∂ (i) χ χ  xα + υ (α ; θ ) χ (i)  ∂ (θ )

. (20) θ χ =0

In view of results (19) and (20), the performance cumulants for the stochastic multiteam Nash game can be obtained as follows:     (i) (i) ∂ ∂  χ χ χ χ χ  κi (α , xα ) = xTα ϒ ( α ; θ ) x + υ ( α ; θ ) , (21)   α  χ  χ ∂ (θ χ )(i) ∂ (θ χ )(i) θ =0

θ =0

for any finite 1 ≤ i < ∞. For notational convenience, the following definitions are introduced:   ∂ (i) χ χ χ  ϒ ( α ; θ ) (22) H (α , i)    χ ∂ (θ χ )(i) θ =0  (i) ∂ χ χ  υ ( α ; θ ) (23) Dχ (α , i)    χ ∂ (θ χ )(i) θ =0

which leads to the change of notation for the result (21) χ

κi (α , xα ) = xTα H χ (α , i)xα + Dχ (α , i) .

(24)

Tactics Change Prediction and Performance Analysis

75

Situation Awareness Level 1: Perception of Elements in Current Situation

ξ χ ; u χ ; χ = 1,… , N

Team Strategies

⎡⎣t0 , t f ⎤⎦

Decision Horizon

(A, B )

Uniformly Stabilizable

χ

(Ω, F , Ft ≥0 , P )

J

(t , x ; u ,…, u 1

0

0

F1χ × × Fkχχ : ⎡⎣t0 , t f ⎤⎦ × (

n× n k χ

G × × G : ⎣⎡t0 , t f ⎦⎤ × (

n×n k χ

χ 1

J χ (t0 , x0 ; u1 ,… , u N ; ξ χ )

χ kχ

χ

;ξ

χ

x (t0 ) = x0

)= x (t )Q x (t ) T

f

χ f

f

tf

N T ⎡ ⎤ + ∫ ⎢ xT (τ )Q χ (τ ) x (τ ) + ∑ (uϑ ) (τ ) R χϑ (τ )uϑ (τ )⎥ dτ ϑ =1 ⎦ t0 ⎣

• Concern performance robustness • Shape team performance distribution • Simultaneous order of decision-making • Share the interaction model • Use feedback information pattern

kχ



kχ

χ 1

d H χ (α ) = F χ (α , H (α ), K 1 (α ),… , K N (α )); H χ (t f )≡ H χf dα d χ D (α ) = G χ (α , H χ (α )); D χ (t f )≡ D χf dα

H χf  Q χf × 0 ×  × 0 and D χf  0 ×  × 0

Long-term Memory Storage Preconception & Objectives

×  m1×n × ×  mN ×n  ( n×n )

G χ  G1χ × × Gkχχ

Environmental Uncertainty

N

) )

F  F × × Fkχχ

is a Chi-Squared r.v.!!

N ⎛ ⎞ dx (t ) = ⎜ A (t ) x (t ) + ∑ B χ (t )u χ (t )⎟ dt + G (t )dw (t ); χ =1 ⎝ ⎠ χ

Level 3: Projection of Future Status

Level 2: Comprehension of Current Situation

Optimize some state x (x0 ; w; u1 ,… , u N )

Info-Processing Mechanism

γ Pχ : Z χ  U Pχ uPχ = γ Pχ (η χ )

η χ = (t , x (t ))

Fig. 1. Situation awareness for dynamic teams.

Remark 1. The results obtained herein demonstrated a successful combination of the compactness offered by logic from the state-space model description (7) and the quantitativity from a priori probabilistic knowledge of adverse environmental disturbances so that the uncertainty of team performance measure (6) can now be represented in a compact and robust way. Such performance cumulants associated with team χ may therefore be referred to as “information” statistics which are extremely valuable for shaping team performance distribution. All the information statistics also possess symmetric, nonnegative and monotone behavior properties as can be readily seen from the works of [LiH76] and [Pha04]. Moreover, Figure 1 depicts a model for situation awareness which illustrates the state of knowledge of team χ about the adverse and dynamic environment. It incorporates the perception of relevant attributes of the decision problem and comprehension of the meaning of the unique property of the interaction model in combination with and in relation to team goals so that future projection of higher-order statistics of the chi-squared performance random variable is obtained with a high confidence level. Finally, the next theorem contains a tractable method of generating performance cumulants in the time domain. This computational procedure is preferred to that of (20) for the reason that the cumulant-generating equations now allow the incorporation of classes of linear feedback strategies in the statistical control problems. Theorem 2 (Performance Cumulants in Multi-Team Decision Problems). Let the dynamics of the stochastic multi-team Nash game by N decision teams   be controlled with the pairs (A, Bχ ) uniformly stabilizable on t0 ,t f as follows:

76

Khanh D. Pham, Stanley R. Liberty, and Gang Jin



N

dx(t) = A(t) +



∑ Bχ (t)K χ (t)

x(t)dt + G(t)dw(t) ,

x(t0 ) = x0 .

χ =1

(25)

Then, the kχ -th cumulant of the team performance measure (9) is given by χ

κkχ (K 1 , . . . , K N ) = xT0 H χ (t0 , kχ )x0 + Dχ (t0 , kχ ) ,

(26)

χ χ for kχ ∈ Z+ . The solutions {H χ (α , r) ≥ 0}r=1 and {Dχ (α , r) ≥ 0}r=1 evaluated at α = t0 satisfy the time-backward matrix differential equations

k

k

 T N d χ ϑ ϑ H (α , 1) = − A(α ) + ∑ B (α )K (α ) H χ (α , 1) dα ϑ =1   − H χ (α , 1) A(α ) + − Qχ (α ) −

N

∑ Bϑ (α )K ϑ (α )

ϑ =1

N

∑ (K ϑ )T (α )Rχϑ (α )K ϑ (α )

(27)

ϑ =1

and, for 2 ≤ r ≤ kχ ,  T N d χ H (α , r) = − A(α ) + ∑ Bϑ (α )K ϑ (α ) H χ (α , r) dα ϑ =1   − H χ (α , r) A(α ) +

N

∑ Bϑ (α )K ϑ (α )

ϑ =1

r−1

2r! H χ (α , s)G(α )W GT (α )H χ (α , r − s) s!(r − s)! s=1

−∑

(28)

and, for 1 ≤ r ≤ kχ ,   d χ D (α , r) = −Tr H χ (α , r)G(α )W GT (α ) , dα

(29)

χ

where the terminal-value conditions are H χ (t f , 1) = Q f , H χ (t f , r) = 0 for 2 ≤ r ≤ kχ , and Dχ (t f , r) = 0 for 1 ≤ r ≤ kχ . Proof. The expression of performance cumulants described in (26) is readily justified by using the result (21) and the definitions (22)–(23). What remains is to show that the solutions H χ (α , r) and Dχ (α , r) for 1 ≤ r ≤ kχ indeed satisfy the dynamical equations (27)–(28). Note that the equations (27)–(28) are satisfied by the solutions H χ (α , r) and Dχ (α , r) and can be obtained by repeatedly taking time derivatives with respect to θ χ of the supporting equations (16)–(17) together with the assump tion of A(α ) + ∑Nϑ =1 Bϑ (α )K ϑ (α ) uniformly stabilizable on t0 ,t f . 2

Tactics Change Prediction and Performance Analysis

77

In order to perform robust strategic planning, it is now desirable to assess uncertainty of a post-design cost functional of permissible decisions made by team χ in the presence of the highly dynamic nature of noncooperative decision making. Since the complete information of a cost functional of permissible decisions taken by team χ is contained in the probability density space, the result in the sequel offers an efficient methodology of how to calculate a finite number of higher-order statistics which then successfully capture the nature of the teams’ conflicts of the underlying game structure and effectively model the distribution and interaction of both rational and irrational decisions imposed by other remaining teams with respect to team χ . Corollary 1 (Decision Cumulants in Multi-Team   Decision Problems). The stochastic multi-team Nash game (25) modeled on t0 ,t f is controlled by N decision teams wherein the pairs (A, Bχ ) are uniformly stabilizable. Then for each fixed decision of the remaining teams, the kχ -th cumulant associated with the chi-squared performance measure of strategy selection for team χ   tf N ! χ 1 N T ϑ T χϑ ϑ Ju K , . . . , K = x (τ ) ∑ (K ) (τ )R (τ )K (τ ) x(τ )d τ (30) ϑ =1

t0

is given by

! κuχ K 1 , . . . , K N = xT0 Huχ (t0 , kχ )x0 (31)   kχ χ with kχ ∈ Z+ fixed. The cumulant-generating solutions Hu (α , r) r=1 evaluated at α = t0 satisfy the time-backward matrix differential equations  T N d χ ϑ ϑ H (α , 1) = − A(α ) + ∑ B (α )K (α ) Huχ (α , 1) dα u ϑ =1   − Huχ (α , 1) A(α ) + −

N

∑



Kϑ

T

ϑ =1

N

∑ Bϑ (α )K ϑ (α )

ϑ =1

(α )Rχϑ (α )K ϑ (α )

(32)

and, for 2 ≤ r ≤ kχ ,  T N d χ H (α , r) = − A(α ) + ∑ Bϑ (α )K ϑ (α ) Huχ (α , r) dα u ϑ =1   − Huχ (α , r) A(α ) +

N

∑ Bϑ (α )K ϑ (α )

ϑ =1

r−1

2r! Huχ (α , s)G(α )W GT (α )Huχ (α , r − s), s!(r − s)! s=1

−∑

χ

where the terminal-value conditions are Hu (t f , r) = 0 for 1 ≤ r ≤ kχ .

(33)

78

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

It is worth the time to observe that this research investigation focuses on the class of statistical control problems whose performance index reflects the intrinsic performance variability introduced by process noise stochasticity. It should also not be forgotten that all the performance cumulant values (26) depend in part on the known initial condition x(t0 ). Although different states x(t) will result in different values for the “performance-to-come” (10), the cumulant values are, however, functions of time-backward evolutions of the cumulant-generating components H χ (α , r) and Dχ (α , r) that totally ignore all the intermediate values x(t), except for the value X(t0 ). This fact makes the new optimization problem as considered in statistical control particularly unique as compared with the more traditional dynamic programming class of investigations. In other words, the time-backward trajectories (27)–(29) should be considered as the “new” dynamical equations for statistical control. From these equations the resulting Mayer optimization [FlR75] and associated value function in the framework of dynamic programming therefore depend on the “new” states H χ (α , r) and Dχ (α , r), not the classical states x(t) as people may often expect. χ For notational simplicity, kχ -tuple variables H χ and  D are introduced  as χ

χ

the new dynamical states for team χ with H χ (·)  H1 (·), . . . , Hkχ (·) and     ! χ χ χ D χ (·)  D1 (·), . . . , Dkχ (·) wherein each element Hr ∈ C 1 t0 ,t f ; Rn×n of ! χ H χ and each element Dr ∈ C 1 [t0 ,t f ]; R of D χ have the representations of χ χ Hr (·) = H χ (·, r) and Dr (·) = Dχ (·, r), with the right members satisfying the dynamic equations (27)–(29) on [t0 ,t f ]. For a compact formulation, the following convenient mappings are defined:   Frχ : t0 ,t f × (Rn×n )kχ × Rm1 ×n × · · · × RmN ×n → Rn×n   Grχ : t0 ,t f × (Rn×n )kχ → R with the rules of action χ F1

χ

α, H , K , . . . , K 1

 χ

−H1 (α ) A(α ) +

!

  − A(α ) + 

N

∑ Bϑ (α )K ϑ (α )

ϑ =1

Frχ α , H χ , K 1 , . . . , K 

N

N

! N

∑B

ϑ =1

T ϑ

 − A(α ) +

χ

ϑ

(α )K (α )

−Qχ (α )−

 

N

N

∑

ϑ =1

H1 (α )

 T K ϑ (α )Rχϑ (α )K ϑ (α )

N

T

∑ Bϑ (α )K ϑ (α )

ϑ =1

Hr χ (α )

r−1

2r! χ Hs χ (α )G(α )W GT(α )Hr−s (α ) s!(r − s)! s=1

−Hr (α ) A(α )+ ∑ B (α )K (α ) − ∑ χ

ϑ =1

ϑ

ϑ

  Grχ (α , H χ )  −Tr Hr χ (α )G(α )W GT (α ) ,

1 ≤ r ≤ kχ .

Tactics Change Prediction and Performance Analysis

79

Now it is straightforward to establish the product mappings  χ χ  F1 × · · · × Fkχ : t0 ,t f × (Rn×n )kχ × Rm1 ×n × · · · × RmN ×n → (Rn×n )kχ  χ χ  G1 × · · · × Gkχ : t0 ,t f × (Rn×n )kχ → Rkχ χ

χ

χ

along with the corresponding notation F χ  F1 × · · · × Fkχ and G χ  G1 × · · · × χ

Gkχ . Thus, the dynamic equations (27)–(29) with the pre-specified order kχ can be rewritten as follows: ! d χ H χ (α ) = F χ α , H χ (α ), K 1 (α ), . . . , K N (α ) , H χ (t f ) = H f (34) dα d χ χ D (α ) = G χ (α , H χ (α )) , D χ (t f ) = D f , (35) dα   χ χ χ where the terminal-value conditions H f  Q f , 0, . . . , 0 and D f  (0, . . . , 0). Note that the product system associated with team χ uniquely determines ! H χ and D χ once an N-tuple of admissible feedback gains K 1 , . . . , K!N is specified. Hence, it is !important to consider H χ ≡ H χ ·, K 1 , . . . , K N and D χ ≡ D χ ·, K 1 , . . . , K N . The corresponding performance index for team χ can be formulated in K 1 , . . . , K N . Definition 2 (Team Performance Index). Fix kχ ∈ Z+ , Pareto profile ξ χ ∈ W χ , χ

χ

χ with μ1 > 0. Then for the given initial and scalar coefficients μ χ = {μr ≥ 0}r=1 condition (t0 , x0 ), team χ minimizes its team-level performance index

k

χ

φ0 : {t0 } × (Rn×n)kχ × Rkχ → R+ , which is defined as a finite linear combination of the first kχ cumulants of chi-squared performance measure (9) and is given by χ

φ0 t0 , H

χ

! !! t0 , K 1 , . . . , K N , D χ t0 , K 1 , . . . , K N 

kχ

∑ μrχ κrχ

K1, . . . , KN

!

r=1

 ! ! = ∑ μrχ xT0 Hr χ t0 , K 1 , . . . , K N x0 + Drχ t0 , K 1 , . . . , K N , (36) kχ

r=1

χ

where μr mutually chosen by team χ represent different levels of influence as deemed ! χ important to uncertainty of the team performance. Solutions Hr α , K 1 , . . . , K N ! kχ  χ kχ and Dr α , K 1 , . . . , K N ≥ 0 r=1 evaluated at α = t0 satisfy the dynamic ≥ 0}r=1   χ χ equations (34)–(35) with the terminal-value conditions H f = Q f , 0, . . . , 0 and χ

D f = (0, . . . , 0). The performance index (36) associated with team χ is a weighted summation of χ a finite number of information statistics with μr representing multiple degrees of

80

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

shaping the probability density function of (6). If all the cumulants of (6) remain bounded as (6) arbitrarily closes to 0, the first cumulant dominates the summation and the cumulant-based optimization problem reduces to the classical LQG problem. Furthermore, we note that the subject optimization is an initial cost problem, in contrast with the more traditional terminal cost class of investigations. One may address an initial cost problem by introducing changes of variables which convert it to a terminal cost problem. However, this modifies the natural context of performance cumulants, which it is preferable to retain. Instead, one may take a more direct dynamic programming approach to the initialcost problem as illustrated below. χ χ χ For the given terminal data t f , H f , D f , the class K χ χ χ χ of admist f ,H f ,D f ;ξ ; μ

sible feedback gains for team χ is defined.

Definition 3 (Admissible Feedback Gains). For kχ ∈ Z+ , Pareto profile ξ χ ∈ W χ , χ

χ

χ

χ with μ1 > 0, there presumably exists a compact subset K ⊂ and μ χ = {μr ≥ 0}r=1

k

mχ χ ∑i=1 mi ×n

which is the set of permissible Pareto gain values associated with team χ R χ and χ ∈ χ . Then, the set of admissible feedback gains K is defined to χ χ t f ,H f ,D f ;ξ χ ; μ χ    ∑mχ mχ ×n χ with values K χ (·) ∈ K for which solutions be the class of C t0 ,t f ; R i=1 i   to the dynamic equations (34)–(35) exist on the finite horizon t0 ,t f . Since cooperation cannot be enforced in the multi-team decision problem with multicriteria objectives, a Nash equilibrium solution concept ensures that no teams have incentive to unilaterally deviate from the equilibrium decision laws in order to further reduce their performance indices. Thus, the Nash game-theoretic framework is suitable to capture the nature of conflicts as one team’s decision is tightly coupled with the decisions of remaining teams. χ

Definition 4 (Nash Equilibrium Solution). Let the performance index φ0 t0 , H χ ! 1 χ −1 χ χ +1 N t0 , K , . . . , K ,K ,K ,...,K , !! associate with!team χ where χ ∈ χ . Then, D χ t0 , K 1 , . . . , K χ −1 , K χ , K χ +1 , K N a Nash equilibrium the N-tuple of admissible feedback gains K∗1 , . . . , K∗N provides   χ

χ −1

χ

χ +1

solution, if φ0 t0 , H χ t0 , K∗1 , . . . , K∗ , K∗ , K∗ , . . . , K∗N ,   χ −1 χ χ +1 D χ t0 , K∗1 , . . . , K∗ , K∗ , K∗ , . . . , K∗N is less than or equal to     χ χ −1 χ +1 χ −1 φ0 t0 , H χ t0 , K∗1 , . . . , K∗ , K χ , K∗ , . . . , K∗N , D χ t0 , K∗1 , . . . , K∗ ,  χ +1 where K χ is any admissible feedback gain of team χ . K χ , K∗ , . . . , K∗N

When solving for a Nash equilibrium solution, it is very important to realize that N teams have different performance indices to minimize. A standard approach for a potential solution from the set of N inequalities as stated above is to solve jointly N optimal control problems defined by these inequalities, each of which depends structurally on the other team’s decision laws. However, a Nash equilibrium solution even under the feedback information structure to this class of problems cannot be unique due to informational nonuniqueness as indicated in [Bas82]. Therein, problems with

Tactics Change Prediction and Performance Analysis

81

informational nonuniqueness under the feedback information pattern and the need for more satisfactory resolution have been addressed via the requirement of the Nash equilibrium solution to have the additional property that its restriction on the interval [t0 , α ] is a Nash solution to the truncated version of the original problem, defined on [t0 , α ]. With such a restriction so defined, the solution is now termed as a feedback Nash equilibrium solution which is now free of any informational nonuniqueness, and thus whose derivation allows a dynamic programming type argument, as proposed in the sequel development. ! Definition 5 (Feedback Nash Equilibrium Solution). Let N-tuple K∗1 , . . . , K∗N be χ χ χ a feedback Nash equilibrium in ×Nχ =1 Kt ,H χ ,D χ ;ξ χ ;μ χ and (H∗ , D∗ ) the corref

f

f

sponding trajectory pair of the dynamic equations ! d H χ (α ) = F χ α , H χ (α ), K 1 (α ), . . . , K N (α ) , dα d χ χ D (α ) = G χ (α , H χ (α )) , D χ (t f ) = D f . dα

H χ (t f ) = H f

χ

! Then, the N-tuple of permissible feedback gains K∗1 , . . . , K∗N when restricted to the interval [t0 , α ] is still a feedback Nash equilibrium solution for each optimal!control χ χ problem with the appropriate terminal-value conditions α , H∗ (α ), D∗ (α ) for all α ∈ [t0 ,t f ]. Next, one may state the corresponding statistical control optimization problems for which the set of static Nash problems needs to be solved for a feedback Nash equilibrium solution and satisfies the aforementioned Nash inequalities. Definition 6 (Statistical Control Optimization). With predetermined kχ ∈ Z+ , χ

χ

χ χ Pareto profile ξ χ ∈ W χ , and  μ = {μr ≥ 0}r=1 with μ1 > 0, the decision optimization for team χ over t0 ,t f is given by

K χ (·)∈K

min χ

t f ,H

χ χ χ χ ,D ;ξ ;μ f f

k

   φ0χ t0 , H χ t0 , K∗1 , . . . , K∗χ −1 , K χ , K∗χ +1 , . . . , K∗N ,   χ −1 χ +1 (37) D χ t0 , K∗1 , . . . , K∗ , K χ , K∗ , . . . , K∗N

  subject to the dynamic equations of motion, for α ∈ t0 ,t f , ! d H χ (α ) = F χ α , H χ (α ), K 1 (α ), . . . , K N (α ) , dα d χ χ D (α ) = G χ (α , H χ (α )) , D χ (t f ) = D f . dα

H χ (t f ) = H f

χ

This optimization problem is in “Mayer form” and can be solved by applying an adaptation of the Mayer form verification theorem of dynamic programming given in [FlR75]. In the framework of dynamic programming, the terminal time and

82

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

χ χ states of a family   of optimization problems are denoted by (ε , Y , Z ) rather than χ χ t f , H f , D f . Thus, the value of these optimization problems depends on their terminal-value conditions.   Definition 7 (Team Value Functions). Suppose (ε , Y χ , Z χ ) ∈ t0 ,t f ×(Rn×n )kχ ×   Rkχ is given. The value function V χ : t0 ,t f × (Rn×n)kχ × Rkχ → R+ ∪ {+∞} associated with team χ is defined as

V χ (ε , Y χ , Z χ )  K χ (·)∈K

inf χ

χ χ t f ,H f ,D f ;ξ χ ;μ χ

   χ χ −1 χ +1 φ0 t0 , H χ t0 , K∗1 , . . . , K∗ , K χ , K∗ , . . . , K∗N ,   χ −1 χ +1 . D χ t0 , K∗1 , . . . , K∗ , K χ , K∗ , . . . , K∗N χ is empty. Unless χ χ t f ,H f ,D f ;ξ χ ; μ χ solutions H χ and D χ on

Conventionally, V χ (ε , Y χ , Z χ ) ≡ ∞ when K

oth-

erwise specified, the dependence of trajectory   χ −1 χ +1 K∗1 , . . . , K∗ , K χ , K∗ , . . . , K∗N

is now omitted for notational clarity. The following results summarize some properties of the value function as necessary conditions for optimality whose verifications can be obtained via parallel adaptations to those of the excellent treatments in [FlR75]. Theorem 3 (Necessary Conditions). The value function associated with team χ evaluated along any time-backward trajectory corresponding to a feedback decision gain feasible for its terminal states is an increasing function of time. Moreover, the value function evaluated along any optimal time-backward trajectory is constant. Concerning a construction of scalar-valued functions W χ (ε , Y χ , Z χ ) which then serve as potential candidates for the value function, these necessary conditions are also sufficient for optimality as shown in the next theorem. Theorem 4 (Sufficient Condition). Let W χ (ε , Y χ , Z χ ) be an extended real  valued function defined on t0 ,t f × (Rn×n )kχ × Rkχ such that W χ (ε , Y χ , Z χ ) ≡ χ χ χ φ0 (ε , Y χ , Z χ ) for team χ and χ ∈ χ . Further, let t f , H f , D f be given terminalχ χ value conditions. Suppose, for each trajectory pair (H , D ) corresponding to χ a permissible decision gain K χ in K , that W χ (α , H χ (α ), D χ (α )) χ χ t f ,H f ,D f ;ξ χ ; μ χ   χ is finite and time-backward increasing on t0 ,t f . If K∗ is a permissible decision χ χ χ! gain in K χ χ χ χ such that for the corresponding trajectory pair H∗ , D∗ , t f ,H f ,D f ;ξ ; μ ! χ χ χ W χ α , H∗ (α ), D  ∗ (α ) is constant, then K∗ is an optimal decision gain and χ χ χ χ W χ tf ,Hf ,Df ≡ V χ tf ,Hf ,Df .

Tactics Change Prediction and Performance Analysis

83

Definition 8 (Reachable Set). Let reachable set {Q χ }Nχ =1 for team χ be defined as follows:   χ Q χ  (ε , Y χ , Z χ ) ∈ [t0 ,t f ] × (Rn×n)kχ × Rkχ : Kε ,Y χ ,Z χ ;ξ χ ;μ χ = 0 . Moreover, it can be shown that the value function associated with team χ is satisfying a partial differential equation at each interior point of Q χ at which it is differentiable. Theorem 5 (Team Hamilton–Jacobi–Bellman (HJB) Equation). Let (ε , Y χ , Z χ ) be any interior point of the reachable set Q χ at which the value function V χ (ε , Y χ , Z χ ) is differentiable. If there exist an N-tuple of feedback Nash equi! χ librium solution K∗1 , . . . , K∗N ∈ ×Nχ =1 Kε ,Y χ ,Z χ ;ξ χ ;μ χ , then the partial differential equation  ∂ χ ∂ V χ (ε , Y χ , Z χ ) vec(G χ (ε , Y χ )) 0 = minχ V (ε , Y χ , Z χ ) + ∂ε ∂ vec(Z χ ) K χ ∈K

∂ V χ (ε , Y χ , Z χ ) vec F χ ε , Y χ , K∗1 , . . . , ∂ vec(Y χ )  χ −1 χ +1 χ N K∗ , K , K∗ , . . . , K∗

+

is satisfied where the boundary condition V

χ

(38)

χ χ! χ χ χ! t0 , H0 , D0 = φ0 t0 , H0 , D0 .

Finally, the following theorem gives the sufficient condition used to verify a feedback Nash decision gain for team χ . Theorem 6 (Verification Theorem). Fix kχ ∈ Z+ , Pareto profile ξ χ ∈ W χ , and

χ χ μ χ = {μrχ ≥ 0}r=1 with μ1 > 0. Also, let W χ (ε , Y χ , Z χ ) be a continuously differentiable solution of the HJB equation (38) which satisfies the boundary condition χ χ! χ χ χ! W χ t0 , H0 , D0 = φ0 t0 , H0 , D0 . (39)     χ χ χ −1 χ +1 Let us have t f , H f , D f ∈ Q χ ; N-tuple K∗1 , . . . , K∗ , K χ , K∗ , . . . , K∗N in k

χ ; χ χ t f ,H f ,D f ;ξ χ ; μ χ

the Cartesian product ×Nχ =1K

and the corresponding solutions

(H χ , D χ ) of the dynamical equations

! d H χ (α ) = F χ α , H χ (α ), K 1 (α ), . . . , K N (α ) , dα d χ χ D (α ) = G χ (α , H χ (α )) , D χ (t f ) = D f . dα

H χ (t f ) = H f

χ

χ χ χ increasing function of α . Then, W   (α , H (α ), D (α )) is a time-backward

If uct

χ −1

K∗1 , . . . , K∗

χ

χ +1

χ ×Nχ =1 K χ χ t f ,H f ,D f ;ξ χ ; μ χ

, . . . , K∗N

is an N-tuple in the Cartesian prod  defined on t0 ,t f with the corresponding solutions

, K∗ , K∗

84

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

  χ χ! H∗ , D∗ of the preceding dynamical equations such that, for α ∈ t0 ,t f , ! ∂ χ χ W χ α , H∗ (α ), D∗ (α ) ∂ε ! !! ∂ χ χ χ W χ α , H∗ (α ), D∗ (α ) · vec F χ α , H∗ (α ), K∗1 (α ), . . . , K∗N (α ) + χ ∂ vec(Y ) ! !! ∂ χ χ χ W χ α , H∗ (α ), D∗ (α ) · vec G χ α , H∗ (α ) , (40) + χ ∂ vec(Z ) 0=

χ

χ χ χ t f ,H f ,D f ;ξ χ ; μ χ

then K∗ is a feedback Nash decision gain in K

and

W χ (ε , Y χ , Z χ ) = V χ (ε , Y χ , Z χ ) ,

(41)

where V χ (ε , Y χ , Z χ ) is the value function associated with team χ .

4 Multi-Cumulant, Pareto and Nash Equilibrium Recall that the optimization problem being considered herein is in “Mayer form” and can be solved by applying an adaptation of the Mayer form verification theorem of dynamic programming given in [FlR75]. In the framework of dynamic programming, team χ with χ ∈ χ often requires denoting the terminaltime and states  of a family χ χ χ χ of optimization problems as (ε , Y , Z ) rather than t f , H f , D f . That is, for   ε ∈ t0 ,t f and 1 ≤ i ≤ kχ , the states of the dynamical system (34)–(35) defined on the interval [t0 , ε ] have terminal values denoted by H χ (ε ) ≡ Y χ and D χ (ε ) ≡ Z χ . Since the cumulant-based performance index (36) is quadratic affine in terms of arbitrarily fixed x0 , this observation suggests a solution to the HJB equation (38) is of the following form as indicated by (42). Corollary 2 (Time Derivatives of Candidate Value Functions). Fix kχ ∈ Z+ and let (ε , Y χ , Z χ ) be any interior points of the reachable set Q χ at which the real-valued function W χ (ε , Y χ , Z χ ) = xT0

kχ

kχ

r=1

r=1

∑ μrχ (Yrχ + Erχ (ε )) x0 + ∑ μrχ (Zrχ + Trχ (ε ))

(42)

  ! χ χ is differentiable. The parametric functions Er ∈ C 1 t0 ,t f ; Rn×n and Tr ∈   ! 1 t0 ,t f ; R are yet to be determined. Moreover, the time derivative of C W χ (ε , Y χ , Z χ ) is given by kχ   d d W χ (ε , Y χ , Z χ ) = ∑ μrχ Grχ (ε , Y χ ) + Trχ (ε ) dε dε r=1

Tactics Change Prediction and Performance Analysis

  ! d Frχ ε , Y χ , K 1 , . . . , K N + Erχ (ε ) x0 . (43) dε

kχ

+ xT0

85

∑ μrχ

r=1

Substituting the “guess” solution (42) into the HJB equation (38) and making use of the result (43) yield  0 = minχ K χ ∈K

k k

μrχ

r=1 χ

∑

+ xT0

χ

∑

xT0

kχ d χ d Er (ε ) x0 + ∑ μrχ Trχ (ε ) dε d ε r=1

k

μrχ Frχ (ε , Y χ , K 1 , . . . , K N )

r=1

χ

x0 + ∑



μrχ Grχ (ε , Y χ )

. (44)

r=1

It is important to observe that 

kχ

∑

μrχ Frχ (ε , Y χ , K 1 , . . . , K N )

r=1



kχ

− ∑ μrχ Yrχ A(ε ) + r=1

= − A(ε ) + 

N

∑ Bϑ (ε )K ϑ

ϑ =1

N

T

∑B

ϑ =1

ϑ

(ε )K

χ

r=1

χ

− μ1 Qχ (ε ) − μ1 kχ

r−1

r=2

s=1

− ∑ μrχ

kχ

∑ μrχ Yrχ

ϑ

N

∑



Kϑ

T

ϑ =1

Rχϑ (ε )K ϑ

2r!

χ

∑ s!(r − s)! Ysχ G(ε )W GT (ε )Yr−s

kχ   χ χ χ μ G ( ε , Y ) = − ∑ r r ∑ μrχ Tr Yrχ G(ε )W GT (ε ) . kχ

r=1

r=1

Differentiating the expression within the braces of (44) with respect to K χ yields the necessary condition for an extremum of (36) on [t0 , ε ], kχ

χ

−2(Bχ )T (ε ) ∑ μrχ Yrχ M0 − 2 μ1 Rχ χ (ε )K χ M0 = 0, r=1

where M0  x0 xT0 . Furthermore, M0 is an arbitrary rank-one matrix, and it must be true that kχ

K χ (ε , Y χ , Z χ ) = −(Rχ χ )−1 (ε )(Bχ )T (ε ) ∑ μˆ sχ Ysχ ,

(45)

s=1

χ

χ

χ

where μˆ s  μr /μ1 . Substituting the team decision gain expression (45) into the right member of the HJB equation (44) yields the value of the minimum k xT0

χ

∑ μrχ

r=1

k

k

χ χ d χ χ Er (ε ) − AT (ε ) ∑ μrχ Yrχ − ∑ μrχ Yrχ A(ε ) − μ1 Qχ (ε ) dε r=1 r=1

86

Khanh D. Pham, Stanley R. Liberty, and Gang Jin



kϑ

N

∑∑

+

ϑ =1 s=1

μˆ sϑ Ysϑ Bϑ (ε )

+∑

N

χ

∑

ϑ

ϑ =1

r=1

− μ1

N

∑B

μrχ Yrχ 

R

ϑϑ

−1



(ε ) B

ϑ

T

 (ε )

kχ

∑ μrχ Yrχ

r=1



kχ





(ε ) R

ϑϑ

−1



(ε ) B

ϑ

T

kϑ

(ε ) ∑



μˆ sϑ Ysϑ

s=1

−1 −1   ϑ ϑ ϑ ϑϑ χϑ ϑϑ ˆ μ Y B ( ε ) R ( ε )R ( ε ) R (ε ) ∑ s s kϑ

ϑ =1 s=1



× B

ϑ

T



kϑ

(ε ) ∑

μˆ vϑ Yvϑ

v=1

kχ

+ ∑ μrχ r=1

kχ

−∑

r=2

μrχ

2r! χ χ T Ys G(ε )W G (ε )Yr−s x0 ∑ s=1 s!(r − s)!

r−1

kχ   d χ Tr (ε ) − ∑ μrχ Tr Yrχ G(ε )W GT (ε ) . dε r=1

(46)

 χ kχ  χ kχ It is now necessary to exhibit Er (·) r=1 and Tr (·) r=1 which render the left χ k

χ are evaluated along solution side of (46) equal to zero for ε ∈ [t0 ,t f ], when {Yr }r=1 χ χ trajectories. Studying the expression (46) reveals that Er (·) and Tr (·) for 1 ≤ r ≤ kχ satisfying the time-forward matrix-valued differential equations

d χ χ χ E (ε ) = AT (ε )H1 (ε ) + H1 (ε )A(ε ) + Qχ (ε ) dε 1   kϑ −1   T N χ ϑ ϑϑ ϑ ϑ ϑ (ε ) B (ε ) ∑ μˆ s Hs (ε ) −H1 (ε ) ∑ B (ε ) R ϑ =1

 −

∑∑

ϑ =1 s=1 N

+

kϑ

N



s=1

μˆ sϑ Hsϑ (ε )Bϑ (ε )





kϑ

∑ ∑ μˆ sϑ Hsϑ (ε )Bϑ (ε )

ϑ =1 s=1



× B

ϑ

T

kϑ

(ε ) ∑

v=1



μˆ vϑ Hvϑ (ε )

R

ϑϑ

Rϑ ϑ

−1

−1

  T χ ϑ (ε ) B (ε ) H1 (ε )

−1  (ε )Rχϑ(ε ) Rϑ ϑ (ε )

Tactics Change Prediction and Performance Analysis

87

and, for 2 ≤ r ≤ kχ , d χ E (ε ) = AT (ε )Hr χ (ε ) + Hrχ (ε )A(ε ) dε r   kϑ −1   T N χ ϑ ϑϑ ϑ ϑ ϑ (ε ) B (ε ) ∑ μˆ s Hs (ε ) − Hr (ε ) ∑ B (ε ) R ϑ =1



kϑ

N

∑∑

−

ϑ =1 s=1

s=1

μˆ sϑ Hsϑ (ε )Bϑ (ε )

 R

ϑϑ

−1



(ε ) B

ϑ

T



(ε ) Hr χ (ε )

r−1

2r! χ Hs χ (ε )G(ε )W GT (ε )Hr−s (ε ) , s!(r − s)! s=1

+∑

together with, for 1 ≤ r ≤ kχ ,   d χ Tr (ε ) = Tr Hr χ (ε )G(ε )W GT (ε ) dε will work. Furthermore, at the boundary condition, it is necessary to have χ χ! χ χ χ! W χ t0 , H0 , D0 = φ0 t0 , H0 , D0 . Or, equivalently, kχ ! ! χ χ χ χ H μ + E (t ) x + 0 0 ∑ r l0 r ∑ μrχ Dl0 + Trχ (t0 ) kχ

xT0

r=1

r=1

kχ

= xT0

kχ

χ

χ

∑ μrχ Hl0 x0 + ∑ μrχ Dl0 .

r=1

r=1

Thus, matching the boundary condition yields the corresponding initial value condiχ χ tions Er (t0 ) = 0 and Tr (t0 ) = 0. Applying the feedback Nash decision gain specified in (45) along the solution trajectories of the equations (34)–(35), these equations become the Riccati-type matrix-valued differential equations d χ χ χ H (ε ) = −AT (ε )H1 (ε ) − H1 (ε )A(ε ) − Qχ (ε ) dε 1   kϑ −1   T N χ ϑ ϑϑ ϑ ϑ ϑ (ε ) B (ε ) ∑ μˆ s Hs (ε ) +H1 (ε ) ∑ B (ε ) R ϑ =1



kϑ

N

∑∑

+

ϑ =1 s=1

−

N



s=1

μˆ sϑ Hsϑ (ε )Bϑ (ε )

kϑ





∑ ∑ μˆ sϑ Hsϑ (ε )Bϑ (ε )

ϑ =1 s=1



× R

ϑϑ

−1

R

ϑϑ

Rϑ ϑ

−1 −1

(ε ) B

ϑ

T

(ε )Rχϑ(ε )

 kϑ  T ϑ ϑ ϑ (ε ) B (ε ) ∑ μˆ v Hv (ε ) v=1



 χ

(ε ) H1 (ε )

88

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

and, for 2 ≤ r ≤ kχ , d H χ (ε ) = −AT (ε )Hr χ (ε ) − Hrχ (ε )A(ε ) dε r   kϑ −1   T N χ ϑ ϑϑ ϑ ϑ ϑ (ε ) B (ε ) ∑ μˆ s Hs (ε ) + Hr (ε ) ∑ B (ε ) R ϑ =1

 +

N

kϑ

∑∑

ϑ =1 s=1

s=1

μˆ sϑ Hsϑ (ε )Bϑ (ε )

 R

ϑϑ

−1



(ε ) B

ϑ

T

 (ε ) Hr χ (ε )

r−1

2r! χ Hs χ (ε )G(ε )W GT (ε )Hr−s (ε ) , s!(r − s)! s=1

−∑

together with, for 1 ≤ r ≤ kχ ,   d χ Dr (ε ) = −Tr Hr χ (ε )G(ε )W GT (ε ) , dε χ

χ

χ

where the terminal-value conditions H1 (t f ) = Q f , Hr (t f ) = 0 for 2 ≤ r ≤ kχ , χ and Dr (t f ) = 0 for 1 ≤ r ≤ kχ . Thus, whenever the equations above admit solutions χ

k

χ

χ

k

χ

k

k

χ χ χ χ and {Dr (·)}r=1 , then the existence of {Er (·)}r=1 and {Tr (·)}r=1 are {Hr (·)}r=1 assured. By comparing equations, one may recognize that they are related to one another by

d χ d Er (ε ) = − Hr χ (ε ) , dε dε d χ d T (ε ) = − Drχ (ε ) , dε r dε χ

χ

for 1 ≤ r ≤ kχ . Enforcing the initial value conditions of Er (t0 ) = 0 and Tr (t0 ) = 0 uniquely implies that Erχ (ε ) = Hr χ (t0 ) − Hrχ (ε ) , Trχ (ε ) = Drχ (t0 ) − Drχ (ε ) , for all ε ∈ [t0 ,t f ] and yields a value function W χ (ε , Y χ , Z χ ) = V χ (ε , Y χ , Z χ ) = xT0

k

k

r=1

r=1

∑ μrχ Hrχ (t0 )x0 + ∑ μrχ Drχ (t0 )

,

Tactics Change Prediction and Performance Analysis

89

for which the sufficient condition (40) of the verification theorem is satisfied. Therefore, the feedback Nash decision gain (45) for team χ minimizing the performance index (36) becomes optimal, kχ

χ

χ

K∗ (ε ) = −(Rχ χ )−1 (ε )(Bχ )T (ε ) ∑ μˆ sχ Hs∗ (ε ) .

(47)

s=1

Theorem 7 (Multi-Cumulant, Pareto and Nash Strategy). Consider a stochastic N-team Nash game under feedback structure and together with (A, Bχ )  information + uniformly stabilizable on t0 ,t f . Fix kχ ∈ Z , Pareto profile ξ χ ∈ W χ , and μ χ = χ

χ

k

χ with μ1 > 0. Then, the feedback Nash equilibrium solution for team χ { μr ≥ 0}r=1 with its corresponding performance index (36) is achieved by the team decision gain

kχ

χ χ K∗ (α ) = −(Rχ χ )−1 (α )(Bχ )T (α ) ∑ μˆ sχ Hs∗ (α ),

(48)

s=1

χ

χ

χ

where μˆ s  μr /μ1 mutually chosen by cooperative team members mχ represent different preferable levels of influence on the uncertainty of the team performance  χ kχ distribution. Optimal solutions Hs∗ (α ) ≥ 0 s=1 satisfy the time-backward matrix differential equations  T N d χ χ ϑ ϑ H (α ) = − A(α ) + ∑ B (α )K∗ (α ) H1∗ (α ) d α 1∗ ϑ =1   χ

− H1∗ (α ) A(α ) + − Qχ (α ) −

N

∑

ϑ =1

N

∑ Bϑ (α )K∗ϑ (α )

ϑ =1

 T K∗ϑ (α )Rχϑ (α )K∗ϑ (α ) ,

χ

χ

H1∗ (t f ) = Q f , (49)

and, for 2 ≤ s ≤ kχ ,  T N d χ χ ϑ ϑ Hs∗ (α ) = − A(α ) + ∑ B (α )K∗ (α ) Hs∗ (α ) dα ϑ =1   χ

− Hs∗ (α ) A(α ) + s−1

N

∑ Bϑ (α )K∗ϑ (α )

ϑ =1

2s! χ χ Hv∗ (α )G(α )W GT (α )Hs−v,∗ (α ) , v!(s − v)! v=1

−∑

χ

Hs∗ (t f ) = 0 . (50)

Remark 2. It is observed that to have the feedback Nash solution (48)   equilibrium for team χ be defined and continuous for all α ∈ t0 ,t f , the optimal solutions

90

Khanh D. Pham, Stanley R. Liberty, and Gang Jin u*χ (t ) = K*χ (t ) x (t ) kχ

K*χ (α ) = − (R χχ ) (α )(B χ ) (α )∑ μˆ sχ H sχ* (α ) −1

−1

s =1

(

)

η Hχ = α , H χ (α ) T

N N d ⎡ ⎤ ⎡ ⎤ H1*χ (α ) = − ⎢ A (α ) + ∑ Bϑ (α ) K*ϑ (α )⎥ H1*χ (α ) − H1*χ (α ) ⎢ A (α ) + ∑ Bϑ (α ) K*ϑ (α )⎥ dα ϑ =1 ϑ =1 ⎣ ⎦ ⎣ ⎦ N

− ∑ (K*ϑ ) (α ) R χϑ (α ) K*ϑ (α ) − Q χ (α ); T

ϑ =1

H1*χ (t f )= Q χf

T

N N d ⎡ ⎤ ⎡ ⎤ H sχ* (α ) = − ⎢ A (s ) + ∑ Bϑ (α ) K*ϑ (α )⎥ H sχ* (α ) − H sχ* (α ) ⎢ A (α ) + ∑ Bϑ (α ) K*ϑ (α )⎥ dα ϑ =1 ϑ =1 ⎣ ⎦ ⎣ ⎦ s −1

2s ! H vχ* (α )G (α )WGT (α ) H sχ−v ,* (α ); v !(s − v )!

−∑ v =1

H sχ* (t f )= 0;

2 ≤ s ≤ kχ

Fig. 2. Strategy selection for dynamic teams. χ

χ

H∗ (α ) and D∗ (α ) to the equations (49)–(50) when evaluated at α = t0 must also χ χ exist. Therefore, !it is necessary that the optimal solutions H∗ (α ) and D∗ are finite  for all α ∈ t0 ,t f . Moreover, the optimal solutions of (49)–(50) exist and are continuously differentiable in a neighborhood of t f . Using a result in [Die60], these soluχ χ tions can further be extended to the left of t f as long as H∗ (α ) and D∗ (α ) remain finite. Hence, the existences of unique and continuously differentiable solutions umχχ +

amχχ

S Aw itua ar tio en n es s

n io a t es s tu S i ar en Aw

S Se trat lec egy tio n

rmχχ

u1χ a1χ +

gy te n r a io S t lect Se

y1χ

Cooperative Multi-Person Coordination

r1χ

y2χ S Aw ituat are ion ne ss St Se rate lec gy tio n

r2χ a2χ +

u2χ

Fig. 3. Lower-level multi-person coordination.

ymχχ

Tactics Change Prediction and Performance Analysis

91

uN +

aN

S Aw itua ar tio en n es s

n io a t ess tu S i ar en Aw

S Se trat lec egy tio n

rN

u1 a1

+

gy te n r a io S t lect Se

y1

yN

r1

Global Multi-Team Nash Coordination

Cooperative Multi-Person Coordination

y2 S Aw ituat are ion ne ss St Se rate lec gy tio n

r2 a

2

+ u2

Fig. 4. Higher-level multi-team coordination. χ

χ

to the equations (49)–(50) are certain if H∗ (α ) and D∗ (α ) are bounded for all α ∈ [t0 ,t f ). As a result, the candidate value functions W χ (α , H χ , D χ ) are continuously differentiable. Using the knowledge inferred by situation awareness Figure 1, team χ will then effectively make appropriate decision strategies in the dynamic environment as summarized in Figure 2. Last but not least, Figures 3–5 provide further insights into the nature of the class of multi-team decision problems by realizing Decentralized Dynamic Teams



d 1 H * (α ) = F 1 (α , H *1 (α ), K*1 (α ), , K*N (α )) dα k1 T −1 ⎡ ⎤ u*1 (t ) = − ⎢(R11 ) (t )(B1 ) (t )∑ μˆ s1 H s1* (t )⎥ x (t ) s =1 ⎣ ⎦

u1 (t )

η1 = (t , x (t ))

d H *N (α ) = F N (α , H *N (α ), K*1 (α ), , K*N (α )) dα kN −1 T ⎡ ⎤ u*N (t ) = − ⎢(R NN ) (t )(B N ) (t )∑ μˆ sN H sN* (t )⎥ x (t ) s =1 ⎣ ⎦

η N = (t , x (t ))

u N (t )

N ⎛ ⎞ dx (t ) = ⎜ A (t ) x (t ) + ∑ B χ (t )u χ (t )⎟ dt + G (t )dw (t ), χ = 1 ⎝ ⎠

x (t0 ) = x0 ,

tf

t ∈ ⎡⎣t0 , t f ⎤⎦

N T ⎡ ⎤ J χ (t0 , x0 ; u1 , , u N ; ξ χ )= xT (t f )Q χf x (t f )+ ∫ ⎢ xT (τ )Q χ (τ ) x (τ ) + ∑ (uϑ ) (τ ) R χϑ (τ )uϑ (τ )⎥ dτ ϑ =1 ⎦ t0 ⎣

Stochastic Multi-Team Decision System

Fig. 5. Interactions in dynamic decision making.

92

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

that the aggregation of dynamic teams in fact depends on their respective objectives: i.e. cooperation within each team and noncooperation among different teams.

5 Multi-Cumulant, Pareto and Minimax Solution In situations where team χ has some reason to believe that the remaining teams are no longer playing their feedback Nash equilibrium strategies, team χ will therefore assume that the other teams disregard their own performance indices and seek to do maximal harm to its performance. Team χ may then choose a pessimistic strategy, called the multi-cumulant, Pareto and minimax strategy to minimize its loss against any rational and irrational actions taken by other teams. This situation is particularly appropriate in hostile environments with adversarial operations and where team members can get together only once to decide on a team minimax profile of strategies that they want to adopt. Definition 9 (Team Minimax Solution). An admissible feedback gain Kχ is pesχ simistic for team χ if, for any admissible feedback gains K χ ∈ K χ χ χ χ, t f ,H f ,D f ;ξ ; μ

max

K 1 ,...,K χ −1 ,K χ +1 ,...,K N

 χ φ0 t0 , H

χ

! t0 , K 1 , . . . , K χ −1 , K χ , K χ +1 , . . . , K N , D χ t0 , K 1 , . . . , K χ −1 , Kχ , K χ +1 , . . . , K N

!

is less than or equal to max

K 1 ,...,K χ −1 ,K χ +1 ,...,K N

 χ φ0 t0 , H

χ

! t0 , K 1 , . . . , K χ −1 , K χ , K χ +1 , . . . , K N , D χ t0 , K 1 , . . . , K χ −1 , K χ , K χ +1 , . . . , K N

!

.

With team minimax strategies so defined, it is noted that once adversarial teams all χ choose their decision laws to damage the performance φ0 of team χ at the maximal χ extent, the team χ can then calculate its minimax performance index φ 0 as follows:  χ

φ 0  minχ K χ ∈K

max . . .

1 K 1 ∈K

max

max

χ −1 χ +1 χ +1 K χ −1 ∈K K ∈K

. . . max

N K N ∈K

χ

φ0 .

(51)

In this defensive view, if team χ uses this pessimistic strategy Kχ , its performance χ χ index is not worse than φ 0 . Probably, the performance index φ0 of team χ would not be the worst because not all N − 1 adversarial teams have chosen the combination of χ strategies to excessively worsen the performance index φ0 . Intuitively, finding the multi-cumulant, Pareto and minimax strategy for the team χ is now equivalent to solving a two-person, zero-sum game where the adversarial χ teams of the team χ choose all but the Kχ and try to maximize φ0 while the team

Tactics Change Prediction and Performance Analysis

93

χ minimizes it. Applying the results of the previous development, the next theorem follows. Theorem 8 (Multi-Cumulant, Pareto and Minimax Strategy). Consider a stochastic N-team decision problem under the feedback information structure and together with (A, Bχ ) uniformly stabilizable. Fix kχ ∈ Z+ , Pareto profile ξ χ ∈ W χ , χ

χ

χ with μ1 > 0. Associate with (25) a standard IQF perand μ χ = { μr ≥ 0}r=1 formance measure (9) where Rχ χ (·) > 0 and Rχϑ (·) < 0 whenever ϑ = χ and ϑ = 1, . . . , N. Then, the multi-cumulant, Pareto, minimax strategy for team χ is achieved by the feedback gain

k

kχ

Kχ (α ) = −(Rχ χ )−1 (α )(Bχ )T (α ) ∑ μˆ sχ Hsχ (α ),

(52)

s=1

χ

χ

χ

where μˆ s  μr /μ1 mutually chosen by cooperative team members mχ represent different levels of influence as deemed important to uncertainty of the team performance kχ χ distribution. Solutions {Hs (α ) ≥ 0}s=1 satisfy the time-backward matrix differential equations  T N d χ χ H (α ) = − A(α ) + ∑ Bϑ (α )Kϑ (α ) H1 (α ) dα 1 ϑ =1   N

χ

∑ Bϑ (α )Kϑ (α )

− H1 (α ) A(α ) + − Q( α ) −

N

∑



ϑ =1

Kϑ

T

ϑ =1

(α )Rχϑ (α )Kϑ (α ) ,

χ

χ

H1 (t f ) = Q f

(53)

and, for 2 ≤ s ≤ kχ ,  T N d χ ϑ ϑ H (α ) = − A(α ) + ∑ B (α )K (α ) Hsχ (α ) dα s ϑ =1   − Hsχ (α ) A(α ) +

N

∑ Bϑ (α )Kϑ (α )

ϑ =1

s−1

2s! χ Hvχ (α )G(α )W GT (α )Hs−v (α ), Hsχ (t f ) = 0. v!(s − v)! v=1

−∑

(54)

Remark 3. It is important to note that optimal team decision laws in the aforementioned multi-team noncooperative games are linear functions of the current decision states. The optimally decentralized team decision gains (48) and (52) operate dynamically on the time-backward histories of the linear combination of information statistics satisfying the equations of dynamics (49)–(50) and (53)–(54) from the final to the current time. Moreover, it is obvious to realize that these information statistics

94

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

then depend upon the process noise characteristics. In other words, competing teams employing optimal decision gains (48) and (52) have deliberately traded the property of the certainty equivalence principle that they would obtain from the special case of LQG control, for the adaptability to deal with highly dynamic uncertainty and nonstationary team objectives.

6 Conclusions This research work proposed a hierarchical game-theoretic approach to the difficult challenges of strategy selection and performance analysis problems within stochastic, multi-team, and noncooperative games. It was of interest to find a good approach to deal with choices that noncooperative teams may take to reach a feedback Nash equilibrium solution via complete statistical characterization both in team performance and in courses of action. The approach was built upon the statistical control theory that has recently been developed for optimal stochastic regulator problems. Taking these control theoretical results as the starting point, it was shown that higher-order statistics associated with performance measures for noncooperative teams provide an effective capability of shaping the probabilistic distributions of team-level performances and decision strategy profiles. Some new solution concepts were outlined: 1) the multi-cumulant, Pareto and Nash strategy for multi-team Nash games where there are certain cooperations among various rival teams; and 2) the multi-cumulant, Pareto and minimax strategy for the case where there is no cooperation among rival teams. These technical results illustrate the increasing flexibility in team decision strategies which allow multiple degrees of shaping the probability density function for the team performance measure. The additional cumulants needed in team decision strategies over the noise characteristics from the decision-making process have been termed as information statistics. With these information statistics, the statistical control problem has therefore two important properties. The first property is to trade off the certainty equivalence principle for adaptability to unexpected events and multi-objectives. The second property is that multi-resolution representations of team performance uncertainty and learning decision strategies from logic and probability can be combined to maintain resulting representations of team performance and decision-making uncertainty that are now compact and robust. Future research is also needed to better understand the issue of selecting a cooperative profile ξ χ for team χ from a Pareto parameterization subset W χ wherein different impact levels on the team outcome resulting from the actions of the team χ mχ members are designated as nonnegative and scalar-valued components {ξi }i=1 . As can be seen from the previous analysis, the choice of team cooperative profile ξ χ for team χ was determined by a mutual agreement among all the team members. However, in some cases, there may exist team leaders among various teams who have different objective functions than their team members and ultimately have the responsibility to make these choices. Team leaders need to evaluate how multi-cumulant and Nash strategies carried out by their team members would affect the overall outcomes of the teams. Hence, the theory of games should once again be employed for

Tactics Change Prediction and Performance Analysis

95

such a higher level of command and control analysis—this time a noncooperative game between team leaders. Finally, there is a need to relax the assumption on the closed-loop with no memory strategies employed by competing teams in situations of adversarial noisy environments, where acts of active denial and deception from each team are present.

Acknowledgments This material is based upon work supported in part by the U.S. Air Force Research Laboratory-Space Vehicles Directorate and the U.S. Air Force Office of Scientific Research under grant number LRI 00VS17COR. Much appreciation from the first author goes to Dr. Benjamin K. Henderson, the branch technical advisor of Spacecraft Components Technology, for serving as the reader of this work and providing helpful criticism.

References [Bas82]

T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, Academic Press, London, 1982. [Cru01] J. B. Cruz, Jr., M. A. Simaan, A. Gacic, H. Jiang, B. Letellier, M. Li, and Y. Liu, Game-Theoretic Modeling and Control of Military Operations, IEEE Transactions on Aerospace and Electronic Systems, Vol. 37, No. 4, pp. 1393–1405, October 2001. [Dav77] M. H. A. Davis, Linear Estimation and Stochastic Control, A Halsted Press, John Wiley & Sons, New York, 1977. [Die60] J. Dieudonne, Foundations of Modern Analysis, Academic Press, New York and London, 1960. [DiS05] R. W. Diersing and M. K. Sain, The Third Generation Wind Structural Benchmark: A Nash Cumulant Robust Approach, Proceedings of the American Control Conference, pp. 3078–3083, Portland, Oregon, June 8–10, 2005. [DiS06] R. W. Diersing and M. K. Sain, Nash and Minimax Bi-Cumulant Games, The 45th IEEE Conference on Decision and Control, pp. 2571–2576, San Diego, California, December 13–15, 2006. [DSPW07] R. Diersing, M. K. Sain, K. D. Pham, and C.-H. Won, Output Feedback Multiobjective Cumulant Control with Structural Applications, Proceedings of the American Control Conference, pp. 2666–2671, New York City, New York, 2007. [FlR75] W. H. Fleming and R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, 1975. [Jac73] D. H. Jacobson, Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relation to Deterministic Games, IEEE Transactions on Automatic Control, Vol. AC-18, pp. 124–131, 1973. [Kli64] A. Klinger, Vector-Valued Performance Criteria, IEEE Transactions on Automatic Control, Vol. AC-9, No. 1, pp. 117–118, 1964. [LiH76] S. R. Liberty and R. C. Hartwig, On the Essential Quadratic Nature of LQG Control-Performance Measure Cumulants, Information and Control, Vol. 32, No. 3, pp. 276–305, 1976.

96 [LSC03]

Khanh D. Pham, Stanley R. Liberty, and Gang Jin

Y. Liu, M. A. Simaan, and J. B. Cruz, Jr., Game Theoretic Approach to Cooperative Teaming and Tasking in the Presence of an Adversary, Proceedings of the American Control Conference, pp. 5375–5380, Denver, Colorado, June 4–6, 2003. [PLSS99] K. D. Pham, S. R. Liberty, M. K. Sain, and B. F. Spencer, Jr., Generalized Risk Sensitive Building Control: Protecting Civil Structures with Multiple Cost Cumulants, Proceedings of the American Control Conference, pp. 500–504, San Diego, California, June 1999. [PLS99] K. D. Pham, S. R. Liberty, and M. K. Sain, Evaluating Cumulant Controllers on a Benchmark Structure Protection Problem in the Presence of Classic Earthquakes, Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing, pp. 617–626, Monticello, Illinois, September 22–24, 1999. [PLSS00] K. D. Pham, S. R. Liberty, M. K. Sain, and B. F. Spencer, Jr., First Generation Seismic-AMD Benchmark: Robust Structural Protection by the Cost Cumulant Control Paradigm, Proceedings of the American Control Conference, pp. 1–5, Chicago, Illinois, June 28–30, 2000. [PSL02a] K. D. Pham, M. K. Sain, and S. R. Liberty, Robust Cost-Cumulants Based Algorithm for Second and Third Generation Structural Control Benchmarks, Proceedings of the American Control Conference, pp. 3070–3075, Anchorage, Alaska, May 08–10, 2002. [PSL02b] K. D. Pham, M. K. Sain, and S. R. Liberty, Finite Horizon Full-State Feedback kCC Control in Civil Structures Protection, Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences, Proceedings of the Workshop Held in Lawrence, Kansas, Edited by B. Pasik-Duncan, SpringerVerlag, Berlin Heidelberg, Germany, Vol. 280, pp. 369–383, September 2002. [PSL02c] K. D. Pham, M. K. Sain, and S. R. Liberty, Cost Cumulant Control: StateFeedback, Finite-Horizon Paradigm with Application to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, Vol. 115, No. 3, pp. 685–710, December 2002. [PJSSL04] K. D. Pham, G. Jin, M. K. Sain, B. F. Spencer, Jr., and S. R. Liberty, Generalized LQG Techniques for the Wind Benchmark Problem, Special Issue of ASCE Journal of Engineering Mechanics on the Structural Control Benchmark Problem, Vol. 130, No. 4, pp. 466–470, April 2004. [Pha04] K. D. Pham, Statistical Control Paradigms for Structural Vibration Suppression, Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 2004. [PSL04] K. D. Pham, M. K. Sain, and S. R. Liberty, Infinite Horizon Robustly Stable Seismic Protection of Cable-Stayed Bridges Using Cost Cumulants, Proceedings of the American Control Conference, pp. 691–696, Boston, Massachusetts, June 30, 2004. [PSL05] K. D. Pham, M. K. Sain, and S. R. Liberty, Statistical Control for Smart BaseIsolated Buildings via Cost Cumulants and Output Feedback Paradigm, Proceedings of the American Control Conference, pp. 3090–3095, Portland, Oregon, June 8–10, 2005. [Pha05] K. D. Pham, Minimax Design of Statistics-Based Control with Noise Uncertainty for Highway Bridges, Proceedings of DETC 2005/2005 ASME 20th Biennial Conference on Mechanical Vibration and Noise: Active Control of Vibration and Acoustics I, DETC2005-84593, Long Beach, California, September 24–28, 2005.

Tactics Change Prediction and Performance Analysis [PhRo7]

[Pha07b]

[Pha08b]

[Pha07c]

[Pha07d]

[PLR08c]

[Pha08d]

[Zad63]

97

K. D. Pham and L. Robertson, Statistical Control Paradigm for Aerospace Structures Under Impulsive Disturbances, The 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, AIAA 2007-1755, Honolulu, Hawaii, April 23–26, 2007. K. D. Pham, Cost Cumulant-Based Control for a Class of Linear-Quadratic Tracking Problems, Proceedings of the American Control Conference, pp. 335– 340, New York, New York, 2007. K. D. Pham, On Statistical Control of Stochastic Servo-Systems: PerformanceMeasure Statistics and State-Feedback Paradigm, The 17th International Federation of Automatic Control, Accepted, Seoul, Korea, 2008. K. D. Pham, Multi-Cumulant Control for Zero-Sum Differential Games: Performance-Measure Statistics and State-Feedback Paradigm, The 7th International Conference on Cooperative Control and Optimization, In Press, Gainesville, Florida, January 31–February 02, 2007. K. D. Pham, Cooperative Solutions in Multi-Person Quadratic Decision Problems: Finite-Horizon and State-Feedback Cost-Cumulant Control Paradigm, The 46th IEEE Conference on Decision and Control, Accepted, pp. 2484–2490, New Orleans, Loussiana, December 12–14, 2007. K. D. Pham, S. Lacy, and L. Robertson, Multi-Cumulant and Non-Inferior Stategies for Multi-Player Pursuit-Evasion, Proceedings of American Control Conference, Accepted, Seattles, Washington, 2008. K. D. Pham, Non-Cooperative Outcomes for Stochastic Multi-Player Nash Games: Novel Decision Strategies for Multi-Resolution Performance Robustness, The 17th International Federation of Automatic Control, Accepted, Seoul, Korea, 2008. L. A. Zadeh, Optimality and Non-Scalar-Valued Performance Criteria, IEEE Transactions on Automatic Control, Vol. AC-8, No. 1, pp. 59–60, 1963.

A Multiobjective Cumulant Control Problem Ronald W. Diersing,1 Michael K. Sain,2 and Chang-Hee Won3 1 2 3

Department of Engineering, University of Southern Indiana, Evansville, IN 47712, USA. [email protected] Freimann Professor of Electrical Engineering, University of Notre Dame, Notre Dame, IN 46556, USA. [email protected] Department of Electrical and Computer Engineering, Temple University, Philadelphia, PA 19122, USA. [email protected]

Summary. The H2 /H∞ control problem is well known in the control community. It mixes the results of two powerful control techniques; to balance two objectives: minimizing the H2 norm of the system, while constraining the system’s H∞ norm. In the presence of random noise, this is akin to solving a Nash game with the players’ objectives to minimize the mean of their costs. In this chapter, recent trends in minimizing further cumulants will be analyzed, in particular one in which wishes to minimize the variance and other cumulants of a cost, while constraining the system’s H∞ norm. This problem formulation will begin for a class of nonlinear systems with nonquadratic costs. Sufficient conditions for a Nash equilibrium for a two player game in which the control wishes to minimize the variance of its costs and the disturbance wishes to minimize the mean of its cost are found. The case of linear systems and quadratic costs is applied and equilibrium solutions are determined. Further cumulants are also examined. The results of the control formulation are applied to a problem in structural control, namely, the third generation structural benchmark for tall buildings subject to high winds.

1 Introduction One problem in multiobjective control is mixed H2 /H∞ control [BH89, ZGBD94]. Using a Nash game approach [LAH94, CZ01] for this problem, the control wishes to minimize its cost, which corresponds with an H2 norm on the system; while the disturbance wants to minimize its cost, corresponding to an H∞ norm constraint. When a random noise is added to the system, the players want to instead minimize the mean of their costs. One could view this problem as a first cumulant Nash game, because the first cumulant is the mean. Cumulants can be very powerful in control. As moments can be found through the first characteristic function of a random variable, the cumulants can be determined as derivatives of the second characteristic function (which is the natural logarithm of the first). So it can be seen that if one knows all of the cumulants of a cost function, then the probability density function can be completely characterized. In recent times the k cost cumulant (kCC) and the minimum cost variance (MCV) C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 5, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

100

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

control problems have gained attention [PSL02a, b]– [PJSSL04], [SWSL00]. These control methods generalize the approach of minimizing the mean of a cost function which is so prevalent in the area of control. They let the control minimize a linear combination of the cumulants. In the MCV problem this means minimizing a linear combination of the mean and the variance of a cost function, whereas the kCC goes beyond these two most well-known cumulants. These methods have been applied successfully to vibration control problems, in particular the control of structures excited by winds and seismic disturbances. This is one way of characterizing uncertainty in a system. The H∞ norm is another. So by combining these into a multiobjective cumulant control problem, we see that the different kinds of uncertainties in a system can be accounted for. With this motivation, the chapter is organized as follows. The development is carried out for a class of nonlinear systems with nonquadratic costs. It is applied to the case when the system is linear and costs are quadratic. The Nash game involves two players: a control and a disturbance. Later the disturbance will be given as the result of some “structured” uncertainty inherent in the system. Lastly the control will be applied to the third generation benchmark for buildings under seismic excitation.

2 Preliminaries Consider the following stochastic differential equation: dx(t) = f (t, x(t), u(t), w(t))dt + σ (t, x(t))d ξ (t),

(1)

where x(t0 ) = x0 is a random variable independent of ξ , ξ is d-dimensional Brownian motion on the probability space (Ω , F , P), x(t) ∈ Rn is the state, u(t) ∈ U is the control, w(t) ∈ W is the disturbance, and t ∈ T = [t0 ,t f ]. Let Q0 = (t0 ,t f ) × Rn and Q¯ 0 be its closure, that is Q¯ 0 = T × Rn . Assume the functions f and σ to be Borel measurable and of class C1 (Q¯ 0 × U × W ) and C1 (Q¯ 0 ) respectively. This means that the functions f and σ have continuous partial derivatives of first order. Furthermore assume that f and σ satisfy the following conditions. (i) There exists a constant C such that  f (t, x, u, w) ≤ C(1 + x + u + w) σ (t, x) ≤ C(1 + x) for all (t, x, u, w) ∈ Q¯ 0 × U × W , (t, x) ∈ Q¯ 0 , and where  ·  is the Euclidean norm. (ii) There is a constant K so that  f (t, x, ˜ u, ˜ w) ˜ − f (t, x, u, w) ≤K(x˜ − x + u− ˜ u + w˜ − w) σ (t, x) ˜ − σ (t, x) ≤Kx˜ − x for all t ∈ T ; x, x˜ ∈ Rn ; u, u˜ ∈ U ; and w, w˜ ∈ W .

A Multiobjective Cumulant Control Problem

101

Now we shall assume some conditions on the strategies of the control and disturbance. First assume that the strategies are of the form u(t) = μ (t, x(t)) and w(t) = ν (t, x(t)). Furthermore the functions μ : Q¯ 0 → U and ν : Q¯ 0 → W are assumed to be Borel measurable and to satisfy (i) for some constant C˜ ˜ + x) and ν (t, x) ≤ C(1 ˜ + x) μ (t, x) ≤ C(1 (ii) there exists a constant K˜ such that ˜ x˜ − x)  μ (t, x) ˜ − μ (t, x) ≤ K( ˜ x˜ − x), ˜ − ν (t, x) ≤ K( ν (t, x) where t ∈ T and x, x˜ ∈ Rn . Often we will suppress the dependence on t and x and refer to the strategies as μ and ν . If the strategies μ and ν satisfy these conditions, then they are admissible strategies. We can rewrite the stochastic differential equation as dx(t) = f˜(t, x(t))dt + σ (t, x(t))d ξ (t)

x(t0 ) = x0 ,

(2)

where the strategy (μ , ν ) has been substituted into f and is now called f˜. The conditions of Theorem V4.1 of [FR75] are now satisfied and we see that if Ex(t0 )2 < ∞, then a solution of (1) exists. Furthermore the solution x(t) is unique in the sense that if there exists another solution x(t) ˜ with x(t ˜ 0 ) = x0 , then the two solutions have the same sample paths with probability 1. The resulting process is a Markov diffusion process ( [FR75] pg. 123) and the moments of x(t) are bounded. Let C1,2 (Q¯ 0 ) be the class of functions Φ that have continuous first partial derivatives with respect to t and continuous second partial derivatives with respect to x: ¯ 0 ) be the class of functions Φ (t, x) Φt , Φxi , Φxi x j for i, j = 1, 2, · · · , n. Now let C1,2 p (Q 1,2 that are of class C (Q¯ 0 ) but where Φ , Φt , Φxi , Φxi ,x j satisfy a polynomial growth condition. A polynomial growth condition for a function Φ is such that there exist constants c1 and c2 so that Φ (t, x) ≤ c1 (1 + xc2 ) for all (t, x) ∈ Q¯ 0 . This yields the Dynkin formula

 t f  Φ (t, x) =E −O μ ,ν Φ (s, x(s))ds|x(t) = x (3)  t + E Φ (t f , x(t f ))|x(t) = x , where O μ ,ν is the backward evolution operator given by O μ ,ν =

∂ ∂ + f  (t, x, u, w) ∂t ∂x   1 ∂2  + tr σ (t, x)W (t)σ (t, x) 2 , 2 ∂x

(4)

where E{d ξ (t)d ξ (t)} = W (t), superscript  denotes transpose, and tr refers to the trace operator. The expectation in (3) will now be referred to as Etx .

102

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

3 Problem Definition The game described by (1) has with it two cost functions. The first cost function, J1 , is to be associated with the control u and the second, J2 , is for the disturbance w. Both players wish to minimize their respective cost functions. When the arguments of the state, control, or disturbance w are missing, it should be assumed that they are just suppressed. The players’ cost functions are given by tf J1 (t, x, u, w) = L1 (τ , x, u, w)d τ + ψ1 (x f ) (5) t

tf J2 (t, x, u, w) =

L2 (τ , x, u, w)d τ + ψ2(x f ),

(6)

t

where L1 , L2 are the running cost functions, ψ1 , ψ2 are the terminal cost functions for each player respectively, and x(t f ) = x f . Assume the running cost Li satisfies a polynomial growth condition Li (t, x, u, w) ≤ k(1 + xc + uc + wc ) and the terminal cost ψi satisfies a polynomial growth condition ψi (t, x) ≤ k(1 + xc ), where k, c are some constants and for i = 1, 2. The game to be considered here is one in which the first player, the control u, wishes to minimize a performance index consisting of a linear combination of cumulants given by

φ1 (t, x, u, w) = Etx {J1 (t, x, u, w)} + γ Vartx {J1 (t, x, u, w)},

(7)

where γ is some positive constant and Vartx is the normal definition of variance only using the conditional expectation. On the other hand, the second player, the disturbance w, wishes to minimize the mean of its cost function. That is, the disturbance has φ2 (t, x, u, w) = Etx {J2 (t, x, u, w)} (8) as its own performance index. Because both players will be assumed to have feedback information available to them, UF will be the information pattern for the control and WF will be the information pattern for the disturbance. Thus, UF is the class of all feedback strategies μ already described, and similarly for WF . Now we define what is meant by a Nash equilibrium solution to the game. Definition 1. The pair (μ ∗ , ν ∗ ) is a Nash equilibrium solution if it satisfies the inequalities φ1 (0, x, μ ∗ , ν ∗ ) ≤ φ1 (0, x, μ , ν ∗ )

φ2 (0, x, μ ∗ , ν ∗ ) ≤ φ2 (0, x, μ ∗ , ν ) ∀μ ∈ UF and ∀ν ∈ WF . Now let V1 (t, x; μ , ν ) = Etx {J1 (t, x, u, w)} and V2 (t, x; μ , ν ) = Etx {J12 (t, x, u, w)} be the first and second moments of the cost function J1 (t, x, u, w). Definition 2. A function M : Q¯ 0 → R+ is an admissible mean cost function if there exists an admissible strategy μ such that M(t, x) = V1 (t, x; μ , ν ∗ ) for t ∈ T, x ∈ Rn .

A Multiobjective Cumulant Control Problem

103

From now on we shall assume that M is an admissible mean cost function. Definition 3. M defines a class of admissible strategies UM such that μ ∈ UM if and only if the strategy μ is admissible and satisfies Definition 2. Definition 4. An MCV control strategy μ ∗ ∈ UM is one that minimizes the second moment, i.e. V2 (t, x, μ ∗ , ν ∗ ) = V2 (t, x) ≤ V2 (t, x, μ , ν ∗ ) for t ∈ T, x ∈ Rn , ν ∗ ∈ WF , where μ ∈ UM . Furthermore the variance is found through V (t, x) = V2 (t, x) − M 2 (t, x).

4 Two Cumulant Case 4.1 Nonlinear Nash Solution We shall begin this section by giving several lemmas that will be used in the proof of the control’s Nash equilibrium strategy. The first lemma will help by providing a necessary condition for the mean of the cost function. These lemmas follow the work in [SWS92] closely. ¯ 0 ) be an admissible mean cost function and μ be an Lemma 1. Let M ∈ C1,2 p (Q admissible control strategy such that it satisfies Definition 2. Under these assumptions the admissible mean cost function M satisfies ∗

O μ ,ν M(t, x) + L1 (t, x, μ , ν ∗ ) = 0,

(9)

where M(t f , x f ) = ψ1 (x f ). Now we have the following Verification Lemma for the mean of the cost function. It provides sufficient conditions for the mean value function. Here the set Q is to be an open subset of Q0 . ¯ be a solution to Lemma 2 (Verification Lemma). Let M ∈ C1,2 p (Q) ∩C(Q) ∗

O μ ,ν M(t, x) + L1 (t, x, μ , ν ∗ ) = 0

(10)

with boundary condition M(t f , x f ) = ψ1 (x f ). Then M(t, x) = V1 (t, x; μ , ν ∗ ) for all μ ∈ UM . Now that we have the results for the mean of the cost, we have the following Verification Lemma for the second moment of the cost. ¯ be a nonnegative soluLemma 3 (Verification Lemma). Let V2 ∈ C1,2 p (Q) ∩ C(Q) tion to the partial differential equation   ∗ min O μ ,ν V2 (t, x) + 2M(t, x)L1 (t, x, μ , ν ∗ ) = 0 (11) μ ∈UM

104

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

with boundary condition V2 (t f , x f ) = ψ12 (x f ). Then V2 (t, x) ≤ V2 (t, x; μ , ν ∗ ) for every μ ∈ UM , and (t, x) ∈ Q¯ 0 . If μ also satisfies   ∗ min O μ˜ ,ν V2 (t, x) + 2M(t, x)L1 (t, x, μ˜ , ν ∗ ) μ˜ ∈UM (12) ∗ = O μ ,ν V2 (t, x) + 2M(t, x)L1 (t, x, μ , ν ∗ ) for all (t, x) ∈ Q¯ 0 , then V2 (t, x) = V2 (t, x; μ , ν ∗ ). Proof. The proof follows closely that of Theorem 4.2 of [SWSL00].

2

From these lemmas, we can begin to discuss the Nash equilibrium solution. The following theorem provides sufficient conditions for the Nash equilibrium solution. Theorem 1. Consider the two player game described by (1), (7), and (8). Let M be ¯ with an associated UM . Also an admissible mean cost function, M ∈ C1,2 p (Q) ∩C(Q), 1,2 ¯ consider the function V ∈ C p (Q) ∩C(Q) that is a solution to  ∂V ∂V (t, x) + f  (t, x, μ , ν ∗ ) (t, x) min μ ∈UM ∂t ∂x   1 ∂ 2V  (13) + tr σ (t, x)W (t)σ (t, x) 2 (t, x) 2 ∂x 2   ∂M (t, x) =0 +  ∂x σ (t,x)W (t)σ  (t,x) ¯ that satisfies with V (t f , x f ) = 0 and the function P ∈ C1,2 p (Q) ∩C(Q)

∂P ∂P (t, x) + f  (t, x, μ ∗ , ν ) (t, x) min ν ∈WF ∂t ∂x   ∂ 2P 1  + tr σ (t, x)W (t)σ (t, x) 2 (t, x) 2 ∂x  + L2 (t, x, μ ∗ , ν ) = 0

(14)

with P(t f , x f ) = ψ2 (x f ). If the strategies μ ∗ and ν ∗ are the minimizing arguments of (13) and (14), then the pair (μ ∗ , ν ∗ ) constitutes a Nash equilibrium solution. Proof. The proof for this is given in [DS07].

2

5 Higher Order Cumulant Cases 5.1 Definitions In this section we give some definitions. The approach taken by Won in [Won05] will be used.

A Multiobjective Cumulant Control Problem

105

Definition 5. A function M j : Q¯ 0 → R+ is an admissible jth moment cost function if there exists a control strategy μ such that M j (t, x) = V j (t, x; μ , ν ∗ ), where M j ∈ C1,2 (Q¯ 0 ) and j = 1, 2, · · · , k − 1. Definition 6. Let the class of admissible control laws UM be such that if μ ∈ UM then μ is such that it satisfies the equality from the definitions of M1 , · · · , M j , where 1 ≤ j < k. Note that the first cumulant is the same as the first moment. The first and second cumulant cost functions K1 , K2 ∈ C1,2 (Q¯ 0 ) are given by K1 (t, x) = M1 (t, x) and K2 (t, x) = M2 (t, x) − M12 (t, x) respectively. Definition 7. Let Mi for i = 1, · · · , j + 1 be the ith admissible moment cost functions. The ( j + 1)-st admissible cumulant cost function K j+1 (t, x) is defined by j

( j)! M j−i (t, x)Ki+1 (t, x), i=0 i!( j − i)!

K j+1 (t, x) = M j+1 (t, x) − ∑

(15)

where Ki for i = 1, · · · , j is the ith admissible cumulant cost function. If μ ∈ UM , then K j+1 (t, x) = Λ j+1 (t, x). Definition 8. Let K1 , · · · , K j be admissible 1st, · · · , jth cumulant cost functions. The control strategy μ ∗ is the control’s equilibrium solution if it is such that M j+1 (t, x) = V j+1 (t, x; μ ∗ , ν ∗ ) ≤ V j+1 (t, x; μ , ν ∗ ) for all μ ∈ UM . Furthermore the ( j + 1)-st cumulant cost function is given by K j+1 (t, x) = Λ j+1 (t, x; μ ∗ , ν ∗ ) ≤ Λ j+1 (t, x; μ , ν ∗ ). 5.2 The jth Moment It will be assumed that the disturbance has played its equilibrium strategy. The moment recursion formulae were first given in the paper by Sain 1967 [Sai67]. This paper showed that for the optimal control problem, the ( j + 1)-st moment of the cost function, V j+1 (t, x; μ , ν ∗ ), satisfies ∗

O μ ,ν V j+1 (t, x; μ , ν ∗ ) + ( j + 1)V j (t, x; μ , ν ∗ )L(t, x; μ , ν ∗ ) = 0,

(16)

∗

where O μ ,ν is the backward evolution operator. If the first j moments’ cost functions are admissible moment cost functions, then they satisfy ∗

O μ ,ν M1 (t, x)+L1 (t, x, μ , ν ∗ ) = 0 ∗

O μ ,ν M2 (t, x)+2M1 (t, x)L1 (t, x, μ , ν ∗ ) = 0 .. . ∗

O μ ,ν M j (t, x)+ jM j−1 (t, x)L1 (t, x, μ , ν ∗ ) = 0.

(17)

106

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

Before moving further, the following useful lemma will be presented. Lemma 4. Consider the running cost function L1 (t, x, μ , ν ∗ ), which is denoted by Lt . Then the equality  t f

tf ( j + 1)

Ls

j Lr dr

t

 t f ds =

 j+1 Lr dr

s

(18)

t

holds. Proof. First we should change the limits of integration:  t f

tf Ls

j Lr dr

t

ds = (−1)

j

 s

t Ls

Lr dr

tf

s

j ds.

tf

Now recall that for two differential functions F and G we can integrate by parts, t t F(s)g(s)ds = F(t)G(t) − F(t f )G(t f ) − f (s)G(s)ds, tf

where f (s) =

tf

dF(s) ds ,

s G(s) =

g(r)dr. Let g(s) = Ls and tf



j

s

Lr dr

F(s) =

.

tf

With these definitions we see that  s

 j−1 Lr dr

f (s) = jLs tf s

G(s) =

Lr dr, tf

which then yields (−1)

j

 s

t Ls tf



j Lr dr

ds = (−1)

tf

j

( j+1)

t

Ls ds

− (−1)

tf

which is



t ( j + 1)

Ls tf

and the lemma is proved.

Lr dr tf



t

 t ds =

j

s

jLs tf

j

s

j

Lr dr

ds,

tf

( j+1) Ls ds

tf

2

A Multiobjective Cumulant Control Problem

107

Now consider the jth moment equation. We can show that a function M j that satisfies this equation is in fact the jth moment. ¯ that satisfies Lemma 5. Consider a function M j ∈ C1,2 p (Q) ∩C(Q) O k M j (t, x) + jM j−1 (t, x)L1 (t, x, μ , ν ∗ ) = 0,

(19)

where M j−1 is an admissible ( j − 1) moment cost function. Then M j (t, x) = V j (t, x; μ , ν ∗ ). Proof. Suppose that M j−1 is indeed an admissible ( j − 1) cost function and that M j satisfies (19). Since M j ∈ C1,2 p (Q), the Dynkin formula can be used to give

 t f  jM j−1 (s, x(s))Ls ds + ψ j (x(t f )) , (20) M j (t, x) = Etx t

where L1 (t, x(t), μ (t, x(t)), ν ∗ (t, x(t))) is denoted by Lt . But if M j−1 is an admissible ( j − 1) cost function, then it is such that M j−1 = V j−1 (t, x; μ , ν ∗ ). Therefore we have    tf

M j (t, x) = Etx

j−1

tf

Lr dr + ψ (x(t f ))

jLs Esx t

ds + ψ j (x(t f )) ,

(21)

s

which gives  tf M j (t, x) = Etx

 jLs

Esx 

t

 t f



s

 t f

tf

= Etx Esx

jLs t

 tf

 t f

jLs

= Etx t

 j−1 j ds + ψ (x(t f )) Lr dr + ψ (x(t f ))  j−1   Lr dr + ψ (x(t f )) ds + Etx ψ j (x(t f ))

s

 j−1 j Lr dr + ψ (x(t f )) ds + ψ (x(t f )) .

s

Recall the binomial formula for two real numbers p, q: n   n n−m m n (p + q) = ∑ p q . m m=0 This formula can now be applied to the term in the integral that is raised to the tf Lr dr and q = ψ (x(t f )). This yields ( j − 1)-st power where p = s

 t f s

 j−1 Lr dr + ψ (x(t f )) =

j−1 

∑

i=0

  t f  j−1−i j−1 Lr dr ψ m (x(t f )); i s

108

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

and, by looking at the term in the expectation, we have j−1 

  t f  j−1−i j−1 L dr ψ i (x(t f ))ds + ψ j (x(t f )) r ∑ i t s i=0 tf  t f  j−1−i j−1  j−1 =∑ jLs Lr dr ψ i (x(t f ))ds + ψ j (x(t f )). i t s i=0 tf

jLs

But notice that j

j−1 i

j−1 

=

∑

i=0

j i

!

( j−1)! = j ( j−1−i)!i! =

tf

( j − i)Ls

t

j! ( j−i)!i! ( j − i).

 t f

This results in

 j−1−i Lr dr

ψ i (x(t f ))ds + ψ j (x(t f ));

s

and, by Lemma 4, we have j−1 

∑

i=0

j i

  t f

 j−i  t f j Ls ds ψ i (x(t f ))ds + ψ j (x(t f )) = Ls ds + ψ (x(t f )) .

t

t

Thus M j (t, x) = V j (t, x; μ , ν ∗ ) for μ admissible and the lemma is proved.

2

Now consider the following equation: ∗

min {O μ ,ν M j+1 (t, x) + ( j + 1)M j (t, x)L1 (t, x, μ , ν ∗ )} = 0,

μ ∈UM

(22)

where M j+1 (t, x) is a suitably smooth solution to (22) and UM is a controller class which will be given later. Suppose that the moment that is desired to be minimized is the ( j + 1)-st moment. ¯ be the jth admissiTheorem 2 (Verfication Theorem). Let M j ∈ C1,2 p (Q) ∩ C(Q) ble moment cost function with an admissible class of control strategies, UM . If the ¯ satisfies function M j+1 ∈ C1,2 p (Q) ∩C(Q)   (23) min O k M j+1 (t, x) + ( j + 1)M j (t, x)L(t, x, μ , ν ∗ ) = 0, μ ∈UM

then M j+1 (t, x) ≤ V j+1(t, x; μ , ν ∗ ) for all μ ∈ UM and (t, x) ∈ Q. Furthermore if there is a μ ∗ such that   ∗ μ ∗ = arg min O μ ,ν M j+1 (t, x) + ( j + 1)M j (t, x)L1 (t, x, μ , ν ∗ ) (24) μ ∈UM

then M j+1 (t, x) = V j+1 (t, x; μ ∗ , ν ∗ ). Proof. The proof of this theorem follows that of Lemma 5 where the equality now becomes an inequality. 2

A Multiobjective Cumulant Control Problem

109

5.3 Third and Fourth Cumulant Cases Here we will examine the case of the third cumulant. ¯ be admissible first Theorem 3 (Verification Theorem). Let K1 , K2 ∈ C1,2 p (Q)∩C(Q) and second cumulant cost functions respectively. Also assume that K3 ∈ C1,2 p (Q) ∩ ¯ is a solution to the partial differential equation C(Q)  

  ∂ K1 ∂ K2 μ ,ν ∗  (t, x) σ (t, x)W (t)σ (t, x) (t, x) =0 min O K3 (t, x) + 3 μ ∈Um ∂x ∂x (25) ¯ that satisfies and the function P ∈ C1,2 p (Q) ∩C(Q)

∂P ∂P (t, x) + f  (t, x, μ ∗ , ν ) (t, x) min ν ∈WF ∂t ∂x   1 ∂ 2P (26) + tr σ (t, x)W (t)σ  (t, x) 2 (t, x) 2 ∂x  + L2 (t, x, μ ∗ , ν ) = 0 with P(t f , x f ) = ψ2 (x f ). If the strategies μ ∗ and ν ∗ are the minimizing arguments of (25) and (26), then the pair (μ ∗ , ν ∗ ) constitutes a Nash equilibrium solution and K3 (t, x) = Λ3 (t, x, μ ∗ , ν ∗ ) ≤ Λ3 (t, x, μ , ν ∗ ), P(t, x) = Etx {J2 (t, x, μ ∗ , ν ∗ )}. Proof. To start the proof, assume that the control’s Nash equilibrium solution has ¯ Then we have a minimal mean of the cost been played and P ∈ C1,2 p (Q) ∩ C(Q). problem for the disturbance w. Assume the disturbance plays the strategy ν (t, x(t)), which may or may not minimize (14). This yields Oμ

∗ ,ν

P(t, x) + L2 (t, x, μ ∗ , ν ) ≥ 0.

But by the Dynkin formula and (27) we have

 t f  ∗ P(t, x) = Etx −O μ ,ν P(s, x)ds + ψ2 (x f ) t

(27)

(28)

∗

≤ Etx {J2 (t, x, μ , ν )}, where Etx is as previously defined. Notice that if the disturbance plays a strategy ν ∗ that minimizes (14), we have P(t, x) = Etx {J2 (t, x, μ ∗ , ν ∗ )}, and thus if μ ∗ is the control’s Nash equilibrium solution, then Definition 1 is satisfied and ν ∗ is the disturbance’s Nash equilibrium strategy. For the second part of the proof, let the disturbance play its Nash equilibrium strategy ν ∗ . Recall Theorem 2. If M3 is an appropriate function and satisfies ∗

min O μ ,ν M3 (t, x) + 3M2(t, x)L1 (t, x; μ , ν ∗ ) = 0,

μ UM

110

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

then the minimizing argument of this function is the optimal control strategy. Furthermore, by definition the third moment cost function is related to the third cumulant cost function by M3 = K3 + 3K1 K2 + K13 . Using this, the previous partial differential equation becomes ∗

min O μ ,ν [K3 + 3K1 K2 + K13 ] + 3(K2 + K12)L1 = 0,

(29)

μ UM

where, again, the arguments have been suppressed for notational purposes. It will be shown that this Hamilton–Jacobi–Bellnlan equation reduces to that of (25). μ ,ν Let the backward evolution operator be given as O μ ,ν = O1 + O2 . Applying this decomposition to (29) gives  μ ,ν ∗

∗

min O μ ,ν [K3 ] + O1

μ UM

[3K1 K2 + K13 ] + O2[3K1 K2 + K13 ] + 3(K2 + K12 )L

= 0. (30)

By use of the chain rule  μ ,ν ∗

∗

min O μ ,ν [K3 ] + 3(K2 + K12 )O1

μ UM

μ ,ν ∗

[K1 ] + 3K1O1

[K2 ]

+O2 [3K1 K2 + K13 ] + 3(K2 + K12 )L Notice that

(31) = 0.

∗

O μ ,ν K1 (t, x) + L1 (t, x, μ , ν ∗ ) = 0,

(32)

where K1 is the admissible first cumulant cost function. Similarly, the second cumulant is given as K2 (t, x) = M2 (t, x) − M12 (t, x). Using this gives ∗

O μ ,ν [K2 (t, x) + K12 (t, x)] + 2K1(t, x)L1 (t, x, μ , ν ∗ ) =     ∂ K1 ∂ K1 μ ,ν ∗  (t, x) σ (t, x)W (t)σ (t, x) (t, x) = 0 K2 (t, x) + O ∂x ∂x which by substitution and reduction gives      ∂ K1  μ ,ν ∗  ∂ K1 − 3K1O2 [K2 ] min O [K3 ] + 3K1 σW σ μ UM ∂x ∂x −3(K2 + K12)O2 [K1 ] + O2[3K1 K2 + K13 ]

(33)

(34) = 0.

Next consider the last term in (34), O2 [3K1 K2 + K13 ]. By definition of O2 and taking the first partial, the term is

A Multiobjective Cumulant Control Problem

111

   ∂ K1 ∂ K2 1 3  ∂ 2 ∂ K1 3K2 + 3K1 + 3K1 O2 [3K1 K2 + K1 ] = tr σ W σ 2 ∂x ∂x ∂x ∂x      2  ∂ K2 ∂ K1  ∂ K1 1 = tr σ W σ  3 + 3K2 2 ∂x ∂x ∂ x2 

   2  ∂ K1 ∂ K2  ∂ K2 +3 + 3K1 ∂x ∂x ∂ x2     2  ∂ K1 ∂ K1  ∂ K1 + 6K1 + 3K12 ∂x ∂x ∂ x2 =3(K2 + K12 )O2 [K1 ] + 3K1 O2 [K2 ]  + 3K1

∂ K1 ∂x



σW σ





∂ K1 ∂x





∂ K1 +3 ∂x



σW σ





∂ K2 ∂x

 ,

which by substituting into (34) yields the desired HJB equation and the proof is complete. 2 With the third cumulant case complete, attention can now be turned to the fourth cumulant. Let K1 , K2 , K3 respectively be admissible first, second, and third cumulant cost functions. Thus, K1 , K2 , K3 satisfy ∗

O μ ,ν K1 (t, x) + L(t, x; μ , ν ∗ ) = 0     ∗ ∂ K1 ∂ K1 (t, x) σ (t, x)W (t)σ  (t, x) (t, x) = 0 O μ ,ν K2 (t, x) + ∂x ∂x     ∂ K1 ∂ K2 μ ,ν ∗  (t, x) σ (t, x)W (t)σ (t, x) (t, x) = 0. O K3 (t, x) + 3 ∂x ∂x

(35)

¯ be admissible Theorem 4 (Verification Theorem). Let K1 , K2 , K3 ∈ C1,2 p (Q) ∩C(Q) first, second, and third cumulant cost functions respectively. Also assume that K4 ∈ ¯ is a solution to the partial differential equation C1,2 p (Q) ∩C(Q)      ∂ K1 ∂ K3 μ ,ν ∗  (t, x) σ (t, x)W (t)σ (t, x) K4 (t, x) + 4 min O μ ∈Um ∂x ∂x (36)     ∂ K2 ∂ K2  (t, x) = 0; +3 σ (t, x)W (t)σ (t, x) ∂x ∂x then K4 (t, x) ≤ Λ4 (t, x; μ , ν ∗ ) for all μ ∈ UM and for (t, x) ∈ Q. Furthermore if the control law μ ∗ is such that it is the minimizing argument in (36), then K4 (t, x) = Λ4 (t, x; μ ∗ , ν ∗ ) and μ ∗ is the control’s equilibrium solution for the four cumulant, two player game.

112

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

¯ Recall (15), for Proof. Assume that K1 , K2 , K3 , K4 are all of class C1,2 p (Q) ∩ C(Q). j = 3, the fourth moment yields M4 = M3 K1 + 3M2 K2 + 3M1 K3 + K4 . Using previous results for K1 , K2 , K3 , the fourth moment may be expressed as K4 + 3K1 K3 + 6K12 K2 + ¯ M4 is also of class C1,2 ¯ K14 + 3K22 . Since K1 , K2 , K3 , K4 ∈ C1,2 p (Q) ∩C(Q), p (Q) ∩C(Q). Recall that in Theorem 2, if M4 satisfies  ∗

min

μ ∈UM

O μ ,ν M4 (t, x) + 4M3 (t, x)L1 (t, x, μ , ν ∗ )

= 0,

then M4 (t, x) ≤ V4 (t, x; μ , ν ∗ ). Considering this, substitute in the previous equation for M4 and M3 , which gives  ∗

O μ ,ν [K4 + 3K1 K3 + 6K12K2 + K14 + 3K22 ]

min

μ ∈UM

+4[K3 + 3K1 K2 + K13 ]L1

= 0,

where the arguments have been suppressed for notational purposes. Now by using μ ,ν ∗ the definitions of O1 and O2 , the HJB equation becomes  μ ,ν ∗

∗

O μ ,ν K4 + O1

min

μ ∈UM

[3K1 K3 + 6K12K2 + K14 + 3K22]

+ O2 [K4 + 3K1K3 + 6K12K2 + K14 + 3K22 ] + 4[K3 + 3K1K2 + K13 ]L1

= 0,

which through differentiation yields  min

μ ∈UM

μ ,ν ∗

∗

O μ ,ν K4 + 4K3 O1 μ ,ν ∗

+ 6K12O1

μ ,ν ∗

[K1 ] + 4K1O1 μ ,ν ∗

[K2 ] + 4K13O1

μ ,ν ∗

[K3 ] + 12K1K2 O1 μ ,ν ∗

[K1 ] + 6K2O1

[K2 ]

[K1 ]

+ O2 [3K1 K3 + 6K12K2 + K14 + 3K22] + 4[K3 + 3K1K2 + K13 ]L1

= 0.

Grouping like terms reduces it further to  min

μ ∈UM

μ ,ν ∗

∗

O μ ,ν K4 + 4(K3 + 3K1 K2 + K13 )O1 μ ,ν ∗

+ 6(K12 + K2 )O1

μ ,ν ∗

[K2 ] + 4K1O1

[K1 ]

[K3 ]

+ O2 [3K1 K3 + 6K12K2 + K14 + 3K22] + 4[K3 + 3K1K2 + K13 ]L1

= 0.

A Multiobjective Cumulant Control Problem

113

Using (35) and reducing terms, the equation becomes  min

μ ∈UM

∗

O μ ,ν K4 − 4(K3 + 3K1 K2 + K13 )O2 [K1 ]      ∂ K1  ∂ K1 − 6(K12 + K2 ) O2 [K2 ] + σW σ  ∂x ∂x      ∂ K1 ∂ K2 − 4K1 O2 [K3 ] + 3 σW σ  ∂x ∂x + O2 [3K1 K3 + 6K12 K2 + K14 + 3K22]

(37)

= 0.

Because O2 is linear, the expression can be determined term by term. The first term to consider is the one involving 4K1 K3 . Differentiating gives    ∂ ∂ K1 ∂ K3 1 4K3 + 4K1 O2 [4K1 K3 ] = tr σ W σ  2 ∂x ∂x ∂x     1 ∂ K1 ∂ K3 = tr σ W σ  4 2 ∂x ∂x     ∂ 2 K1 ∂ K3 ∂ K1  ∂ 2 K3 + 4K3 +4 + 4K1 ∂ x2 ∂x ∂x ∂ x2     ∂ K1  ∂ K3 =4K3 O2 [K1 ] + 4K1 O2 [K3 ] + 4 . σW σ ∂x ∂x Similarly the terms for 6K12 K2 and K14 can be computed as       ∂ K1 1 2  ∂ 2 ∂ K2 12K1 K2 + 6K1 O2 [6K1 K2 ] = tr σ W σ 2 ∂x ∂x ∂x        ∂ K1 ∂ K1 ∂ K1 ∂ K2  1 = tr σ W σ  12K2 + 12K1 2 ∂x ∂x ∂x ∂x   2   ∂ 2 K1 ∂ K2 + 6K12 + 12K1 + 12K1K2 ∂ x2 ∂ x2     ∂ K1  ∂ K1 =12K1 K2 O2 [K1 ] + 6K12O2 [K2 ] + 6 σW σ ∂x ∂x     ∂ K1 ∂ K2 + 12 σW σ ∂x ∂x 

and

∂ K2 ∂x



∂ K1 ∂x





114

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

O2 [K14 ]

    1  ∂ 3 ∂ K1 4K1 = tr σ W σ 2 ∂x ∂x      2   1 ∂ K1 ∂ K1  ∂ K1 = tr σ W σ  12K12 + 4K13 2 ∂x ∂x ∂ x2     ∂ K1 ∂ K1 . =4K13 O2 [K1 ] + 6K12 σW σ  ∂x ∂x

Finally the 3K22 term is expressed as     ∂ K2 1 2  ∂ 6K2 O2 [3K2 ] = tr σ W σ 2 ∂x ∂x       2  ∂ K ∂ K2  ∂ K2 1 2 = tr σ W σ  6 + 6K2 2 ∂x ∂x ∂ x2     ∂ K2 ∂ K2 . =6K2 O2 [K2 ] + 3 σW σ  ∂x ∂x By substituting the expressions into (37), we find  min

μ ∈UM

∗

O μ ,ν K4 − 4(K3 + 3K1 K2 + K13 )O2 [K1 ]      ∂ K1   ∂ K1 O2 [K2 ] + σW σ ∂x ∂x      ∂ K1 ∂ K2 + 4K3 O2 [K1 ] − 4K1 O2 [K3 ] + 3 σW σ  ∂x ∂x     ∂ K1  ∂ K3 + 12K1K2 O2 [K1 ] σW σ + 4K1 O2 [K3 ] + 4 ∂x ∂x         ∂ K1  ∂ K1 ∂ K1  ∂ K2 2 + 12 σW σ σW σ + 6K1 O2 [K2 ] + 6 ∂x ∂x ∂x ∂x     ∂ K1 ∂ K1 + 4K13 O2 [K1 ] + 6K12 σW σ  ∂x ∂x     ∂ K2 ∂ K 2 = 0; + 6K2 O2 [K2 ] + 3 σW σ  ∂x ∂x − 6(K12 + K2 )

and reducing this yields  min

μ ∈Um

O

μ ,ν ∗

   ∂ K1 ∂ K3  (t, x) σ (t, x)W (t)σ (t, x) (t, x) K4 (t, x) + 4 ∂x ∂x     ∂ K2 ∂ K2  (t, x) σ (t, x)W (t)σ (t, x) (t, x) = 0, +3 ∂x ∂x 

A Multiobjective Cumulant Control Problem

115

which is the HJB equation for the fourth cumulant. Note that the proof for the disturbance’s equilibrium strategy and the HJB-type equation is the same as in the third cumulant case. 2 5.4 Linear Quadratic Case Now we consider the case when the system given is linear and the costs are quadratic. The system is described by dx(t) =[A(t)x(t) + B(t)u(t) + D(t)w(t)]dt + E(t)d ξ (t) z1 (t) =H1 (t)x(t) + G1 (t)u(t) z2 (t) =H2 (t)x(t) + G2 (t)u(t), where x(t0 ) = x0 and z1 , z2 are the regulated outputs of the system. It also will be assumed that Hi Hi = Qi , Gi Hi = 0, and Gi Gi = Ri for i = 1, 2, where Qi is positive semidefinite and Ri is positive definite. Furthermore the costs will be assumed to be quadratic:  tf

J1 = t0 tf

J2 = t0

z1 (t)z1 (t)dt

(δ 2 w (t)w(t) − z2 (t)z2 (t))dt,

where Q1f = Q2f = 0. Before moving on, we will provide some definitions from [CZ01]. First consider the 2-norm of a function z(t) described by tf 2 ||z((t)||2,[t0 ,t f ] = E{||z(t)||2 }dt, t0

where ||z(t)||2 = z (t)z(t). Furthermore, the induced norm on the system Tzw will be defined as ||z||2,[t0 ,t f ] ||Tzw ||∞,[t0 ,t f ] = sup w ||w||2,[t0 ,t f ] for all w = 0 bounded power signals, that is one in which 2-norm of w exists. Notice that minimizing the performance index of the disturbance then imposes a constraint on the input-output properties of the disturbance w to the regulated output z2 . To see this consider that for the performance index E{J2 } ≥ 0 we have 

 t f 2   (δ w (t)w(t) − z2 (t)z2 (t))dt ≥ 0, E t0

but this is the same as tf t0

  E ||z2 (t)||2 dt ≤ δ 2

tf t0

  E ||w(t)||2 dt.

116

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

For all w = 0, we have

||z2 ||22,[t0 ,t f ] ||w||22,[t

≤ δ 2.

0 ,t f ]

So trying to maximize the mean of J2 is equivalent to sup w

||z2 ||2,[t0 ,t f ] ||w||2,[t0 ,t f ]

≤δ

and says that δ is a constraint on the H∞ norm of the system. The problem is illustrated in Figure 1 where G is the plant transfer function and Δ is a structured plant uncertainty. That is the interesting part of this problem. The cost cumulant control problem involves the first two cumulants, but also has the ability to incorporate some uncertainty into our control designing equations. Notice that if we let γi = 0 for i > 0, then we have the H2 /H∞ control problem. This suggests that the multiobjective cumulant control is a generalization of H2 /H∞ control. We now apply the previous results to the linear quadratic cost cumulant problem. Here we will assume that the value functions are of a quadratic nature, that is, K j (t, x) = x (t)K j (t)x(t) + k j (t) for k = 2, 3, 4. Recall that for j = 1 the admissible first cumulant cost function satisfies ∗

O μ ,ν K1 (t, x) + x (t)Q(t)x(t) + μ  (t, x(t))R(t)μ (t, x(t)) = 0, and for 1 < j < k we have j−1

2 j! Ks (t)σ (t, x)W (t)σ  (t, x)K j−s (t) = 0, s!( j − s)! s=1

∗

O μ ,ν K j (t, x) + ∑ and finally for j = k  min O

μ ∈UM

μ ,ν ∗

 2k!  Ks (t)σ (t, x)W (t)σ (t, x)Kk−s (t) = 0. Kk (t, x) + ∑ s=1 s!(k − s)! k−1

Δ



w

z2 -

ξ

- z1

G

y

u K



Fig. 1. Block diagram of System with Uncertainty.

A Multiobjective Cumulant Control Problem

117

Now with the assumption of the form of K j for j = 1, · · · , k, we have x K˙j x + k˙ j + (Ax + Bμ + Dν ∗ ) K j x + x K j (Ax + Bμ + Dν ∗ ) + x Qx + μ  Rμ = 0, and for 1 < j < k we have x K˙j x + k˙ j + (Ax + Bμ + Dν ∗ ) K j x + x K j (Ax + Bμ + Dν ∗ ) j−1

2 j! Ks σ W σ  K j−s = 0, s!( j − s)! s=1

+∑

and finally for j = k  min x K˙k x + k˙ k + (Ax + Bμ + Dν ∗ ) Kk x + x Kk (Ax + Bμ + Dν ∗ ) μ ∈UM

 2k! Ks σ W σ  Kk−s = 0. +∑ s=1 s!(k − s)! k−1

Using Lagrange multipliers and minimizing we obtain the optimal control law as  

μ ∗ (t, x) = −R−1 (t)B (t)

k

∑ γ j K j (t)x(t)

,

(38)

j=1

which gives Riccati equations 1 1 K˙1 + AK1 + K1A − 2 PDD K1 − 2 K1 DD P + Q − K1BR−1 B K1 δ δ k   − ∑ γs Ks BR−1 B K1 + K1 BR−1 B Ks = 0, s=2

(39) 1 1 K˙j + A K j + K j A − 2 PDD K j − 2 K j DD P δ δ k   − ∑ γs Ks BR−1 B K j + K j BR−1 B Ks 

s=2

(40)

  j! Ks EW E  K j−s + K j−sEW E  Ks = 0, s=1 s!( j − s)! j−1

+∑

1 P˙ + A P + PA − 2 PDD P − C2 C2 δ



k

∑ γs R−1 B Ks

D2 D2

s=1

k

∑ γs R−1B Ks

s=1

k   − ∑ γs Ks BR−1 B P + PBR−1B Ks = 0, s=1

where K1 (t f ) = K j (t f ) = P(t f ) = 0.

(41)

118

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

6 Wind Benchmark 6.1 Problem Statement The third generation benchmark problem for wind-excited buildings considers a 76story concrete building proposed for Melbourne, Australia. This problem is discussed in [YASW04]. The building model has been subjected to wind tunnel tests. The data from these tests have resulted in the wind forces for use in this benchmark problem. For control purposes there is an active tuned mass damper on the top floor of the building. Due to the large computational tasks that are involved for a 77 degree of freedom (DOF) building, a reduced order model is used. The evaluation model will be given by x˙ = Ax + Bu + Ew, (42) where x = [x¯ , x˙¯ ] . The quantity x¯ is a column vector given as the displacements of the 3rd, 6th, 10th, 13th, 16th, 20th, 23rd, 26th, 30th, 33rd, 36th, 40th, 43rd, 46th, 50th, 53rd, 56th, 60th, 63rd, 66th, 70th, 73rd, and 76th floors, in that order, as well as xm which is the displacement of the mass damper. The matrices A, B, and E are of the size 48 × 48, 48 × 1, and 48 × 77 respectively. Along with the evaluation model there are regulated output z and output y equations given by z = Cz x + Dzu + Fzw y = Cy x + Dy u + Fyw + v,

(43)

where x˜ = [x1 , x30 , x50 , x55 , x60 , x65 , x70 , x75 , x76 , xm ] , z = [x˜ , x˜˙ , x˜¨ ] , and y = [x˜˙ , x˜¨ ] . The matrices Cz , Dz , Fz , Cy , Dy , and Fy are appropriately dimensioned. The elevation view of this building is shown in Figure 2. To evaluate the performance of each control method there are twelve performance criteria. The criteria are based on the results of the simulation of the evaluation model with control and 900 sec of wind data. The first criterion measures the effect of the controller on the maximum floor acceleration. That is, the criterion is given by J1 =

max(σx¨1 , σx¨30 , σx¨50 , σx¨55 , σx¨60 , σx¨65 , σx¨70 , σx¨75 ) , σx¨75o

where σxi is the root-mean-square (RMS) acceleration of the ith floor and σx¨75o = 9.142 cm/s is the RMS uncontrolled acceleration of the 75th floor. The second performance criterion is given by σx¨ 1 J2 = ∑ i 6 i σx¨io for i = 50, 55, 60, 65, 70, 75 and where σx¨io is the uncontrolled RMS acceleration of the ith floor. These two performance criteria have not included the top floor, floor 76.

A Multiobjective Cumulant Control Problem

119

Fig. 2. Elevation view for wind benchmark problem.

In the next two performance criteria this floor is included. They are given by

σx76 σx76o σx 1 J4 = ∑ i 7 i σxio J3 =

for i = 50, 55, 60, 65, 70, 75, 76 and σx76o = 10.137 cm, the uncontrolled displacement of the 76th floor. The previous performance criteria dealt with the performance of the building. While this is the main objective, one cannot focus on this without some constraints on the control and actuator. The actuator’s physical constraints are that the RMS control force, σu , must not be greater than 100 kN and that the RMS actuator stroke, σxm , must not be greater than 30 cm. While these constraints are physical constraints, there are also criteria designed to determine the control effort. These criteria are given by

120

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

σxm σx76o  T 1/2 1 J6 = (x˙m (t)u(t))2 dt , T 0 J5 =

where T is the total time of integration. With the RMS performance taken into account, we now give the performance criteria for the peak response. The first four criteria are given by max(x¨ p1 , x¨ p30 , x¨ p50 , x¨ p55 , x¨ p60 , x¨ p65 , x¨ p70 , x¨ p75 ) x¨ p75o x¨ pi 1 J8 = ∑ 6 i x¨ pio x p76 J9 = x p76o xp j 1 J10 = ∑ 7 j x p jo J7 =

for i = 50, 55, 60, 65, 70, 75, j = i, 76. Also x pi , x pio are the controlled and uncontrolled peak displacements of the ith floor respectively, and x¨ pi , x¨ pio are respectively the controlled and uncontrolled peak accelerations of the ith floor. Similar to the RMS case, the actuator constraints are maxt |u(t)| ≤ 300 kN, maxt |xm (t)| ≤ 95 cm. Furthermore the control effort is measured by J11 =

x pm x p76o

J12 = max |x˙m (t)u(t)|, t

where x pm is the peak actuator displacement. One then wants to design a controller and to test that controller with the preceding performance criteria. In the design of the controller there are several constraints. One is that the designer may choose only 6 outputs for the design. Thus one must choose yr from 6 elements in y, so that yr is a vector of at most dimension 6. Furthermore the control compensator order must not exceed 12. 6.2 Wind Benchmark Results With the control algorithm now in place, this method of design is applied to the third generation benchmark for wind-excited structures. Following [YASW04], a 12-state reduced order model is used for the control design. This design model is x˙r =Ar xr + Br u + Er ξ yr =Cyr xr + Dyr u + Fyr ξ zr =Czr xr + Dzr u + Fzr ξ + vr ,

(44)

A Multiobjective Cumulant Control Problem

121

where xr = [x16 , x30 , x46 , x60 , x76 , xm , x˙16 , x˙30 , x˙46 , x˙60 , x˙76 , x˙m ] , zr is the same as the z for the evaluation model, and yr = [x¨50 , x¨76 , x¨m ] . Also, the disturbance ξ is the wind excitation and vr is the sensor noise, and furthermore, the two are uncorrelated. From the baseline linear-quadratic-Gaussian (LQG) controller designed in [YASW04], a cost function tf J = ((Cyr xr + Dyr u) Q(Cyr xr + Dyr u) + uRu)dt (45) 0

will be used for J1 . From (45) we have     HCyr HDyr xr + u z1 = 0 R as the regulated output for the control, where H  H = Q. To help account for some uncertainty, we will add the disturbance w as shown in Figure 3. The weighting function is given by Wz2 = 2.14 × 10−4I. The design model will now be given as x˙r = Ar xr + Br u + Dr w + Er ξ , where Dr is a 12 × 12 matrix with the first six columns equal to that of Ar , while the last six columns are zero. The multiobjective control methodology presented in this chapter has then been simulated using the benchmark problem. To help assess this control paradigm, it was compared with two other control designs. The first control design was the baseline LQG. The second was the 2CC or MCV discussed in [PSL02-Q]-[PJSSL04], [SWSL00]. For the MCV design, the parameter γ was set to be 8 × 10−8. The simulated results are displayed in Table 1. First consider the RMS performance criteria J1 –J6 of the wind benchmark problem. Notice that for the MCV and multiobjective controllers, the criteria that measure the performance of the building, J1 –J4 , shown substantial improvement from the LQG case. For the MCV case, there is 11.7%, 11.0%, 4.3%, and 4.3% reduction in these cases, and a reduction of 2.2%, 2.1%, 1.0%, and 1.0% for the multibjective case. In particular, for J1 and J2 there is improvement. There are also two criteria that deal with control effort, J5 and J6 . It would be expected that with the decrease in

z2 Δ  Wz2  - K

w ξ

-

P

-

u Fig. 3. Block diagram with uncertainty.

122

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won Table 1. Benchmark results (Δ K = 0). J1 J2 J3 J4 J5 J6 J7 J8 J9 J10 J11 J12 σxm σu maxt |xm | maxt |u|

LQG 0.369 0.417 0.578 0.580 2.271 11.99 0.381 0.432 0.717 0.725 2.299 71.87 23.03 34.07 74.27 118.2

MCV 0.326 0.371 0.553 0.555 2.720 19.96 0.346 0.419 0.696 0.705 2.756 122.3 27.57 50.26 89.01 194.1

MCC 0.361 0.408 0.572 0.574 2.310 12.82 0.363 0.421 0.705 0.713 2.279 77.62 23.42 38.41 73.61 143.9

J1 –J4 , the results for J5 and J6 would be larger. While this is the case, it can be seen from the σu and σxm results that the control effort is still within its bounds. We shall now examine the wind benchmark’s peak performance criteria, J7 – J12 . Similarly to the RMS criteria, the MCV and multiobjective control methods show improvement in the building performance criteria, J7 –J10. The results show a decrease of 9.2%, 3.0%, 2..9%, and 2.8% respectively for J7 –J10 , in the MCV case. For the multiobjective case, we have a decrease of 4.7%, 2.5%, 1.7%, and 1.7% for J7 –J10 respectively. Also, we see that the results for the control effort for these two control paradigms are larger than the J11 and J12 of the LQG case. As before, even though there is more control effort being used, it does not exceed the bounds set in the wind benchmark problem. We have compared the MCV and multiobjective control results with the LQG results, but what about the differences between the results from the MCV and multiobjective control methods? It should be noted that the multiobjective method presented in this paper is an extension of the MCV control paradigm. It can be seen that the MCV control on the wind benchmark problem performs better than the multiobjective method. This can be a result of the multiobjective method being more robust. Since it is more robust in its design, some performance will suffer. This added robustness can be seen in Table 2. This table shows what happens when the stiffness matrix is changed by −15% and 15%. While both the MCV and MCC control methods perform well for the case of Δ K = 15%, this is not true for the case of −15%. The MCV control results show that it performs well, but in doing so greatly exceeds the actuator constraint. Recall that the peak actuator stroke must be within 95 cm. In the MCV case it is not. The multiobjective control methodology however shows a 5.4%, 5.3%, 2.8%, and 2.7% improvement over LQG for J1 –J4 and similarly a 6.8% and 3.5% improvement for J7 and J8 . Actually in this case we can

A Multiobjective Cumulant Control Problem

123

Table 2. Benchmark results.

Δ K = −15% LQG 0.387 0.438 0.711 0.712 2.709 16.61 0.488 0.539 0.770 0.779 2.836 118.3 27.46, 44.32 91.60 164.3

J1 J2 J3 J4 J5 J6 J7 J8 J9 J10 J11 J12 σxm σu maxt |xm | maxt |u|

MCV 0.339 0.387 0.679 0.681 3.299 27.26 0.425 0.499 0.724 0.733 3.326 199.3 33.44 ,64.27 107.4 235.3

MCC 0.366 0.415 0.691 0.693 2.714 17.08 0.455 0.520 0.785 0.795 2.938 129.6 27.52 48.50 94.89 183.1

Δ K = 15% LQG MCV 0.365 0.332 0.409 0.376 0.487 0.472 0.489 0.474 1.812 2.252 8.463 15.44 0.411 0.355 0.443 0.434 0.607 0.625 0.614 0.633 1.852 2.254 52.68 102.4 18.37 22.83 28.29 43.63 59.83 72.81 105.6 174.4

MCC 0.359 0.406 0.485 0.487 1.899 9.949 0.398 0.443 0.614 0.622 1.894 66.30 19.25 33.35 61.17 133.4

see that the MCC control method performs better than it did for Δ K = 0. Moreover, the MCC results show that it also satisfies the actuator constraints.

7 Conclusion In this chapter, we have extended existing cumulant game theory to include higher order cumulants. Sufficient conditions for a type of equilibrium solution were determined. For a linear system and quadratic cost case, a multiobjective cumulant control method was developed, in which the control was to minimize higher order cumulants, while the H∞ norm of the system was constrained. This control paradigm was then applied to the third generation benchmark for tall buildings subjected to high wind and was compared with other control methods.

References [BO99] [BH89]

[CZ01] [DS05]

T. Basar, G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd ed., SIAM, Philadelphia, 1999. D. S. Bernstein, W. M. Hassad, LQG Control with an H∞ Performance Bound: A Riccati Equation Approach, IEEE Transactions on Automatic Control, vol. 34, no. 3, pp. 293–305, 1989. X. Chen, K. Zhou, Multiobjective H2 /H∞ Control Design, SIAM Journal of Control and Optimization, vol. 40, no. 2, pp. 628–660, 2001. R. W. Diersing, M. K. Sain, The Third Generation Wind Structural Benchmark: A Nash Cumulant Robust Approach, Proceedings American Control Conference, pp. 3078–3083, Portland, Oregon, June 8–10, 2005.

124 [DS07]

Ronald W. Diersing, Michael K. Sain, and Chang-Hee Won

R. W. Diersing, M. K. Sain, Multi-cumulant, Multi-objective Structural Control: A Circuit Analog, Analog Integrated Circuits and Signal Processing, 56:1/2 August, 2008. [DZB89] J. Doyle, K. Zhou, B. Bodenheimer, Optimal Control with Mixed H2 and H∞ Performance Objectives, Proceedings American Control Conference, pp. 2065–2070, 1989. [FR75] W. H. Fleming, R. W. Rishel, Deterministic and Stochastic Optimal Control, Springer-Verlag, New York, 1975. [Fle77] W. H. Fleming, Functions of Several Variables, Springer-Verlag, New York, 1977. [LAH94] D. J. N. Limebeer, B. D. O. Anderson, D. Hendel, A Nash Game Approach to Mixed H2 /H∞ Control, IEEE Transactions on Automatic Control, vol. 39, no. 1, pp. 69–82, Jan. 1994. [PSL02a] K. D. Pham, M. K. Sain, S. R. Liberty, Finite Horizon Full-State Feedback kCC Control in Civil Structures Protection, Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences, Proceedings of a Workshop held in Lawrence, Kansas, Edited by B. Pasik-Duncan, Springer-Verlag, BerlinHeidelberg, Germany, vol. 280, pp. 369–383, September 2002. [PSL02b] K. D. Pham, M. K. Sain, S. R. Liberty, Cost Cumulant Control: State-Feedback, Finite-Horizon Paradigm with Application to Seismic Protection, Special Issue of Journal of Optimization Theory and Applications, Edited by A. Miele, Kluwer Academic/Plenum Publishers, New York, vol. 115, no. 3, pp. 685–710, December 2002. [PJSSL04] K. D. Pham, G. Jin, M. K. Sain, B. F. Spencer, Jr., S. R. Liberty, Generalized LQG Techniques for the Wind Benchmark Problem, Special Issue of ASCE Journal of Engineering Mechanics on the Structural Control Benchmark Problem, vol. 130, no. 4, April 2004. [Sai65] M. K. Sain, Relative Costs of Mean and Variance Control for a Class of Linear Noisy Systems, Proceedings 3rd Annual Allerton Conference on Circuit and System Theory, pp. 121–129, 1965. [Sai66] M. K. Sain, A Sufficient Condition for the Minimum Variance Control of Markov Processes, Proceedings 4th Annual Allerton Conference on Circuit and System Theory, pp. 593–599, 1966. [Sai67] M. K. Sain, Performance Moment Recursions, with Application to Equalizer Control Laws, Proceedings 5th Annual Allerton Conference on Circuit and System Theory, pp. 327–336, 1967. [SWS92] M. K. Sain, C. H. Won, B. F. Spencer, Jr., Cumulant Minimization and Robust Control, Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences, Ed. T. E. Duncan and B. Pasik-Duncan, Springer-Verlag, New York, pp. 411–425, 1992. [SWSL00] M. K. Sain, C. H. Won, B. F. Spencer Jr., S. R. Liberty, Cumulants and Risk Sensitive Control: A Cost Mean and Variance Theory with Applications to Seismic Protection of Structures, Proceedings 34th Conference on Decision and Control, Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games, vol. 5, J. A. Filor, V. Gaisgory, K. Mizukami (Eds), Birkhauser, Boston, 2000. [Won05] C. H. Won, Nonlinear n-th Cost Cumulant Control and Hamilton-Jacobi-Bellman Equations for Markov Diffusion Process, Proceedings 44th IEEE Conference on Decision and Control, pp. 4524–4529, Seville, Spain, December 2005.

A Multiobjective Cumulant Control Problem

125

[YASW04] J. N. Yang, A. K. Agrawal, B. Samali, J. C. Wu, Benchmark Problem for Response Control of Wind-Excited Tall Buildings, Journal of Engineering Mechanics, vol. 130, no. 4, pp. 437–446, 2004. [ZGBD94] K. Zhou, K. Glover, B. Bodenheimer, J. Doyle, Mixed H2 and H∞ Performance Objectives I: Robust Performance Analysis, IEEE Transactions on Automatic Control, vol. 39, no. 8, pp. 1564–1574, 1994.

Part II

Algebraic Systems Theory

Systems over a Semiring: Realization and Decoupling Ying Shang1 and Michael K. Sain2 1 2

Department of Electrical and Computer Engineering at the Southern Illinois University, Edwardsville, IL 62026, USA. [email protected] Department of Electrical Engineering, University of Notre Dame, Notre Dame, IN 46556, USA.† [email protected]

Summary. Systems over a semiring can be understood as systems evolving with variables without negatives, for example, genetic regulatory networks, communication networks, and manufacturing systems. This paper generalizes the original system theory invented by R. E. Kalman for traditional linear systems. Kalman constructed the “state” module, the input/output map, and a canonical realization in terms of reachability and observability. The advantage of this theory is that the frequency domain and the state variable approaches are merged into a single framework. A remarkable new feature of this system theory is that the realization process displays distinctly different characters when proceeding from the observability point of view than when proceeding from the reachability point of view. This new theory, moreover, provides computational methods for the crucial invariant sets in the disturbance decoupling problem, which is a standard problem in geometric control originated by W. M. Wonham. Along with the theoretical results, queueing networks are used to illustrate the main results.

1 Introduction Traditional system theory focuses on linear time-invariant systems whose coefficients belong to a field. Recent applications in communication networks [LT01], genetic regulatory networks [Dj02], and queueing systems [BCOQ92] require a new system theory for linear time-invariant systems with coefficients in a semiring. A semiring is understood as a set of objects without inverses with respect to the corresponding operations, for example, the max-plus algebra [BCOQ92], the min-plus algebra [LT01], and the Boolean semiring [Gol99]. Intuitively, systems over a semiring are not equipped with “additive inverses.” Nowadays, researchers still only have a basic understanding of systems over some special semirings, for example, discreteevent systems over the max-plus algebra [CGQ99] and network calculus in the minplus algebra [LT01]. Systems over a field evolve with variables taking values in modules. Systems over a semiring, on the other hand, evolve with variables taking values in semimodules. The fundamental differences between modules and semimodules slow down † Professor

Sain’s work is supported by the Frank M. Freimann Chair in Electrical Engi-

neering. C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 6, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

130

Ying Shang and Michael K. Sain

the progress of a well-established system theory for systems over a semiring. The first difference between modules and semimodules is that semimodules do not have inverses corresponding to the corresponding binary operations. Due to the lack of negatives in the state variables, the transfer function for a system over a semiring is represented by power series instead of matrix inverses. Another difference is that the kernel equivalence and the morphism equivalence are not the same any more for morphisms between semimodules. This difference results in a new discovery in Kalman’s realization theory, where the realization process displays distinctly different characters when proceeding from the observability point of view than when proceeding from the reachability point of view. Furthermore, there are two different images for a morphism between two semimodules, namely, image and proper image. The proper image of a morphism is the set of states which are mapped from the domain of the morphism. The image is a subset of the codomain of the morphism and if each state in the image operates with any element in the proper image, then the result remains in the proper image. The existence of two different images leads to two different exact sequences. Proper choices of exact sequences provide computational methods for the crucial invariant sets, which can be used in the solvability conditions of the disturbance decoupling problem (DDP), which is a standard geometric control problem originated by W. M. Wonham [Won79]. There are three main contributions of this chapter. First of all, Kalman’s realization theory is generalized to systems over a semiring. Second, a commutative diagram is established using exact sequences. This diagram not only establishes a connection between the geometric control method and the frequency domain method, but also provides types of calculation for different invariant sets in the kernel of the output map. Moreover, these invariant sets are used to establish necessary and sufficient solvability conditions for the DDP. The remainder of this chapter is organized as follows. Sect. 2 presents mathematical preliminaries needed in this paper. Sect. 3 defines linear systems over a semiring. Sect. 4 presents pole and zero semimodules for a given transfer function and the extension of Kalman’s realization diagram to linear systems over a semiring. Sect. 5 introduces different invariant sub-semimodules and the calculation methods using a commutative diagram. Sect. 6 presents the necessary and sufficient solvability conditions for the DDP. Sect. 7 uses queueing networks to illustrate our main results. Sect. 8 concludes the chapter. Because this chapter is a summary of the results in authors’ previous work, all proofs are omitted.

2 Mathematical Preliminaries A semigroup (S, ) is a set S together with a binary operation  : S × S → S which is associative. A monoid (M, , eM ) is a semigroup (M, ) with the unit element eM for , i.e. eM  x = x  eM = x for all x ∈ M. A group (G, , eG ) is a monoid which has inverses with respect to the operation , for each element in G. An Abelian group is a group with a commutative operation . For generality, symbols are used to represent the operations, instead of the traditional addition and multiplication.

Systems over a Semiring: Realization and Decoupling

131

A semiring (R, , eR , , 1R ) is a set R with two binary operations  and , such that (R, , eR ) is a monoid in which  is commutative, (R, , 1R ) is a monoid,  is distributive on both sides over , and r  eR = eR = eR  r, for all r ∈ R. Let (R, , eR , , 1R ) be a semiring, and (M, M , eM ) be a commutative monoid. M is called a left R-semimodule if there exists a map μ : R × M → M, denoted by μ (r, m) = rm, for all r ∈ R and m ∈ M, such that for any r, r1 , r2 ∈ R and m, m1 , m2 ∈ M, the following conditions are satisfied: 1. 2. 3. 4. 5.

r(m1 M m2 ) = rm1 M rm2 ; (r1  r2 )m = r1 m M r2 m; r1 (r2 m) = (r1  r2 )m; 1R m = m; Operations with unit elements: reM = eM = eR m.

This paper denotes the unit semimodule as e. A sub-semimodule K of M is subtractive if, for k ∈ K and m ∈ M, kM m ∈ K implies m ∈ K. A morphism f : M → N from an R-semimodule (M, M , eM ) to another R-semimodule N = (N, N , eN ) is a map satisfying f (m1 M m2 ) = f (m1 ) N f (m2 ) and f (rm) = r f (m), for all m, m1 , m2 ∈ M and r ∈ R. The kernel of a morphism f : M → N is defined as Ker f = {x ∈ M| f (x) = eN }.

(1)

There are two different images for an R-semimodule morphism f : M → N. The proper image is defined to be the set of all values f (m), for m ∈ M, f (M) = {n ∈ N|n = f (m), m ∈ M}.

(2)

The image of f is defined as Im f = {n ∈ N|n N f (m) = f (m ) for some m, m ∈ M}.

(3)

If the two images coincide, then the morphism f is called image regular or iregular. The two different images coincide for morphisms between R-modules. A f

g

sequence A − →B− → C of semimodules is called exact if Im f = Ker g. The sequence is called proper exact if f (A) = Ker g. In this case, the morphism f is i-regular, i.e., f (A) = Im f . The sequence f

g

e −−−−→ A −−−−→ B −−−−→ C −−−−→ e is called a short exact sequence (or proper exact sequence) if it is exact (or proper exact) at each semimodule. Given an R-semimodule morphism f : M → N, two elements m and m in M are equivalent with respect to the morphism equivalence relation ≡ f , i.e. m ≡ f m , if and only if f (m) = f (m ). The Bourne equivalence relation is introduced in ( [Gol99], p. 164) for an R-semimodule. If K is a sub-semimodule of an R-semimodule M, then the Bourne relation is defined by setting m ≡K m if and only if there exist two elements k and k of K such that m M k = m M k . The factor semimodule M/ ≡K

132

Ying Shang and Michael K. Sain

induced by ≡K is also written as M/K. If K is equal to the kernel of an R-semimodule morphism f : M → N, then two elements are kernel equivalent, i.e., m ≡Ker f m if and only if there exist two elements k, k in Ker f , such that m M k = m M k . Applying f on both sides, we obtain that f (m) = f (m ), i.e. m ≡ f m . Hence the kernel equivalence relation ≡Ker f and the morphism equivalence relation ≡ f satisfy the partial order ≤, i.e. ≡Ker f ≤ ≡ f . If an R-semimodule morphism f : M → N satisfies ≡ f ≤ ≡Ker f , then f is called a steady or k-regular morphism between semimodules. A module morphism f : M → N is always steady because the kernel equivalence coincides with the morphism equivalence.

3 Systems over a Semiring A system over a semiring R is defined in the following form: x(k + 1) = Ax(k) 2 Bu(k), y(k) = Cx(k)  Du(k),

(4)

where the three free finitely generated R-semimodules (X , 2, eX ), (U, , eU ), and (Y, , eY ) are the state semimodule, the input semimodule, and the output semimodule, respectively. The R-semimodule morphisms between them are denoted by A : X → X, B : U → X, C : X → Y , and D : U → Y . Let R(z) denote the set of formal Laurent series in z−1 , with coefficients in R, with finite left support, and having the property that, for any element a(z) ∈ R(z), there exists an element r(z) in R[z], such that r(z)a(z) ∈ R[z]. In like manner, let X (z) denote the set of formal Laurent series in z−1 , with coefficients in X , with finite left support, and having the property that, for any element x(z) in X (z), there exists an element r(z) in R[z] such that r(z)x(z) ∈ Ω X , which is a polynomial semiring with coefficients in X. U(z) and Y (z) are defined similarly to X (z). The transfer function G(z) : U(z) → Y (z) of this system of the form (4) is G(z) = CBz−1  CABz−2  CA2 Bz−3  · · ·  D.

(5)

The transfer function G(z), as defined in Eq. (5), is of course in a natural way an R(z)-morphism from the R(z)-semimodule U(z) to the R(z)-semimodule Y (z). The derivation of the transfer function representation uses power series instead of matrix inverses due to the lack of inverses.

4 Generalization of Kalman’s Realization Theory This section generalizes the Kalman realization diagram [KFA69] to systems over a semiring. Without loss of generality, assume D = 0 for a given transfer function G(z) : U(z) → Y (z), G(z) = CBz−1  CABz−2  CA2 Bz−3  · · · .

Systems over a Semiring: Realization and Decoupling

133

The Kalman input semimodule contains sequences of input signals starting at a finite negative time and continuing up to the time zero. The Kalman output semimodule contains sequences of output signals from the time instant 1 into the future. Hence, the Kalman input semimodule is defined as the set of polynomial inputs, Ω U = U[z]. The Kalman output semimodule is defined as the set of strictly proper outputs, denoted as Γ Y = Y (z)/Ω Y . The Kalman input/output map G# (z) from the input semimodule Ω U to the output semimodule Γ Y can be constructed from the commutative diagram shown in Figure 1, where i is the inclusion and p is the projection. Using the standard construction in automata theory [Kal65], the states of a system can be viewed as equivalence classes in the input semimodule induced by the transfer function. Considering Kalman’s input/output map G# (z) : Ω U → Γ Y , the commutative diagram is obtained as shown in Figure 2. In this diagram, The pole semimodule or the state semimodule of output type is defined by XO (G) =

G(Ω U) . G(Ω U) ∩ Ω Y

(6)

XO (G) is actually the proper image of G# (z). Notice that the pole semimodule is a quotient semimodule induced by the Bourne relation. There are two other equivalence relations introduced in the semimodule literature [Taka81], namely the Takahashi relation and the Iizuka relation. The Bourne relation yields better properties in short exact sequences and Kalman’s realization diagram, so we choose the Bourne relation in the study of systems over a semiring. The pole semimodule or the state

U (z )

G(z )

Y (z ) p

i

G # (z )

ΩU

ΓY

Fig. 1. Kalman input/output map G# (z). G # (z )

ΩU

~ B G # (z )

ΓY

~ C ΩU G −1 (ΩY ) I ΩU

Id

G (ΩU ) G (ΩU ) I ΩY

Fig. 2. The state semimodules of G# (z).

134

Ying Shang and Michael K. Sain

semimodule of input type is defined by XI (G) =

ΩU . G−1 (Ω Y ) ∩ Ω U

(7)

XI (G) is actually Ω U/Ker G# , because Ker G# = G−1 (Ω Y ) ∩ Ω U. The mappings B# : Ω U → X and C# : X → Γ Y are defined by # · u) = ABu; B(z # = Cxz−1 Y (z) C(Ax)z−2 Y (z) C(A2 x)z−3 · · · mod Ω Y. C(x)

(8) (9)

The mapping Id is an identity map and Y (z) is the operation on Y (z). The pole semimodules XI (G) and XO (G) can have R[z]-semimodule structure if, for any polynomial r(z) ∈ R[z], the scalar multiplication is defined by the action r(z)x = r(A)x. In the module case, pole semimodules of input type and output type have been used by the preference of the researchers, because XI (G) is isomorphic to XO (G). However, this is not the case for systems over a semiring. There exists an R[z]semiisomorphism between XI (G) and XO (G), that is a unit kernel R[z]-epimorphism. Lemma 1. [Sh06] Given a transfer function G(z) : U(z) → Y (z) and the pole semimodules of input and output type as shown in Eq. (6) and Eq. (7), there exists an R[z]-semimodule semiisomorphism G(z), that is a unit kernel R[z]-semimodule epimorphism, from XI (G) to XO (G). The semiisomorphism becomes an isomorphism if and only if G# (z) is steady. Therefore, the diagram in Figure 2 can be modified using an exact sequence (see Figure 3.) Kalman’s realization theory characterizes a canonical realization in [KFA69] by the property that is reachable from the unit element and observable with respect to the unit input. A realization (A, B,C) of a transfer function G(z) is canonical if and only if it is both reachable from the unit element and observable with respect to the unit input. In other words, if a transfer function can be factored through G # (z )

ΩU

~ B G # (z )

e

ΓY

~ C

ΩU G −1 (ΩY ) I ΩU

Id

G (z ) G (ΩU ) G (ΩU ) I ΩY

e

Fig. 3. The modification of Fig. 2.

Systems over a Semiring: Realization and Decoupling

135

X by an onto map g and a one-to-one map h, then X is a canonical realization for the given transfer function. If g is an onto map, but h is not a one-to-one map, then X is called a reachable realization. On the other hand, if h is a one-to-one map, but g is an onto map, then X is called an observable realization. From the Kalman’s realization diagram, the pole semimodule of output type XO (G) is a canonical realization of the Kalman input/output map, because G# (z) is an onto map from Ω U to XO (G) and Id is a one-to-one map. The pole semimodule of input type XI (G) is a reachable realization of G# (z), but not observable, because B# is onto but C# is not necessarily one-to-one. Using Lemma 1, the pole semimodule of input type becomes a controllable and observable realization if and only if Kalman’s input/output map is steady.

5 Controlled Invariance This section generalizes different kinds of invariant spaces [CP98, Won79] to systems over a semiring, including (A, B)-invariant sub-semimodules, (A, B)-invariant sub-semimodules of feedback type, controllability sub-semimodules, and precontrollability sub-semimodules. Relationships between zero semimodules and these invariant sub-semimodules are established. These connections can provide computational methods for the invariant semimodules in the solvability conditions of the DDP. We are given a system over a semiring R described by the following difference equation: x(k + 1) = A x(k) 2 B u(k), y(k) = C x(k),

(10)

where the three free finitely generated R-semimodules, (X , 2, eX ), (U, , eU ), and (Y, , eY ), are the state semimodule, the input semimodule, and the output semimodule, respectively, and A : X → X , B : U → X, and C : X → Y are R-semimodule morphisms. 5.1 (A,B)-Invariant Sub-Semimodules Given a linear system of the form (10) over a semiring R, a sub-semimodule V of the state semimodule X is called • (A,B)-invariant, or controlled invariant, if and only if, for all x0 ∈ V , there exists a sequence of control inputs, u = {u1 , u2 , · · · }, such that every component in the state trajectory produced by this input, x(x0 ; u) = {x0 , x1 , · · · }, remains inside of V. • (A,B)-invariant of feedback type if and only if there exists a state feedback F : X → U such that (A 2 BF)V ⊂ V , where F is called a friend of V . Unlike the case of systems over a field, (A, B)-invariant sub-semimodules of feedback type are not identical to (A, B)-invariant sub-semimodules for systems over a

136

Ying Shang and Michael K. Sain

semiring. In the module case, a submodule V of the state module X is (A, B)-invariant if and only if AV ⊂ V 2 Im B. This condition does not hold for the semimodule case because there is no subtraction. A modified condition is stated in Lemma 3. Lemma 2. A sub-semimodule V of the state semimodule X is an (A, B)-invariant " X B, where sub-semimodule if and only if AV ⊂ V 2 " X B  {x ∈ X |∃b ∈ B, s.t. x 2 b ∈ V }, V2 and B  B(U). Lemma 3. V is an (A, B)-invariant sub-semimodule of a sub-semimodule K of X if and only if " K B), V = V ∩ A−1 (V 2

(11)

where A−1 is the set inverse map of A : X → X. The family of the controlled invariant sub-semimodules is closed under the operation 2. The set of controlled invariant sub-semimodules in a sub-semimodule K of X is a upper semilattice relative to sub-semimodule inclusion ⊂ and operation 2. Therefore, there exists the supremal element V ∗ in the family of controlled invariant sub-semimodules a sub-semimodule K in X and it can be computed by the following algorithm. Theorem 1. Let {Vk }k≥0 be the family of sub-semimodules defined recursively by V0 = K " K B). Vk+1 = Vk ∩ A−1 (Vk 2

(12)

If there exists ∩k∈NVk , then any (A, B)-invariant sub-semimodule of K is contained in ∩k∈NVk , namely the supremal controlled invariant sub-semimodule V ∗ is also contained in ∩k∈NVk . Moreover, if the algorithm in Eq. (12) terminates in r steps, then V ∗ = Vr . 5.2 Controllability and Pre-Controllability Sub-Semimodules Given a linear system of the form (10) over a semiring, a sub-semimodule R of the state semimodule X is a controllable sub-semimodule of the pair (A, B) if R = A|B = B 2 AB · · · 2 An B 2 · · · = A∗ B, def

where B = B(U). If there exist maps F : X → U and G : X → U such that R = (A 2 BF)|BG(X ) , where

(13)

Systems over a Semiring: Realization and Decoupling

137

(A 2 BF)|BG(X) = BG(X)2(A2BF)BG(X )2(A2BF)2 BG(X ) · · · , then R is called a controllability sub-semimodule. A controllability sub-semimodule R is obviously an (A, B)-invariant subsemimodule of feedback type. Given a linear system of the form (10) and a subsemimodule K of the state semimodule X , a family of sub-semimodules is defined as SK = {S ⊂ X|S = K ∩ (AS 2 B)}.

(14)

The following proposition states that there exists a least element of the family, which is a component in S and is contained in any member of S. Proposition 1. Let {Sk }k≤0 be the family of sub-semimodules defined recursively by S0 = {e} Sk = K ∩ (ASk−1 2 B).

(15)

If there exists a well-defined ∪k∈N Sk , then it is the least element S∗ (K ) of the family in Eq. (14). Definition 1. A sub-semimodule R of the state semimodule X is called a precontrollability sub-semimodule if R is (A, B)-invariant and R = S∗ (R), which is the least element in the family defined by Eq. (14) for K = R. Pre-controllability sub-semimodules are generalized from pre-controllability submodules by Conte and Perdon [CP98]. There is another definition of the precontrollability sub-semimodule R, which is (A, B)-invariant and, for any x# ∈ R, there exists a time t ≥ 0 and an input sequence u = {u(k)}tk=0 such that every component in the state sequence x(eX ; u)tk=0 , starting from eX and produced by the input sequence u, remains in R and x(t) = x#. The family of the pre-controllability sub-semimodules of K in X is closed under 2 operation; therefore the family has a supremal element, denoted as R ∗ (K ), or simply R ∗ . If V ∗ (K ), or V ∗ , denotes the supremal controlled invariant sub-semimodule of K , then R ∗ can be characterized as the least element of the family RV ∗ = {R ⊂ X such that R = V ∗ ∩ (AR 2 B)}.

(16)

This conclusion can be proved by the following proposition. Proposition 2. Let {Rk }k≤0 be the family of sub-semimodules defined recursively by R0 = {e} Rk = V ∗ ∩ (ARk−1 2 B).

(17)

If there exists ∪k∈N Rk , then the supremal pre-controllability sub-semimodule R ∗ (K ) in K is equal to R∗ (V ∗ ) = ∪k∈N Rk , which is the least element of the family in Eq. (16).

138

Ying Shang and Michael K. Sain

5.3 Relation between Γ -Zeros and Invariance This subsection presents a connection between special zero semimodules and the supremal controlled invariant sub-semimodule and the supremal pre-controllability sub-semimodule in Ker C. For a linear system of the form (10) over a semiring R, the Γ -zero semimodule ZΓ (G(z)) of the transfer function G(z) is ZΓ (G(z)) =

G−1 (Ω Y ) . G−1 (Ω Y ) ∩ Ω U

(18)

A short exact sequence for the zero semimodule of output type ZO (G(z)) and the Γ -zero semimodule of G(z) is β

α

e −−−−→ Γ −−−−→ ZΓ (G(z)) −−−−→ ZO (G(z)) −−−−→ e,

(19)

where Ker G ; Ker G ∩ Ω U G−1 (Ω Y ) . ZO (G(z)) = −1 G (Ω Y ) ∩ (Ω U U(z) Ker G)

Γ =

ZO (G(z)) is called the zero semimodule of output type for G(z). These semimodules, Γ , ZΓ (G(z)), and ZO (G(z)), are R[z]-semimodules. Theorem 2. Given a linear system of the form (10) over a semiring R, if the supremal (A, B)-invariant sub-semimodule in Ker C is denoted by V ∗ and the supremal pre-controllability sub-semimodule in Ker C is denoted by R ∗ , then there exists the following commutative diagram with exact rows and columns. e ⏐ ⏐ %

e ⏐ ⏐ % i

e −−−−→ Ker p2 −−−2−→ ⏐ ⏐ %

Γ ⏐ ⏐ α%

i

e ⏐ ⏐ % −−−2−→

p

R∗ ⏐ ⏐ %i

−−−−→ e

p

V∗ ⏐ ⏐π %

−−−−→ e

e −−−−→ Ker p1 −−−1−→ ZΓ (G(z)) −−−1−→ ⏐ ⏐ ⏐ ⏐ β% % e

(20)

p

−−−−→ ZO (G(z)) −−−−→ V ∗ /R ∗ −−−−→ e ⏐ ⏐ ⏐ ⏐ % % e

e

Theorem 2 establishes the relationships between the Γ -zero semimodule and the supremal (A, B)-invariant sub-semimodule V ∗ , Γ and the supremal

Systems over a Semiring: Realization and Decoupling

139

pre-controllability sub-semimodule R ∗ , and the zero semimodule of output type ZO (G(z)) and the quotient V ∗ /R ∗ . Lemma 4. Given an R-semimodule B and its sub-semimodule A, the sequence i

p

e→A− →B− → B/A → e

(21)

is a short exact sequence, where i is an insertion and p is a natural projection. The middle column of the diagram in Eq. (20) is exactly the short exact sequence in Eq. (19). The right column is directly from Lemma 4. Define an R-morphism p : G−1 (Ω Y ) → Ω U as p (u(δ )) = u poly , where u(δ ) = u poly U(z) usp . Therefore, this mapping induces the two R-morphisms, p1 : ZΓ (G(z)) → XI (G(z)) and p2 : Z2 → XI (G(z))), where p2 = p1 ◦ α and XI (G(z)) is the pole semimodule of input type: XI (G(z)) =

ΩU . G−1 (Ω Y ) ∩ Ω U

That the diagram in Eq. (20) is exact can be proved by the following three lemmas. Lemma 5. The proper image of p1 is equal to the supremal controlled invariant subsemimodule V ∗ in Ker C, namely p1 (ZΓ (G(z))) = V ∗ . Lemma 6. The proper image of p2 is equal to the supremal pre-controllability subsemimodule R ∗ of Ker C, namely p2 (Γ ) = R ∗ . Lemma 7. There exists an R[z]-semiisomorphism p from ZO (G) to V ∗ /R ∗ . The differences between this diagram and the case of traditional systems over fields in [WS83] include that the sequence is exact, not proper exact, and the mapping p is an R[z]-semiisomorphism, not an R[z]-isomorphism.

6 DDP for Systems over a Semiring A linear system over a semiring R with a disturbance signal dk ∈ D is defined as xk+1 = Axk 2 Buk 2 Sdk , yk = Cxk , uk = Fxk ,

(22)

where xk ∈ X, uk ∈ U, and dk ∈ D. Then the system (22) is disturbance decoupled if A2BF|Im S ⊂ ker C, which is a trivial extension to Lemma 4.1 in [Won79]. Let K = ker C and T = Im S. The DDP is to find (if possible) a state feedback F : X → U such that A2BF|T ⊂ K . For linear systems over a field, the (A, B)controlled invariant subspaces are equivalent to the (A, B)-controlled invariant subspaces of feedback type. The solvability of DDP can be obtained by knowing the supremal controlled invariant sub-semimodule in the kernel of C.

140

Ying Shang and Michael K. Sain

Theorem 3. [Won79] The DDP is solvable for linear systems over a field if and only if the supremal controlled invariant subspace V ∗ in K contains T . Proposition 3. The DDP is solvable for a linear system of the form (22) over a semiring R, if and only if there exists a controlled invariant sub-semimodule of feedback type VFB ⊂ K which contains T . Proposition 4. If the DDP is solvable for a linear system of the form (22) over a semiring R, then the supremal controlled invariant sub-semimodule V ∗ in K and p1 (ZΓ (G(z))) both contain T . The preceding proposition shows that, for linear dynamical systems over a semi-ring, the necessary and sufficient solvability condition for systems over a field becomes only necessary. The commutative diagram in the preceding section presents a computational method for V ∗ using the map p1 : ZΓ (G(z)) → V ∗ . Then p1 (ζ ) = V ∗ , for any ζ ∈ ZΓ (G(z)).

7 Application to Queueing Networks In this section, the DDP is studied for two queueing networks modeled by a timed Petri net [Cas93]. A Petri net is a four-tuple (P, T, A, w), where P is a finite set of places, T is a finite set of transitions, A is a set of arcs, and w is a weight function, w : A → {1, 2, 3, · · ·}. I(t j ) represents the set of input places to the transition t j and O(t j ) represents the set of output places from the transition t j , i.e., I(t j ) = {pi : (pi ,t j ) ∈ A} and O(t j ) = {pi : (t j , pi ) ∈ A}. A marking x of a Petri net is a function x : P → {0, 1, 2, · · · }. The number represents how many tokens there are in a place. A marked Petri net is a five-tuple (P, T, A, w, x0 ) where (P, T, A, w) is a Petri net and x0 is the initial marking. For a timed Petri net, when the transition t j is enabled for the kth time, it does not fire immediately, but it has a firing delay, v j,k , during which the tokens are kept in the input places of t j . The clock structure associated with the set of timed transitions, TD ⊆ T , of a marked Petri net (P, T, A, w, x) is a set V = {v j : t j ∈ TD } of lifetime sequences v j = {v j,1 , v j,2 , · · · },t j ∈ TD , v j,k ∈ R+ , k = 1, 2, · · · . A timed Petri net is a six-tuple (P, T, A, w, x, V) where (P, T, A, w, x) is a marked Petri net and V = {v j : t j ∈ TD } is a clock structure. We consider a queueing system with two servers as shown in Figure 4, where the second server has a controller for the customer arrival times and a service disturbance, such as service breakdown. The timed Petri net model for this queueing system is shown in Figure 5. There are four places for the server i = 1, 2: Ai (arrival), Qi (queue), Ii (idle), and Bi (busy). For server 2, there are two more places: C (server 1 completion) and D (disturbance). So P = {A1 , Q1 , I1 , B1 ; A2 ,C, Q2 , I2 , B2 , D}. The transitions (events) for each server are ai (customer arrives), si (service starts), and ci (service completes and customer departs). For server 2, there are transitions u (control input) and d (disturbance). The timed transition TD = {a1 , a2 , c1 , c2 }. The clock

Systems over a Semiring: Realization and Decoupling

Disturbance

Control

⊕

Arrival

141

Server 2

Server 1

Departure

Fig. 4. A queueing system with two servers.

structure of this model has constant sequences va1 = {k1 , k1 , · · · }, vc1 = {k2 , k2 , · · · }, va2 = {k3 , k3 , · · · }, and vc2 = {k4 , k4 , · · · }. The rectangles present the timed transitions. The initial marking is x0 = {1, 0, 1, 0, 1, 1, 1, 1, 0, 1}. Max-plus algebra is to replace the traditional addition and multiplication by max and + operations for the set R ∪ {−inf}, i.e. def

a ⊕ b = max{a, b}, def

a ⊗ b = a + b. The set RMax = (R ∪ {−inf}, ⊕, −inf, ⊗, 0) is a semifield, because there are inverses with respect to the ⊗ operator. Denote ε = −inf and e = 0. We define aik as the kth arrival time of customers for service i, sik as the kth service starting time for service i, and cik as the kth service completion and the customer departure time, where i = {1, 2}. If we define xk = [a1k , s1k , c1k , a2k , s2k , c2k ]T and assume the output is the customer arrival time of the second server, then we can write the system equation using the max-plus algebra as xk+1 = A xk ⊕ B uk ⊕ S dk , yk = C xk ,

A2

A1 k1

a1

a2

I2 s2

d

s1

D

B1 c1

k3

Q2

I1

Q1

C

k2

B2 c2

k4

Fig. 5. The timed Petri net model for the queueing system.

142

Ying Shang and Michael K. Sain

where the system matrices are ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ k1 ε k2 ε ε ε ε ε ⎢ε ⎥ ⎢ε ⎥ ⎢ k1 ε k2 ε ε ε ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ε ⎥ ⎢ε ⎥ ⎢ k1 ε k2 ε ε ε ⎥ ⎢ ⎢ ⎥ ⎥ ⎥ ,B = ⎢ ⎥, S = ⎢ A=⎢ ⎢ ε ⎥ , and C = [ε , ε , ε , e, ε , ε ]. ⎥ ε ε k ε ε ε e 2 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ε ⎦ ⎣ε ⎦ ⎣ ε ε ε k3 ε k4 ⎦ ε ε ε ε k3 ε k4 e The transfer function G(z) = CBz−1 ⊕ CABz−2 ⊕ · · · = z−1 . Therefore, 2 # poly = u1 z ⊕ u2 z ⊕ · · · for any ui ∈ U. Then Bu poly = p1 (ZΓ (G(z))) = 2 ABu1 ⊕ A Bu2 ⊕ · · · , which equals the sub-semimodule generated by e5 and e6 , where e1 = [e, ε , . . . , ε ]T , · · · , e6 = [ε , ε , · · · , e]T . Pick F = [ f1 , f2 , f3 , f4 , ε , ε ] for any fi ∈ RMax , i = 1, · · · , 4, then (A ⊕ BF)p1 (ZΓ (G(z))) ⊂ p1 (ZΓ (G(z))). Therefore p1 (ZΓ (G(z))) is (A ⊕ BF)-invariant. K = Ker C equals the sub-semimodule generated by (e1 , e2 , e3 , e5 , e6 ), therefore, there exists an (A ⊕ BF)-invariant subsemimodule in the kernel of the output map C containing the proper image of S. Hence, DDP for this queueing system is solvable. G−1 (Ω Y ) = u

8 Conclusion This paper is motivated by applications of systems over a semiring in communication networks, manufacturing systems, and queueing systems. Kalman’s realization theory is generalized to systems over a semiring. A remarkable new feature of this system theory is that the realization process displays distinctly different characters when proceeding from the observability point of view than when proceeding from the reachability point of view. This new theory, moreover, provides computational methods for the crucial invariant sets in the DDP. Necessary and sufficient solvability conditions for the DDP are presented in terms of the supremal controlled invariant sub-semimodule in the kernel of the output map. These results can be used to design a compensator for systems over a semiring, for example, reducing data dropout effects in a certain region of a communication network, designing the time table for each railway station, and controlling the departure time in a queueing system.

References [At96]

Al-Thani, H.M.J.: k-projective semimodules, KOBE Journal of Mathematics, 13, 49–59 (1996) [BCOQ92] Baccelli, F.L., Cohen, G., Olsder, G.J., Quadrat, J.P.: Synchronization and Linearity: An Algebra for Discrete Event Systems. John Wiley & Sons, New York (1992) [Bly77] Blyth, T.S.: Module Theory: An Approach to Linear Algebra. Clarendon Press, Oxford (1977)

Systems over a Semiring: Realization and Decoupling [Cas93]

143

Cassandras, C.G.: Discrete Event Systems: Modeling and Performance Analysis, Irwin, Boston (1993) [CGQ99] Cohen, G., Gaubert, S., Quadrat, J.P.: Max-plus algebra and system theory: where we are and where to go now, Annual Reviews in Control, 23, 207–219 (1999) [CPW88] Conte, G., Perdon, A.M., Wyman, B.F.: Fixed poles in transfer function equations, SIAM J. Control and Optimization, 26, 2, 356–368 (1988) [CP98] Conte, G., Perdon, A.M.: The block decoupling problem for systems over a ring, IEEE Transactions on Automatic Control, 43, 1600–1604 (1998) [DDD98] De Vries, R., De Schutter, B., De Moor, B.: On max-algebraic models for transportation networks, Proceedings of the International Workshop on Discrete Event Systems, 457–462 (1998) [Dj02] De Jong, H.: Modeling and simulation of genetic regulatory systems: a literature review, Journal of Computational Biology, 9, 1, 67–103 (2002) [Gol99] Golan, J.S.: Semirings and Their Applications. Kluwer Academic Publishers, Boston (1999) [KFA69] Kalman, R.E., Falb, P.L., Arbib, M.A.: Topics in Mathematical System Theory. McGraw-Hill, New York (1969) [Kal65] Kalman, R.E.: Algebraic structure of linear dynamical systems, I. the module of σ , Proceedings of the National Academy of Sciences of the United States of America, 54, 6, 1503–1508 (1965) [LT01] Le Boudec, J.-Y., Thiran, P.: Network Calculus. Springer-Verlag, New York (2001) [MHSC05] Maia, C.A, Hardouin,L., Santos-Mendes, R., Cottenceau, B.: On the Model Reference Control for Max-Plus Linear Systems, 44th IEEE Conference on Decision and Control, 7799–7803 (2005) [SSS05] Sain, P.M., Shang,Y., Sain, M.K.: Reachability analysis for N-squared state charts over a Boolean semiring applied to a hysteretic discrete event structural control model, Proceedings of American Control Conference, 3072–3077 (2005) [Sain81] Sain, M.K.: Introduction to Algebraic Systems Theory. Academic Press, New York (1981) [Sh06] Shang, Y.: Semimodule Morphic Systems: Realization, Model Matching, and Decoupling, Ph.D. thesis, University of Notre Dame, Indiana (2006) [SS05] Shang, Y., Sain, M.K.: On the zero semimodules of systems over a semiring with applications to queueing systems, Proceedings of American Control Conference, 225–230 (2005) [Taka81] Takahashi, M.: On the Bordism categories II: elementary properties of semimodules, Mathematics Seminar Notes, 9, 495–530 (1981) [Won79] Wonham, W.M.: Linear Multivariable Control: A Geometric Approach, Springer-Verlag, New York (1979) [WS83] Wyman B.F., Sain, M.K.: On the zeros of a minimal realization, Linear Algebra and Its Applications, 50, 621–637 (1983)

Modules of Zeros for Linear Multivariable Systems Cheryl B. Schrader1 and Bostwick F. Wyman2 1

2

College of Engineering, Boise State University, 1910 University Drive, Boise, ID 83725-2100, USA (208) 426-1153 [email protected] Department of Mathematics, The Ohio State University, 231 West 18th Avenue, Columbus, OH 43210, USA (614) 292-4901 [email protected]

Summary. How should we interpret the motto: The zeros of a linear system become the poles of its inverse system? Rudolf Kalman first introduced the idea of the set of poles of a system as a module over an appropriate polynomial ring. Through the lens of the motto above, this historical survey discusses the work of Mike Sain and his coauthors on the module-theoretic approach to zeros of a linear system and the application of these ideas to inverse systems and system design.

1 Introduction Rudy Kalman created the field of mathematical system theory with his contributions to controllability, observability, and the Kalman filter. For some time, mathematical system theory concentrated on analytic methods and linear algebra. In 1965 Kalman introduced module-theoretic methods to study the poles of discrete-time linear systems [Ka65, KFA69]. A few years later, Mike Sain pointed out that algebraic methods were of growing importance in system theory [Sa76, Sa81]. One of us (Wyman), who had the good luck to be recruited by Kalman [RWK72], had more good luck when he met Mike in 1978 and spend the academic year 1978–79 with him at Notre Dame. Mike’s lectures about zeros in his course on multivariable linear control systems inspired a long, and still continuing, stream of research in the module theory of zeros. This short note will attempt to review some of the work inspired by Mike Sain and carried out by Mike and his coauthors. The papers discussed here were the product of collaboration and discussion between Mike Sain, the authors of this chapter, and others. We would like to acknowledge Professors Giuseppe (“Pino”) Conte and Anna Maria Perdon-Conte of the Universit´a Politecnico delle Marche, Ancona, Italy, Dr. Steve Giust, and our deceased colleague, Dr. Marek Rakowski (1956–2003). C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 7, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

146

Cheryl B. Schrader and Bostwick F. Wyman

Section 2 is a whirlwind introduction to the part of the theory of rings and modules over rings of rational functions that we need. Section 3 is a summary of the classical theory of pole modules (due to Kalman) and the theory of pole modules “at the point at infinity” due to Conte and Perdon. Section 4 presents the module theory of zeros, including the idea of generic zeros arising from the null space and the cokernel of a transfer function matrix. Section 5 presents a small part of the extensive research of applications of module theory to system design. Section 6 introduces the Wedderburn–Forney construction which allows us to count generic zeros and to give a structural algebraic meaning to the principle that “the number of zeros of a transfer function matrix equals the number of poles.” Finally, Section 7 presents two open problems that have mystified us for years. The reader should think of this chapter as one about algebra. For a survey of the history of zeros of multivariable systems and applications to engineering, please see [ScS89]. Before moving to the crux of this paper, the authors would like to thank Mike Sain for introducing them to each other. By collecting and disseminating work on module-theoretic approaches to systems, Mike nurtured further research spanning decades, disciplines, and countries. His networking inspired interesting questions and fascinating answers connecting fields of work, modes of thought, and, most importantly, people. For that, we are truly grateful.

2 Rings and Modules Let k be a field of scalars. Denote by k[z] the ring of polynomials in one indeterminate z over k, and by k(z) the field of rational functions. Within k(z) let O∞ = {g(z) =

n(z) : deg d(z) ≥ degn(z)} d(z)

be the ring of proper rational functions (“no pole at infinity”). The ring O∞ is a discrete valuation ring with maximal ideal m∞ = {g(z) =

n(z) : deg d(z) > deg n(z)}. d(z)

That is, m∞ = z−1 O∞ is the ideal of strictly proper rational functions (“vanish at infinity”). For a rational function f (z), we write π+ f (z) for the polynomial part of f (z) and π− f (z) for the strictly proper part of f (z). We have f (z) = π+ f (z) + π− f (z). The operators π+ and π− can operate on other objects (for example, rational vectors or matrices) in the same way. We will also use the field of formal Laurent series

Modules of Zeros

147

k((z−1 )) = {a−N zN + a−N+1 zN−1 + · · · + a−1z + a0 + a1z−1 + a2 z−2 + · · · }. By long division, k(z) ⊂ k((z−1 )). Our discrete-time intuition is that t = 0 is present time, t > 0 indexes past time, and t < 0 indexes future time. In this chapter we will be considering finite dimensional vector spaces over k, typically denoted V, X ,U, or Y . Any vector space V over k gives us (by tensoring up over k) a k(z) vector space V (z) = V ⊗ k(z), a free k[z] module Ω V = V [z] = V ⊗ k[z], and a free O∞ module Ω∞V = V ⊗ O∞ . We will also need the torsion divisible modules Γ V = VΩ(z) V , a module over k[z], and Γ∞V =

V (z) Ω ∞V ,

a module over O∞ . In Laurent series notation,

Γ V = {v1 z−1 + v2 z−2 + · · · : vi ∈ V } represents future strings. Multiplication by z shifts left, and the past disappears. On the O∞ side, past strings form a module

Γ∞V = {v−N zN + v−N+1 zN−1 + · · · + v0 : vi ∈ V }. Multiplication by z−1 shifts right, and the future vanishes. Next, consider k[z] and O∞ modules which are finite dimensional vector spaces over the field k of scalars. Suppose dimk V = n. Then a k[z] action on V is given completely by the action of z on V , and this action is a linear transformation A : V → V , often called the dynamics. Conversely, any such A on V gives a module action. V is a torsion k[z] module, killed by the minimal polynomial of A. Next, look at an O∞ module W , finite dimensional over k. This time the module action is completely determined by the action of z−1 on W . Define the linear transformation N : W → W by Nw = z−1 w. Since every ideal of O∞ is generated by a power of z−1 , N must be nilpotent. Conversely, every nilpotent map defines an O∞ module structure.

3 Pole Modules Suppose we are given a multivariable transfer function, that is, a k(z)-linear transformation G(z) : U(z) → Y (z). If dimk (U) = m and dimk (Y ) = p, then choosing bases gives a p × m matrix representation for G(z) over k(z). A pole of G(z) should be a pole of any of the matrix coeffcients. More than 40 years ago, Kalman [Ka65, KFA69] succeeded in defining a “module of poles” for G(z) which captured the numerical values, added a polynomial module structure, and supplied the minimal state space realization of G(z). Kalman’s pole module is given by X(G) =

ΩU G−1 (Ω Y ) ∩ Ω U

.

For the intuition, think of polynomial (“past time”) inputs, modulo those inputs producing outputs that vanish in future time. Let A : X (G) → X (G) be the dynamics

148

Cheryl B. Schrader and Bostwick F. Wyman

corresponding to the k[z] structure on X(G). The eigenvalues of A are exactly the classical poles of G(z). The natural projection Ω U → X (G) restricts to a k-linear transformation B : U → X(G), and (almost by definition) (A, B) is a reachable pair in the sense of Kalman. To study observability, it is convenient to map C˜ : X (G) → Γ Y by u(z) (modulo G−1 (Ω Y )) → G(z)u(z) (modulo Ω U). Define ρ : Γ Y → Y by “peeling off” the coefficient of z−1 . Finally, let C : X (G) → ˜ Then (C, A) is an observable pair, and (C, A, B) is Kalman’s minimal realY = ρ C. ization with C(zI − A)−1 B = π− G(z). Conte and Perdon studied poles at infinity of (the polynomial part of) a transfer function in [CP82, CP84]. Their treatment is parallel to Kalman’s, replacing polynomials with strictly proper rational functions and the polynomial ring with the ring O∞ . We have z−1 Ω∞U X∞ (G) = −1 −1 . G (z Ω∞Y ) ∩ z−1 Ω∞U This space, sometimes called “the little pole space at infinity,” counts the number of poles at infinity correctly and supplies a nice algebraic structure. On the other hand, it is not sufficient for realization theory. In [CP84] the “big pole space at infinity” is defined as X"∞ (G) =

Ω ∞U . G−1 (z−1 Ω∞Y ) ∩ z−1 Ω∞U

Since X"∞ (G) is an O∞ module, it inherits a nilpotent dynamics map N : X"∞ → X"∞ . The two pole modules are related by N(X"∞ (G)) = X∞ (G). The natural projection B˜ ∞ : Ω∞U → X"∞ (G) following U ⊂ Ω∞U gives an input map B∞ : U → X"∞ (G). To define an output map, first look at G(z) : U(z) → Y (z), which induces C˜∞ : X"∞ (G) → A typical member of

Y (z) z−1 Ω∞Y

Y (z) z−1 Ω

∞Y

.

looks like

y(z) = y−k zk + · · · + y1 z + y0 . Define ρ∞ (y(z)) = y0 and let C∞ = ρ∞C˜∞ . Realization theory can be carried out in this context, yielding π+ G(z) = C∞ (zN − I)−1 B∞ .

Modules of Zeros

149

Having constructed the classical pole module and the two pole modules at infinity, we can form two global pole spaces X (G) = X (G) ⊕ X∞ (G), ,(G) = X (G) ⊕ X "∞ (G). X Since X(G) and X∞ (G) are modules over different rings, these global spaces have no natural module structure. They do have a filtration structure which leads to various indices associated with G(z). The study of these numbers is an industry which is ignored here but is also related to the Wedderburn–Forney spaces discussed in Section 6. For further discussion of these issues, see the papers [Fo75, Gi93, GRWS93, GW92]. A more abstract approach to “local and global pole modules” can be found in [WCP86].

4 Modules of Zeros A transmission zero of a transfer function G(z) : U(z) → Y (z) is, roughly speaking, a scalar λ ∈ k such that the matrix G(λ ) has unexpectedly low rank. There are several difficulties with this definition. If the field k of scalars is not algebraically closed, then λ must be replaced by an irreducible polynomial in k[z], and we must also look at the point at infinity. This problem is not that difficult and can be fixed with some algebraic techniques. More seriously, what happens when we are convinced that one of the poles should also be counted as a zero? The matrix G(λ ) doesn’t make sense in this case. Here is another issue: whatever zeros are, the zeros of a transfer function should appear as poles of its inverse (if it exists). Maybe we should look at right and left inverse transfer functions. The poles of a transfer function are described by nice modules, so shouldn’t we be looking at nice module structures for the zeros as well? Issues such as these appeared frequently in the classical control theory literature in a different form. Zeros of multivariable systems appeared to have “directions,” and zeros and poles in different directions might not cancel out when systems are connected. An early treatment of zeros with a great deal of algebraic sophistication was due to Rosenbrock [Ro70]. His work on polynomial matrices underlies much of the later work reported here. In this chapter we will only discuss the module structures on sets of transmission zeros, although there are also module-theoretic treatments of input-decoupling zeros and output-decoupling zeros available in the literature. The basic papers on the module of zeros for a transfer function are [CP84, WS81a, WS81b, WS83, WS85]. The finitely generated, torsion zero module Z(G) associated with a k(z)-linear map G(z) was defined in [WS81a] to be the k[z]-factor module Z(G) =

G−1 (Ω Y ) + Ω U . ker G(z) + Ω U

150

Cheryl B. Schrader and Bostwick F. Wyman

The intuition here is that a zero should correspond to a rational input which produces no output in the future, modulo inputs which themselves are restricted to past time. There is overwhelming evidence that this construction gives the “correct” module of zeros. It is a finite dimensional vector space over k. If a (right or left) inverse transfer function of G(z) exists, its pole module contains this zero module as a factor module or a submodule, as appropriate. The k[z] structure on Z(G) gives a natural zero dynamics so that its structure is naturally isomorphic to the feedback action ∗ on the controlled invariant space VR∗ in the sense of Wonham and Basile-Marro. (See [Wo74], for example.) If the transfer function G(z) factors coprimely into polynomial matrices, G = ND−1 , then the pole module of G is the cokernel of D and the module of zeros of G is (almost) the cokernel of N. The module of zeros at infinity is defined in [CP84] and has good properties corresponding to the list above for Z(G). We set Z∞ (G) =

G−1 (Ω∞Y ) + Ω∞U . ker G(z) + Ω∞U

We can also define a global module of zeros by taking direct sums, Z (G) = Z(G) ⊕ Z∞ (G). The space Z (G) is a finite dimensional vector space over the scalars k, and does not have a natural module structure. The development of a persuasive theory of “dynamical indices” for global zeros is an intriguing open question. If G(z) is square and invertible, then the modules of transmission zeros defined above capture all available zero information. If G(z) is singular, however, these finite dimensional modules miss some crucial elements. For example, if G(z) has a nonzero nullspace, then in some sense G(z) has unexpected low rank “everywhere.” That is, inputs in ker G(z) produce no outputs at all, so they must be counted as zeros. It is tempting to consider another module, called the Γ -zero module, ZΓ (G) =

G−1 (Ω Y ) + Ω U . ΩU

This construction, crucial in many investigations, is often “very big.” Consider a special submodule of ZΓ (G) defined as follows:

Γ (G) =

ker G(z) + Ω U . ΩU

Now Γ (G) is a divisible k[z] module, infinite dimensional over k. Γ (G) is the divisible part of ZΓ (G), yielding the exact sequence 0 → Γ (G) → ZΓ (G) → Z(G) → 0. The same considerations can be carried out at infinity, but we omit that discussion here. We also omit the parallel situation when G(z) fails to be surjective. In that case

Modules of Zeros

151

it is important to consider a big module of zeros which has a free part instead of a divisible part. In both cases, new zeros captured by infinite dimensional spaces are called generic zeros. They are, perhaps surprisingly, unavoidable in some concrete applications. See the next section for an example.

5 An Application to System Design A crucial application area in the design of linear control systems involves polezero cancellations that occur in various interconnected systems. Conversely, we often need to investigate the properties of desired connections when some components are given. Problems of this sort are often referred to as “minimal factorizations,” “total synthesis design,” or “nominal design.” See, for example, the papers [SWP88, SWP91]. Again, we refer to the survey [ScS89] for history up to 1989. In this paper we discuss pole and zero module considerations which arise in one important special case. Suppose we are given two transfer functions T (z) : U(z) → Y (z) G(z) : U(z) → W (z). Assume that ker G(z) ⊂ ker T (z). Then from linear algebra over k(z) there exist (possibly many) k(z)-linear transformations H(z) : W (z) → Y (z) such that T (z) = H(z)G(z); see Figure 1 below. If T (z) is the identity map, this problem is just the design problem for left inverses ( [WS81a, WS85]). We can ask what poles and zeros are forced to appear in every solution H(z) to our problem. To proceed, we would like to make good algebraic sense of a string of intuitive remarks:

U(z) J

J J

G(z)

T (z) -

Y (z) 6

J J J

H(z) J J J ^ J W (z)

Fig. 1. System design mapping.

152

• • • •

Cheryl B. Schrader and Bostwick F. Wyman

poles of T (z) that do not appear in G(z) must be supplied by H(z), zeros of G(z) that do not appear in T (z) must be cancelled out by poles of H(z), zeros of T (z) that do not appear in G(z) must be supplied by H(z), and poles of G(z) that do not appear in T (z) must be cancelled out by zeros of H(z).

The constructions discussed here can be found in [CPW88, Sa92] and in additional papers cited there. Here we only consider poles and zeros in the whole finite plane (as captured by polynomial module structures). Similar results hold at infinity and for specified sets of “good modes.” All this is done in detail in [CPW88]. Define a module of fixed poles by P=

G−1 (Ω W ) G−1 (Ω W ) ∩ T −1 (Ω Y )

.

The first result in [CPW88] is that there is a natural module injection 0 → P → X (H) for any H such that T = HG. We can say that the fixed pole module P appears in the pole module of any solution to our problem. If we want this result to be useful and to fit in with the intuitive remarks above, we must learn more about the fixed pole module P. To start, consider   T (z) : U(z) → Y (z) ⊕ W (z). G(z)   T ) and a finite dimensional transmisThis transfer function has a pole module X ( G   T ). We can define two new modules, “relative zeros” Z(T, G) sion zero module Z( G and “relative poles” X (T, G), by two natural exact sequences   T ) → Z(G) → Z(T, G) → 0 0 → Z( G and

  T ) → X (G) → 0. 0 → X(T, G) → X( G

  T . In The module Z(T, G) consists of those zeros of Z(G) which are not zeros of G other words, we expect   Z(T, G) to be cancelled out by the poles of any appropriate H. T ) describes the union of the poles of T (z) and G(z), so X (T, G) Analogously, X ( G represents poles of T which must be supplied by H. All of this can be assembled into an exact sequence 0 → X (T, G) → P → Z(T, G) → 0,

Modules of Zeros

153

so that the fixed pole module P mirrors the first two intuition bullets. What about zeros? A similar approach gives good results, but the methodology is a little more complicated since the infinite dimensional module of zeros ZΓ (G) must be used if G(z) is not injective. The matching zero module is defined by Z=

T −1 (Ω Y ) . T −1 (Ω Y ) ∩ G−1 (Ω W )

The map G(z) : U(z) → W (z) defines a module injection 0 → Z → ZΓ (H) =

H −1 (Ω Y ) H −1 (Ω Y ) ∩ G−1 (Ω W )

,

so that Z describes zeros which must appear in any solution H to T = HG. Again, we need to examine the module Z to match our intutition. Define a module Z2 (T, G) by   T → ZΓ (T ) → Z2 (T, G) → 0. 0 → ZΓ G This module measures the zeros of T which are not zeros of G. The pole module analog X2 (T, G) is defined by   T → X (T ) → 0. 0 → X2 (T, G) → X G The module X2 (T, G) describes poles of G that are not also poles of T . These modules can be assembled into another exact sequence 0 → X2 (T, G) → Z → Z2 (T, G) → 0, and we see that the matching zero module Z mirrors the intuition of the second two bullets above.

6 The Fundamental Pole-Zero Exact Sequence A rational function has just as many poles as zeros, if multiplicities are counted correctly and the point at infinity is included. The same result holds for square invertible matrices of rational functions, but not for rectangular or singular matrices. For the classical “fudge terms” to fix the hope that “a transfer function has just as many zeros as poles,” see, for example, Kailath’s book [Kai80], p. 455. We can ask our usual question: what structure lies behind counting the poles and zeros? If we look at the finite dimensional spaces of global poles and zeros, X (G) and Z (G) and count dimensions, we get dimk (Z (G)) ≤ dimk (X (G)),

154

Cheryl B. Schrader and Bostwick F. Wyman

but this inequality is often strict. To proceed we have to figure out how to count the additional zeros. We have already pointed out that the nullspace of G(z) and the failure of G(z) to be surjective supply “generic zeros” which lead to infinite dimensional spaces of zeros. We need to supply a sensible way to count these zeros. The Wedderburn–Forney construction introduced in [WSCP88, WSCP89] attaches a finite dimensional vector space over k, W (C ), to a k(z)-vector space C . This Wedderburn–Forney space associated with a subspace C of V (z) = k(z)n is given by C W (C ) = . C ∩V (z) ⊕ C ∩ z−1 Ω∞V (z) The dimension of W (C ) is finite, and it is exactly the sum of the dynamical indices of C in the sense of Forney and others. Note carefully that the Wedderburn–Forney space is attached not just to the abstract space C but to the imbedding C ⊂ k(z)n . Consider the special case C = ker G(z) ⊂ U(z). In [WSCP89] it was shown that W (ker G(z)) can be identified as a subspace of the global pole space X (G). The main result connects global poles, global zeros, and Wedderburn–Forney spaces. Theorem 1 (Fundamental Pole-Zero Exact Sequence). Let G(z) be a transfer function. Then there is an exact sequence of finite dimensional vector spaces over k, X (G) 0 → Z (G) → → W (im G(z)) → 0, W (ker G(z)) where Z (G) is the global space of zeros of G(z), X (G) is the global space of poles of G(z), and W ( ) is the Wedderburn–Forney construction. This result gives a structural interpretation to the assertion above that “the number of zeros of a transfer function equals the number of poles, provided that the generic zeros arising from the kernel and cokernel are counted correctly.” This important theorem has a great many consequences and adds a structural interpretation to many previous results. For example, a “minimal” polynomial basis in the sense of Forney [Fo75] is exactly a matrix G(z) with polynomial entries, full column rank, and no global zeros at all. In this case the theorem gives an isomorphism X∞ (G) ∼ = W (im G(z)) which identifies the O∞ module structure constants of X∞ (G) with the dynamical indices of the image. For more applications and commentary on the Wedderburn– Forney space, please see [WSCP88, WSCP89] and especially [WSCP91], pp. 136–138. In this short chapter we can only mention that there are strong connections between the module theory and the Wedderburn theory of the Rosenbrock system matrix of an implicit system. For research in this area, please see [ScS91, ScS94a,

Modules of Zeros

155

ScS94b, WS82, WS87]. Finally, the paper [Wy02] gives an interpretation and proof of the fundamental pole-zero exact sequence in terms of the cohomology of sheaves on the projective line, following ideas of Lomadze [Lo90, Lo99].

7 Mysteries In this section we mention two open problem areas in the hope that they will intrigue other researchers. Problem 1 (Global Inverse Transfer Functions). Given a left or right invertible transfer function G(z) with a given global zero module Z (G), find an inverse H(z) with global pole module X (H) of minimal dimension. The basic papers [WS81a, WS81b, WS85] and their analogs at infinity show that there exist inverses which are “essential almost everywhere.” In the original polynomial case, this means that given G(z) left invertible, say, there is an inverse H(z) whose finite pole module X (H) is exactly Z(G). However, examples show that X∞ (H) can be larger than Z∞ (G) in this case. So, we can ask for the minimal dimension of the global pole space. We don’t know the answer, and we have not found any reasonable method of attack. Even the case of a 1 × n row-vector transfer function seems difficult: the answer will have something to do with the degrees of polynomials which appear in the polynomial Euclidean algorithm. Problem 2 (Algebraic Transfer Functions). Suppose we are given an algebraic function field K, or, in other words, a finite extension of the field k(z) of rational functions. If H(z) is a transfer function with coefficients in K, can we define finite dimensional pole and zero spaces for H(z)? This problem already makes sense for algebraic functions, thought of as 1×1 transfer functions. Any such function can be expanded in a formal Laurent series at infinity, a−N zN + a−N+1 zN−1 + · · · + a−1z + a0 + a1z−1 + a2z−2 + · · · just as in Section 2. Classical realization theory only distinguishes between rational and irrational transfer functions, giving a finite dimensional pole space for rational functions and an infinite dimensional pole space for irrational functions. As an article of algebraic faith, we believe that algebraic functions are much nicer than transcendental ones, and we feel that there should be a nice theory for algebraic function fields. We are encouraged by some work of Lomadze, and by the fact that under certain circumstances pole modules are given by cohomology groups. We are still mystified, and we are trying to connect some of these abstract concepts with concrete usable results about Laurent series.

References [AM69]

M.F. Atiyah and I.G. MacDonald, Introduction to Commutative Algebra. Addison-Wesley, Reading, MA: 1969.

156

Cheryl B. Schrader and Bostwick F. Wyman

[BAM93]

J.A. Ball, M. Rakowski, and B.F. Wyman, Coupling Operators, WedderburnForney Spaces, and Generalized Inverses, Linear Algebra and Its Appl., Vol. 203–204, pp. 111–138, 1993. [CP82] G. Conte and A.M. Perdon, Generalized State Space Realizations of Non Proper Rational Transfer Functions, System and Control Letters, Vol. 1, No. 4, pp. 270–276, Jan. 1982. [CP84] G. Conte and A.M. Perdon, Infinite Zero Module and Infinite Pole Module, Proc. 6th Int. Conf. Anal. and Opt. of Syst., Lecture Notes Contr. and Inf. Sci., Vol. 62, pp. 302–315, 1984. [CP85a] G. Conte and A.M. Perdon, Zero Modules and Factorization Problems, Contemporary Mathematics: Linear Algebra and Its Role in Systems Theory. Providence: American Mathematical Society, Vol. 47, pp. 81–94, 1985. [CP85b] G. Conte and A.M. Perdon, On the Causal Factorization Problem, IEEE Trans. Automat. Contr., Vol. AC-30, No. 8, pp. 811–813, August 1985. [CP86] G. Conte and A.M. Perdon, Zeros of Cascade Compositions, in Frequency Domain and State Space Methods for Linear Systems, C.I. Byrnes and A. Lindquist, eds., North Holland: Elsevier Science Publishers B.V., 1986. [CPW86] G. Conte, A.M. Perdon, and B.F. Wyman, Factorizations of Transfer Functions, Proc. 25th Conf. Decision Contr., pp. 1279–1283, Dec. 1986. [CPW88] G. Conte, A.M. Perdon, and B.F. Wyman, Fixed Poles in Transfer Function Equations, SIAM J. Contr. and Opt., Vol. 26, No. 2, pp. 356–368, March 1988. [Fo75] G.D. Forney, Jr., Minimal Bases of Rational Vector Spaces with Applications to Multivariable Linear Systems, SIAM J. Contr., Vol. 13, No. 2, pp. 493–520, May 1975. [Gi93] S.J. Giust, The Global Controllability Filtration for Improper Transfer Functions and the Wedderburn-Forney Construction, Proc. 31st Allerton Conference, pp. 1206–1215, 1993. [GRWS93] S.J. Giust, M. Rakowski, B.F. Wyman, and C.B. Schrader, “Controllability Indices and Wedderburn-Forney Spaces,” Proc. 32nd IEEE CDC, pp. 2866– 2871, 1993. [GW92] S.J. Giust and B. F. Wyman, Global Observability Filtrations for Improper Transfer Functions, Proc. International Symposium Implicit and Nonlinear Systems, pp. 236–243, 1992. [Kai80] T. Kailath, Linear Systems. Englewood Cliffs, NJ: Prentice-Hall, Inc., 1980. [Ka65] R.E. Kalman, Algebraic Structure of Linear Dynamic Systems. I. The Module of Σ , Proceedings National Academy of Sciences (USA), Vol. 54, pp. 1503–1508, 1965. [KFA69] R.E. Kalman, P.L. Falb, and M.A. Arbib, Topics in Mathematical System Theory. New York: McGraw-Hill, 1969. [Lo90] V. Lomadze, Finite-dimensional time-invariant linear dynamical systems: Algebraic theory, Acta Appl. Math., Vol. 19, pp. 149–201, 1990. [Lo99] V. Lomadze, Applications of Vector Bundles to Factorization of Rational Matrices, Linear Algebra and Its Appl., Vol. 288, pp. 249–258, 1999. [Ra92] M. Rakowski, Minimal Factorization of Rational Matrix Functions, IEEE Trans. Circuits Systems, Vol. 39, pp. 440–445, 1992. [Ra93] M. Rakowski, State Space Solution of the Causal Factorization Problem, IEEE Trans. Auto Control, Vol. 38, pp. 1843–1848, 1993. [RW94] M. Rakowski and B. Wyman, Complexity of Generalized Inverses of a Transfer Function, in Systems and Networks: Mathematical Theory and Applications II. U. Helmke, R. Mennicken, and J. Saurer, eds., Berlin: Akademie Verlag, pp. 431–436, 1994.

Modules of Zeros [RWK72]

[Ro70] [Sa76] [Sa81] [Sa92] [SaS92]

[SW87]

[SWP88]

[SWP91] [Sc94] [ScS89] [ScS91]

[ScS94a]

[ScS94b]

[SWG95] [We34] [Wo74] [WCP86]

[WS80]

157

Y. Rouchaleau, B. Wyman, and R.E. Kalman, Algebraic Structures of Linear Dynamical Systems III. Realization Theory Over a Commutative Ring, Proc. Nat. Acad. Sci., Vol. 69, pp. 3404–3406, 1972. H.H. Rosenbrock, State-space and Multivariable Theory. New York: John Wiley & Sons, Inc., 1970. M.K. Sain, “The Growing Algebraic Presence in Systems Engineering: An Introduction,” IEEE Proceedings, Vol. 64, Number 1, pp. 96–111, January 1976. M.K. Sain, Introduction to Algebraic Systems Theory. Academic Press, New York, 1981. M.K. Sain, System Singularity: The Module Theoretic Approach, in Proc. Int. Symposium Implicit and Nonlinear Systems, pp. 252–260, 1992. M.K. Sain and C.B. Schrader, Feedback, Zeros, and Blocking Dynamics, in Recent Advances in Mathematical Theory of Systems, Control, Networks and Signal Processing I. H. Kimura and S. Kodama, eds., Tokyo: Mita Press, pp. 227–232, 1992. M.K. Sain and B.F. Wyman, Extended Zeros and Model Matching, Proc. 25th Allerton Conf. on Communication, Contr., and Computing, pp. 524–533, Oct. 1987. M.K. Sain, B.F. Wyman, and J.L. Peczkowski, Matching Zeros: a Fixed Constraint in Multivariable Synthesis, Proc. 27th Conf. Decision Contr., pp. 2060– 2065, Dec. 1988. M.K. Sain, B.F. Wyman, and J.L. Peczkowski, Extended Zeros and Model Matching, SIAM J. Contr. and Opt., Vol. 29, No. 3, pp. 562–593, 1991. C.B. Schrader, Dynamical Structures on Pencils, Poles, and Fundamental Subspaces, Linear Algebra and Its Appl., Vol. 205–206, pp. 1061–1079, 1994. C.B. Schrader and M.K. Sain, Research on System Zeros: A Survey, Int. J. Contr., Vol. 50, No. 4, pp. 1407–1433, Oct. 1989. C.B. Schrader and M.K. Sain, Module Theoretic Results for Feedback System Matrices, in Progress in System and Control Theory, G. Conte, A. Perdon, and B. Wyman, eds., New York: Birkhauser Boston, Inc., pp. 652–659, 1991. C.B. Schrader and M.K. Sain, Zero Principles for Implicit Feedback Systems, Circuits, Systems, and Signal Processing: Special Issue on Implicit and Robust Systems, Vol. 13, No. 2–3, pp. 273–293, 1994. C.B. Schrader and M.K. Sain, On the Relationship Between Extended Zeros and Wedderburn-Forney Spaces, in Systems and Networks: Mathematical Theory and Applications II. U. Helmke, R. Mennicken, and J. Saurer, eds., Berlin: Akademie Verlag, pp. 471–476, 1994. C.B. Schrader, B.F. Wyman, and S.J. Giust, Controllability, Zeros, and Filtrations for Singular Systems, Proc. 34th IEEE CDC, pp. 2354–2355, 1995. J.H.M. Wedderburn, Lectures on Matrices, A.M.S. Colloquium Publications, Vol. 17, Chapter 4, 1934. W.M. Wonham, Linear Multivariable Control: A Geometric Approach. New York: Springer-Verlag, 1974. B.F. Wyman, G. Conte, and A.M. Perdon, Local and Global Linear System Theory, in Frequency Domain and State Space Methods for Linear Systems, C.I. Byrnes and A. Lindquist, eds. North-Holland: Elsevier Science Publishers B.V., 1986. B.F. Wyman and M.K. Sain, The Zero Module and Invariant Subspaces, Proc. 19th Conf. Decision Contr., pp. 254–255, Dec. 1980.

158

Cheryl B. Schrader and Bostwick F. Wyman

[WS81a]

B.F. Wyman and M.K. Sain, The Zero Module and Essential Inverse Systems, IEEE Trans. Circuits Syst., Vol. CAS-28, No. 2, pp. 112–126, Feb. 1981. [WS81b] B.F. Wyman and M.K. Sain, The Pole Structure of Inverse Systems, Int. Fed. Automat. Contr.; 8th Triennial World Cong., Vol. 3, pp. 76–81, August 1981. [WS81c] B.F. Wyman and M.K. Sain, Exact Sequences for Pole-Zero Cancellation, Proc. Int. Symp. Math. Theory of Networks and Syst., pp. 278–280, August 1981. [WS82] B.F. Wyman and M.K. Sain, Internal Zeros and the System Matrix, Proc. 20th Allerton Conf. on Communication, Contr., and Computing, pp. 153–158, Oct. 1982. [WS83] B.F. Wyman and M.K. Sain, On the Zeros of a Minimal Realization, Linear Algebra and Its Appl., Vol. 50, pp. 621–637, 1983. [WS85] B.F. Wyman and M.K. Sain, On the Design of Pole Modules for Inverse Systems, IEEE Trans. Circuits Syst., Vol. CAS-32, No. 10, pp. 977–988, Oct. 1985. [WS87] B.F. Wyman and M.K. Sain, Module Theoretic Zero Structures for System Matrices, SIAM J. Contr. and Opt., Vol. 25, No. 1, pp. 86–99, Jan. 1987. [WS88] B.F. Wyman and M.K. Sain, Zeros of Square Invertible Systems, in Linear Circuits, Systems and Signal Processing: Theory and Application, pp. 109–114, C.I. Byrnes, C.F. Martin, and R.E. Saeks, eds., North Holland: Elsevier Science Publishers B.V., 1988. [WSCP88] B.F. Wyman, M.K. Sain, G. Conte, and A.M. Perdon, Rational Matrices: Counting the Poles and Zeros, Proc. 27th Conf. Decision Contr., pp. 921–925, Dec. 1988. [WSCP89] B.F. Wyman, M.K. Sain, G. Conte, and A.M. Perdon, On the Zeros and Poles of a Transfer Function, Linear Algebra and Its Appl., Vol. 122/123/124, pp. 123–144, 1989. [WSCP91] B.F. Wyman, M.K. Sain, G. Conte, and A.M. Perdon, Poles and Zeros of a Matrix of Rational Functions, Linear Algebra and Its Appl., Vol. 157, pp. 113–139, 1991. [Wy91] B.F. Wyman, Models and Modules: Kalman’s Approach to Algebraic Systems Theory, Mathematical Systems Theory: The Influence of R.E. Kalman, A.C. Antoulas, ed., New York: Springer, 1991. [Wy02] B.F. Wyman, Poles, zeros, and sheaf cohomology, Linear Algebra and its Appl., Vol. 351/352, pp. 799–807, 2002.

Zeros in Linear Time-Delay Systems Giuseppe Conte and Anna Maria Perdon Dipartimento di Ingegneria Informatica, Gestionale e dell’Automazione, Universit`a Politecnica delle Marche, via Brecce Bianche - 60131 Ancona - Italy. (gconte,perdon)@univpm.it

Summary. The algebraic notion of zero module of a linear, dynamical system, introduced by M. Sain and B. Wyman, allows us to generalize the notion of zero to frameworks that differ from the classical one. In particular, it is possible to define zeros for systems with coefficients in a ring and to relate such a notion to geometric concepts. On this basis, the chapter considers zeros for time-delay linear systems and it investigates their properties in connection with inversion and other control problems.

1 Introduction The notion of zero of a linear, dynamical system has been investigated and studied by several authors from many different points of view (see [SCS89] for a comprehensive discussion of the literature). Among others, the approach based on the notion of zero module, introduced by M. Sain and B. Wyman in the fundamental paper [WyS81], recalled below, provides conceptual and practical tools that, besides being useful in the analysis and synthesis of classical linear systems, can be effectively generalized to a larger class of dynamical systems. In particular, an algebraic notion of zero has been given in [CoP84] for linear, dynamical systems with coefficients in a ring, instead of a field, and some features of such a notion have been investigated. In this Chapter, exploiting the possibility of representing time-delay systems as systems with coefficients in a ring, we show how the algebraic notion of zeros given in that case can be employed for studying control properties of time-delay systems. We mainly use the results of [CoP00b], [CPI01] and [CPM07] to point out the relations between zeros, controlled invariant subspaces and inverse systems and to discuss a formal concept of minimum phase system in the time-delay case. The chapter is organized as follows. In Section 2 the notion of zero module is recalled, together with some of the results of [CoP84]. In Section 3, ring models for time-delay systems are described, in order to approach the problem of system inversion for time-delay systems by means of systems over rings. This is done by introducing a notion of zero dynamics for time-delay systems and by relating its C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 8, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

160

Giuseppe Conte and Anna Maria Perdon

properties to the properties of inverses. The concept of minimum phase time-delay system is also examined and discussed in terms of zero dynamics. Similar considerations are then developed for the problem of tracking a given reference signal in the time-delay framework.

2 Zero Module for Systems with Coefficients in a Ring Let R denote a commutative ring, which is assumed to be Noetherian and integral. By a linear, dynamical system with coefficients in R we mean a quadruple Σ = (A, B,C, X), where 1. X is a finitely generated R-module and 2. A : X → X , 3. B : Rm → X , 4. C : X → R p are R-morphisms. Denoting by t ∈ Z a discrete time variable, Σ describes a dynamic relationship between the variables x ∈ X , u ∈ U = Rm , y ∈ Y = R p by means of the equations  x(t + 1) = Ax(t) + Bu(t) (1) y(t) = Cx(t). In analogy with the classical case of linear, dynamical, discrete-time systems with coefficients in the field ℜ of real numbers, we think of the variables x, u and y as, respectively, the state, input and output of Σ . Besides being considered as abstract algebraic objects, systems with coefficients in a ring have been proved to be useful for modelling and studying particular classes of dynamical systems, for instance when varying parameters or delays (as in the following sections) have to be taken into account. General results concerning the theory of systems with coefficients in a ring and a number of related control problems can be found in [BBV86], [CoP00a], [Son76], [Son81] and in the references therein. Computational issues in the theory of systems with coefficients in a ring have been considered in [PGC03] and [PAC06], providing tools and methods for their solutions. Introducing the ring R[z] of polynomials in the indeterminate z with coefficients in R and its localization R(z) = S−1 R[Z] at the multiplicative set S of all monic polynomials, we can associate to the system Σ its transfer function matrix GΣ = C(zI − A)−1 B, whose elements are in R(z), and the naturally associated R(z)morphism C(zI − A)−1 B : U ⊗ R(z) → Y ⊗ R(z). Each element u(z) of U ⊗ R(z) can be written as u(z) = Σt∞0 ut z−t , with ut ∈ U, and it can be naturally interpreted as a time sequence, from some time t0 to ∞, of inputs. Respectively, each element y(z) of Y ⊗ R(z) can be written as y(z) = Σt∞0 yt z−t , with yt ∈ Y , and it can be naturally interpreted as a time sequence, from some time t0 to ∞, of outputs. From the point of view we consider here, following [WyS81], the zeros of Σ are determined by the transfer function matrix GΣ in an abstract algebraic way. To this

Zeros in Linear Time-Delay Systems

161

aim, let us recall that the R[z]-modules U ⊗ R[z] and Y ⊗ R[z], usually denoted by Ω U and by Ω Y , are naturally embedded into U ⊗ R(z) and into Y ⊗ R(z), respectively. Then, as in [CoP84], we can extend to the framework of systems with coefficients in a ring the definition of zero module first introduced in [WyS81]. Definition 1 ( [WyS81] Section III, [CoP84] Definition 2.1). Given the system Σ = (A, B,C, X) with coefficients in the ring R and transfer function matrix GΣ , the zero module of Σ is the R[z]-module ZΣ defined by ZΣ = (G−1 (Ω Y ) + Ω U)/(KerG + ΩU). The reader is referred to [WyS81] and [CoP84] for motivation in considering the above definition of the zero module and for an interpretation of ZΣ in terms of system dynamics. An important property of ZΣ is that it is a finitely generated R[z]-module ([CoP84] Proposition 2.4). In addition, ZΣ is closely related to the numerator matrix of polynomial matrix factorizations of GΣ , as shown by the following proposition. Proposition 1 ( [CoP84] Proposition 2.5). Let GΣ = D−1 N be a factorization where D = D(z) and N = N(z) are coprime polynomial matrices of suitable dimensions, with D(z) invertible over R(z). Then, the canonical projection pN : Ω Y → Ω Y /N Ω U induces an injective R[z]-homomorphism α : ZΣ → Tor(Ω Y /N Ω U). The above proposition says that ZΣ is a finitely generated, torsion R[z]-module. If R is a principal ideal domain, we describe ZΣ as a pair (Rm , D), where D : Rm → Rm is an R-automorphism of the free R-module Rm . It is reasonable then to interpret the zero dynamics of Σ as in the following definition. Definition 2. Given a system Σ with coefficients in the principal ideal domain R, whose zero module is represented as the pair (Rm , D), the zero dynamics of Σ is the dynamics induced on Rm by D, that is by the dynamic equation z(t + 1) = Dz(t), for z ∈ Rm . When R is a field, as shown in [WyS81], α is actually an isomorphism. In the ring case, additional conditions are required in order to assure that α is an isomorphism. If R is a principal ideal domain and GΣ = D−1 N is a Bezout factorization, α is an isomorphism if N(U ⊗ R(z)) is a direct summand of Y ⊗ R(z) (see [CoP84] Section 2). Concerning the relation between the zero module of an invertible system and its inverse, here we are interested mainly in recalling a number of results from [CoP84] (compare also with [WyS81]). To this aim, let us recall that right invertibility of Σ can be characterized by the fact that GΣ is surjective, while left invertibility of Σ can be characterized by the fact that GΣ is injective and its image ImG is a direct summand of Y (z) = Y ⊗ R(z). In the following, we will denote by Ginv any inverse of a given transfer function G and by Σinv = (Ainv , Binv ,Cinv , Xinv ) its canonical realization (see e.g. [Son 76]). Then, we can recall the following results of [CoP84].

162

Giuseppe Conte and Anna Maria Perdon

Proposition 2. Given a left (respectively, right) invertible system, Σ = (A, B,C, X ), with coefficients in the ring R and transfer function matrix GΣ , let Ginv denote a left (respectively, right) inverse of GΣ and let Σinv = (Ainv , Binv ,Cinv , Xinv ) be its canonical realization. Then, the relation Ginv G = 1 induces a monic R[z]-homomorphism ψ : ZΣ → Xinv (respectively, the relation GGinv = 1 induces an epic R[z]-homomorphism ϕ : Xinv → ZΣ ). On the basis of the above proposition, one is interested in the situations in which the R[z]-homomorphism between the zero module of Σ and the state module Xinv of an inverse is, possibly, an isomorphism. Let us assume, for the rest of the section, that the ring R is a principal ideal domain (this holds for the systems considered in the next section, where we deal with time-delay phenomena) and let us denote by K its field of fractions. Then, in general, using the notation of Proposition 2, we can give the following definition. Definition 3 ( [CoP84] Definition 3.4). Given a transfer function matrix GΣ , a left (respectively, right) inverse Ginv of GΣ is said to be an essential inverse if (Xinv /ψ (ZΣ )) ⊗ K = 0 (respectively, if (Kerϕ ) ⊗ K = 0). The above definition reduces to that given in [WyS81] when R is itself a field. Moreover, it implies that the zero module of Σ and the state module Xinv of an essential inverse have the same rank and, in case of a right inverse, they are isomorphic (see [CoP84], Section 3). A more practical result is the following. Proposition 3. Let GΣ = D−1 N be a Bezout factorization. Then, i ) in the case of right invertibility, there exists an essential right inverse Gess of GΣ if and only if N = QN  , where Q is invertible over R(z) and N  is right invertible over R[z]; moreover, Gess = MQ−1 D, where M is a left inverse over R[z] of N  , is a factorization of Gess where M, Q are right coprime and Q−1 D is Bezout; ii ) in the case of left invertibility, there exists an essential right inverse Gess of GΣ if and only if N = N  Q, where Q is invertible over R(z) and N  is left invertible over R[z]; moreover, Gess = Q−1 PD, where P is a left inverse over R[z] of N  , is a Bezout factorization of Gess . The explicit factorizations of essential inverses provided by Proposition 3 point out the relation between the dynamics of the inverses and the R[z]-module structure of ZΣ and, as a consequence, the zero dynamics of Σ . This is of practical interest in many control problems, also because of the following result, which allows us to interpret the notion of zero module in terms of invariant and reachability submodules of the state module. To introduce it, let us recall that, given a system Σ = (A, B,C, X ) with coefficients in R, a controlled invariant submodule (c.i.s.) of X is a submodule V ⊆ X such that A(V ) ⊆ V + ImB (see [CoP98], [CoP00a], [Hau82] and compare with [BaM92], [Won85]). A c.i.s. is said to have the feedback property if there exists an R-morphism F : X → U such that (A + BF)V ⊆ V (the feedback F is called a f riend of V ). There exists, in general, a maximum c.i.s. contained in Ker C, usually denoted by V ∗ , which, under suitable hypothesis, has the feedback property, and a minimum c.i.s. containing ImB ∩ V∗ , usually denoted by R∗ .

Zeros in Linear Time-Delay Systems

163

Proposition 4. Given a system Σ = (A, B,C, X ), with coefficients in the ring R, whose transfer function matrix has a coprime factorization C(zI − A)−1 B = D−1 N, let N(U ⊗ R(z)) be a direct summand of Y ⊗ R(z) and let V ∗ have the feedback property with a friend F. Then, V ∗ /R∗ endowed with the R[z] structure induced by (A + BF) is isomorphic to ZΣ . In other terms, the above results say that, under suitable hypothesis, the zero dynamics of Σ or, equivalently, that of an essential inverse, coincides with the one induced on V ∗ /R∗ by (A + BF). Now, exploiting the above facts, it is possible to introduce an abstract notion of minimum phase system in the ring framework. Recall, to this aim, that, for systems with coefficients in ring R, stability can be abstractly defined by choosing a suitable set H of monic polynomials in R[z], called a Hurwitz set [DaH84], having the following properties: 1. H contains at least one linear monomial z + a with a ∈ R; 2. H is multiplicatively closed; 3. Any factor of an element in H belongs to H. Then, a system Σ = (A, B,C, X ) is said to be H-stable if det(zI − A) belongs to H. Definition 4. A system Σ = (A,B,C,X), with coefficients in a principal ideal domain R, is said to be H-minimum phase if its zero dynamics is H-stable.

3 Time-Delay Systems and Systems over Rings Let the linear, time invariant, time-delay, dynamical system Σd be described by the set of equations  a A x(t − ih) + Σ b B u(t − ih) x(t) ˙ = Σi=0 i i=0 i Σd = c C x(t − ih), y(t) = Σi=0 i where, denoting by ℜ the field of real numbers, x belongs to the state space ℜn , u belongs to the input space ℜn , y belongs to the output space ℜ p , Ai , i = 0, . . . , a; Bi , i = 0, . . . , b; Ci , i = 0, . . ., c; are matrices of suitable dimensions with entries in ℜ and h ∈ ℜ+ is a given delay. Denoting by δ the delay operator defined, for any time function f (t), by δ f (t) = a A δ i ; B(δ ) = Σ b B δ i ; C(δ ) = Σ c C δ i . Then, it is f (t − h), we write A(δ ) = Σi=0 i i=0 i i=0 i possible to substitute formally the delay operator δ with the algebraic indeterminate Δ , thus associating to Σd the system Σ = (A(Δ ), B(Δ ),C(Δ ), X ) defined, over the principal ideal domain R = ℜ[Δ ] of real polynomials in one indeterminate, by the set of equations  x(t + 1) = A(Δ )x(t) + B(Δ )u(t) Σ= y(t) = C(Δ )x(t), where, by abuse of notation, x belongs to the state module X = ℜ[Δ ]n , u belongs to the input module U = ℜ[Δ ]m and y belongs to the output module Y = ℜ[Δ ] p .

164

Giuseppe Conte and Anna Maria Perdon

The two systems Σd and Σ are different objects from the point of view of dynamics and behavior, but, nevertheless, they share some common features. The most important one is represented by the fact that, although input and output variables have different meanings in the two cases, Σd and Σ have the same signal flow graph. This implies that a number of problems concerning the input/output behavior of Σd can be translated into corresponding problems for Σ and that the solution eventually found in the ring framework can be used to derive a solution in the original time-delay context. The main advantages of working in this way derive from the possibility of using finite dimensional algebraic methods and geometric methods to study, indirectly by means of Σ , the properties of Σd . This approach has been proved to be quite effective in a number of papers [CoP97], [CoP98a, b], [CoP99], [CoP00b], [CPI01], [CPG03a], [CPG03b], [CoP05a], [CoP05b] and [PeA06]. In particular, it provides a way to deal with the notion of zero for time-delay systems and, as a consequence, to extend to them, in a suitable way, the results we have recalled in the previous section. More precisely, we will interpret the zero module and the zero dynamics of Σd as the zero module and the zero dynamics of the associated system Σ . Since the property, for two systems, of one being the inverse of the other is preserved when going from the time-delay framework to the ring framework and back, essential inverses of time-delay systems can be defined as those inverse which correspond to essential ones in the ring framework. Moreover, it is possible to deal with the stability of time-delay systems by means of the notion of H-stability, choosing a suitable Hurwitz set H. Actually, one can define H as H = {p(z, Δ ) ∈ ℜ[z, Δ ], such that p(s, e−hs ) = 0 for all s ∈ C with Re(s) ≥ 0} in order to have correspondence between H-stability in the ring framework and asymptotic stability in the usual sense in the time-delay framework. Minimum phase time-delay systems can then be characterized, under the hypothesis pointed out in the previous section, by means of their zero dynamics, which, as we have seen, can be analyzed using geometric methods. Some consequences of these facts on the solution of a number of control problems are described in the following sections.

4 Inversion of Time-Delay Systems The inversion of time-delay systems has been studied, using methods and results from the theory of systems with coefficients in a ring, in [CoP00b] and [CPI01]. Let us consider, for the sake of illustration, only the case of single input/single output (SISO) systems; the extension to more general situations has been worked out in the above mentioned papers. Given a SISO time-delay system Σd described by the set of equations

Zeros in Linear Time-Delay Systems

165



Σd =

a b Ai x(t − ih) + Σi=0 bi u(t − ih) x(t) ˙ = Σi=0 c y(t) = Σi=0 ci x(t − ih)

(2)

and the corresponding SISO system Σ , defined, over the principal ideal domain R = ℜ[Δ ], by the set of equations  x(t + 1) = A(Δ )x(t) + b(Δ )u(t) (3) Σ= y(t) = c(Δ )x(t), one can apply the Silverman Inversion Algorithm in the ring framework by evaluating recursively y(t + 1), y(t + 2), . . . . Since the general formula (where Δ has been omitted for simplicity) yields k−1 k−i−1 y(t + k) = cAk x(t) + Σi=0 cA bu(t + i),

either cAk−1 b = 0 for all k ≥ 1, or there exists k0 , necessarily lesser than or equal to dim X, such that cAk−1 b = 0 for all k < k0 and cAk0 −1 b = 0. In this case, one gets y(t + 1) = cAx(t) y(t + 2) = cA2 x(t) .. .

y(t + k + 0) = cAk0 x(t) + cAk0−1 bu(t). It is clear, then, that Σ is invertible if and only if cAK0 −1 b = c(Δ )A(Δ )k0 −1 b(Δ ) is an invertible element of R = ℜ[Δ ]. If this is the case, an inverse in the ring framework is given by the system  z(t + 1) = (A − b(cAk0 −1 b)−1 cAk0 )z(t) + b(cAk0−1 b)−1 y(t + k0) Σinv = u(t) = −(cAk0 −1 b)−1 cAk0 z(t) + (cAk0 −1 b)−1 y(t + k0 ). From the expression of the inverse in the ring framework it is easy to derive that of an inverse in the original time-delay framework. The defining matrices of this will simply be obtained by substituting the algebraic indeterminate Δ with the delay operator δ . Now, following [CoP00b], we introduce the following definition. Definition 5. Given the SISO system Σ = (A, b, c, X ) with coefficients in the ring R, assume that there exists k0 , such that cAk−1 b = 0 for k < k0 and cAk0 −1 b = 0. Then, we say that Σ has finite relative degree equal to k0 . If, in addition, cAk0 −1 b is an invertible element of R, we say that the relative degree is pure. Alternatively, if cAk−1 b = 0 for all k ≥ 1, we say that Σ has no finite relative degree. The invertibility of Σ is therefore characterized by the fact that the system has pure, finite relative degree. If we write the transfer function GΣ of Σ as GΣ = c(zI − A)−1 b = d(z)−1 n(z), where d(z), n(z) are polynomials in R[z], that is they are polynomials in the indeterminate z with coefficients in R, and d(z) is monic, it is not difficult to see that

166

Giuseppe Conte and Anna Maria Perdon

the leading coefficient of n(z) is just cAk0 −1 b. If Σ has pure relative degree k0 , then the zero module ZΣ of Σ is isomorphic, for the results recalled in Section 2, to the torsion R[z]-module R[z]/n(z)R[z]. In this case, the notion of H-minimum phase system discussed at the end of Section 2 can be easily characterized in terms of the numerator polynomial n(z). Invertible H-minimum phase SISO, time-delay systems have H-stable essential inverses. Example 1. Assuming that in GΣ one has n(z) = Δ z, no matter what d(z) is, the system Σ turns out to have finite relative degree, but it is not invertible. Computation yields ZΣ = (R, 0) where 0 : R → R is the null morphism and, choosing H as in Section 2, the zero dynamics is not H-stable. Extension of the Silverman Inversion Algorithm and of the above discussion to the multi input/multi output (MIMO) case is possible due to the abstract, algebraic nature of our arguments (see [CoP00b] and [CPI01]). The inverse system Σinv constructed by means of the Silverman Inversion Algorithm, when it exists, is not essential. In fact, its dimension is the same as that of the system Σ , which is equal to the degree of the denominator polynomial d(z); while the dimension of ZΣ , being equal to the degree of the numerator polynomial n(z), is smaller than that. Reducing Σinv to an essential inverse, in general, may be complicated, so, to construct an essential inverse, it is preferable to make use of the explicit decomposition given in Proposition 3. Example 2. Assuming that the transfer function of a given n-dimensional, SISO system Σ , with coefficients in the ring ℜ[Δ ], is GΣ = d(z)−1 n(z) with n(z) = c0 + c1z + · · · + cn−1 zn−1 and cn−1 = 0, the relative degree of Σ is 1 and cAk0 −1 b = cb = cn−1 . If cn−1 is invertible, it is not difficult to see, using a suitable realization of Σ , that the characteristic polynomial of the dynamical matrix of Σinv is p(z) = (cn−1 )−1 (c0 + c1z + · · · + cn−1 zn−1 )z. From the above results and discussion, it follows that the use of inversion as a synthesis procedure in the framework of time-delay systems can be dealt with, in connection with the issue of stability of inverses, by studying zero modules and zero dynamics.

5 Tracking Problems for Time-Delay Systems Given a SISO time-delay system Σd described by the set of equations (2) and the corresponding system Σ , defined, over the principal ideal domain R = ℜ[Δ ], by the set of equations (3), let us consider the problem of designing a compensator which forces Σd to track a reference signal r(t) (see [CPM07]). Working in the ring framework, we consider the extended system  x(t + 1) = Ax(t) + bu(t) ΣE = (4) e(t) = cx(t) − r(t)

Zeros in Linear Time-Delay Systems

167

whose output is the tracking error and, assuming that Σ has pure relative degree k0 , we apply the Silverman Inversion Algorithm. This gives the following relation: e(t + k0 ) = cAk0 x(t) + cAk0 −1 bu(t) − r(t + k0 ). k0 −1 ai zi in such a way that it is in Then, choosing a real polynomial p(z) = zk0 + Σi=0 the Hurwitz set H, we can construct the compensator ⎧ ⎪ ⎨z(t + 1) = Az(t) + bu(t) Σc = u(t) = −(cAk0 −1 b)−1 (cAk0 z(t) − r(t + k0))− ⎪ ⎩ k0 −1 −(cAk0 −1 b)−1 Σi=0 ai e(t + i)

whose action on Σ causes the error to evolve according to the following equation: k −1

0 ai e(t + i) = cAk0 (x(t) − z(t)). e(t + k0) = Σi=0

(5)

H-stability of the compensator is of course a key issue and, since its construction is based on inversion, it can be dealt with as for inverses. If Σ and hence ΣE are H-minimum phase (that is, their zero dynamics is H-stable), an H-stable reduced compensator can be obtained by means of an essential inverse. Then, as in the case of inversion, it is easy to derive a compensator in the timedelay framework from Σc . Its defining matrices are obtained from those of Σc by substituting the algebraic indeterminate Δ with the delay operator δ . Due to (5) and to the corresponding relation in the time-delay framework, the tracking error behaves in a desired way. More precisely, when the initial conditions for Σ are known and the compensator can be initialized accordingly, the compensated system asymptotically tracks the reference signal. If the correct initialization is not possible, if Σ is globally asymptotically stable, the tracking error can be made arbitrarily small for t sufficiently large (see [CPM07]). Extension to the MIMO case is possible (see [CPM07]), although the situation complicates and the results are relatively weaker.

6 Conclusion In this chapter, the notions of zero module and of zero dynamics for systems with coefficients in a ring have been reviewed. These have been shown to be of interest in the study of control problems involving time-delay systems.

References [BaM92] G. Basile, G. Marro, Controlled and Conditioned Invariants in Linear System Theory, Prentice-Hall, New York (1992) [BBV86] J. W. Brewer, J. W. Bunce and F. S. Van Vleck, Linear Systems Over Commutative Rings, Marcel Dekker, New York (1986)

168

Giuseppe Conte and Anna Maria Perdon

[CoP84]

[CoP97]

[CoP98a]

[CoP98] [CoP99]

[CoP00a] [CoP00b]

[CPI01]

[CPG03a]

[CPG03b]

[CoP05a]

[CoP05b]

[CPM07]

[DaH84]

[Hau82] [PGC03]

[PeA06]

G. Conte, A. M. Perdon, An algebraic notion of zeros for systems over rings, Proc. Int. Symp.MTNS’83 (Beer Sheva - Israele, 1983), Lecture Notes in Control and Information Science, Springer-Verlag, vol. 58 (1984) pp 166–182 G. Conte, A.M. Perdon, Noninteracting control problems for delay-differential systems via systems over rings, Journal Europ`een des Syst´emes Automatis´es, 31, (1997) G. Conte, A.M. Perdon and A. Lombardo, Dynamic feedback decoupling problem for delay-differential systems via systems over rings, Mathematics and Computers in Simulation, 1462, (1998) G. Conte, A.M. Perdon, The geometric approach for systems over rings, Proceedings 37th IEEE Conference on Decision and Control, Tampa, FL (1998) G. Conte, A. M. Perdon, Disturbance decoupling with stability for delay differential systems, Proc. 38th IEEE Conference on Decision and Control, Phoenix, AZ (1999) G. Conte, A. M. Perdon, Systems over Rings: Geometric Theory and Applications, Annual Review in Control, no. 24, (2000) G. Conte, A. M. Perdon, Invertibility and Inversion for Systems over Rings and Applications to Delay-differential Systems, Proc. 39th IEEE Conference on Decision and Control, Sydney, Australia (2000) G. Conte, A.M. Perdon and R. Iachini, Inversion Problems for Time-delay Systems via Systems over Rings, Proc. IFAC Symposium on System Structure and Control, Prague, Czech Republic (2001) G. Conte, A. M. Perdon and G. Guidone-Peroli, The Fundamental Problem of Residual Generation for Linear Time Delay Systems, Proc. IFAC Workshop on Linear Time Delay Systems LTDS03, Paris, France (2003) G. Conte, A. M. Perdon and G. Guidone-Peroli, Unknown Input Observer for Linear Delay Systems: a Geometric Approach, Proc. 42th IEEE Conference on Decision and Control, Maui, Hawaii (2003) G. Conte, A.M. Perdon, Unknown Input Observer and Residual Generators for Linear Time Delay Systems, in “Current Trends in Nonlinear Systems and Control”, L. Menini, L. Zaccarian, and C.T. Abdallah Eds, Birkhauser, Boston, MA (2005) G. Conte, A.M. Perdon, Modeling Time-delay Systems by Means of Systems with Coefficients in a Ring, Proc. Workshop on Modeling and Control of Complex Systems, Ayia Napa, Cyprus (2005) G. Conte, A.M. Perdon and C.H. Moog, Inversion and Tracking Problems for Time Delay Linear Systems, in “Applications of Time-Delay Systems” , LNCIS 352, J. Chiasson and J.J. Loiseau Eds., Springer-Verlag (2007) pp 267–284 B. Datta, M. L. Hautus, Decoupling of multivariable control systems over unique factorization domains, SIAM Journal on Control and Optimization, 22, 1 (1984), pp 28–39 M.L. Hautus, Controlled invariance in systems over rings, Springer Lecture Notes in Control and Information Science, vol. 39 (1982) A.M. Perdon, G. Guidone-Peroli and M. Caboara, Algorithms for geometric control of systems over rings, Proc. 2nd IFAC Conference Control Systems (CSD’03), Bratislava, Slovak Republic (2003) A. M. Perdon, M. Anderlucci, Un g´en´nrateur de r´esidus pour syst´emes a´ retard, Proc. Conf´erence Internationale Francophone d’Automatique, CIFA 2006, Bordeaux, France (2006)

Zeros in Linear Time-Delay Systems

169

[PAC06] A.M. Perdon, M. Anderlucci and M. Caboara, Effective computations for geometric control theory, International Journal of Control, vol. 79, no. 11 (2006), pp 1401–1417 [SCS89] C. B. Schrader, M. K. Sain, Research on System Zeros: A Survey, International Journal of Control, vol. 50, (1989) pp 1407–1433 [Son76] E.D. Sontag, Linear systems over commutative rings: A survey, Ricerche di Automatica, 7 (1976) pp 1–34 [Son81] E.D. Sontag, Linear systems over commutative rings: a (partial) updated survey, Control science and technology for the progress of society, Vol. 1 (Kyoto, 1981), pages 325–330. [Won85] W. M. Wonham, Linear Multivariable Control: A Geometric Approach, 3rd Ed., Springer-Verlag, Berlin (1985) [WyS81] B. Wyman, M. K. Sain, The Zero Module and Essential Inverse Systems, IEEE Trans. Circuits and Systems, CAS-28, February (1981), 112–126

Part III

Dynamic Systems Characteristics

On the Status of the Stability Theory of Discontinuous Dynamical Systems Anthony N. Michel1 and Ling Hou2 1 2

Dept. of Electrical Engineering, University of Notre Dame, Notre Dame, IN 46556, USA. [email protected] Dept. of Electrical and Computer Engineering, St. Cloud State University, St. Cloud, MN, 56301, USA. [email protected]

Summary. Dynamical systems can be classified in a variety of ways. Thus, when the time set T = R+ = [0, ∞) we speak of a continuous-time dynamical system and when T = N = {0, 1, 2, · · · } we speak of a discrete-time dynamical system. When the state space X is a finite dimensional linear space, we speak of a finite dimensional dynamical system, and otherwise, of an infinite dimensional dynamical system. When all the motions in a continuous-time dynamical system are continuous with respect to time, we speak of a continuous dynamical system and when at least one of the motions in a continuous-time dynamical system is not continuous with respect to time, we speak of a discontinuous dynamical system (DDS). Continuous dynamical systems may be viewed as special cases of DDSs. The stability analyses of continuous dynamical systems and discrete-time dynamical systems constitute mature subjects. This is not the case for the DDS. Such systems arise in the modeling process of a variety of systems, including hybrid dynamical systems, discrete-event systems, switched systems, systems subjected to impulse effects, and the like. The qualitative analysis of such systems has been of great interest over the past decades. In this chapter we will give an overview of the stability results of the DDS with an emphasis on the authors’ work, along the lines indicated below. Due to space limitations we will present only sample results (concerning uniform asymptotic stability of invariant sets for dynamical systems defined on metric spaces). a) We first establish sufficient conditions for uniform asymptotic stability for DDSs. b) Next, we show that the classical Lyapunov theorem for uniform asymptotic stability for continuous dynamical systems reduces to our uniform asymptotic stability result for DDSs. By using a specific example, we show that the converse to this result is in general not true. We also prove these assertions for discrete-time dynamical systems. To accomplish this, we associate with every discrete-time dynamical system a DDS having identical stability properties. c) The results discussed above constitute sufficient conditions. Next, we show that under some additional mild conditions, these results yield necessary conditions as well (converse theorems) and we identify conditions under which the involved Lyapunov functions are continuous. In addition to proving that our DDS stability results are in general less conservative than the corresponding classical Lyapunov stability results for continuous dynamical systems and discrete-time dynamical systems, we establish here a unifying framework for the stability analysis of continuous dynamical systems, discrete-time dynamical systems, and C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 9, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

174

Anthony N. Michel and Ling Hou

DDSs. Finally, we also point to several references describing applications of the results addressed herein.

1 Dynamical Systems A dynamical system is a four-tuple {T, X, A, S} where T ⊂ R denotes time set (R = (−∞, ∞)); X is the state space (a metric space (X, d) with metric d); A ⊂ X is the set of initial states; and S denotes a family of motions: for any fixed a ∈ A, t0 ∈ T , a mapping p(·, a,t0 ) : Ta,t0 → X is called a motion if p(t0 , a,t0 ) = a where Ta,t0 = [t0 ,t1 ) ∩ T , t1 > t0 , and t1 is finite or infinite, and S is a family of such motions. Thus, S is a subset of the set ∪(a,t0 )∈A×T {Ta,t0 → X } (1) and for any p(·, a,t0 ) ∈ S, we have p(t0 , a,t0 ) = a. Note that in general, for each (a,t0 ) ∈ A × T we allow more than one Ta,t0 to exist in (1) (i.e., we allow more than one motion to initiate from a given pair (a,t0 )). When T = R+ = [0, ∞) we speak of a continuous-time dynamical system and when T = N = {0, 1, 2, · · ·} we speak of a discrete-time dynamical system. When X is a finite dimensional normed linear space, we speak of a finite dimensional dynamical system, and otherwise, of an infinite dimensional dynamical system. Also, when all motions in a continuous-time dynamical system are continuous with respect to t (t ∈ Ta,t0 ), we speak of a continuous dynamical system and when one or more of the motions are not continuous with respect to t, we speak of a discontinuous dynamical system (DDS). Finite dimensional dynamical systems may be determined, e.g., by the solutions of ordinary differential equations, ordinary differential inequalities, ordinary difference equations, and ordinary difference inequalities. In the former two cases we have T = R+ while in the latter two cases T = N. In all cases, X = Rn (the real n-space) and the metric d is determined by any one of the equivalent norms | ·| on Rn . Infinite dimensional dynamical systems arise in a variety of ways. For example, they may be determined by the solutions of differential-difference equations and Δ functional differential equations. In these cases T = R+ , X = C([−r, 0], Rn ) = Cr , r > 0 (the space of real continuous functions from the interval [−r, 0] to Rn ) with metric determined by the norm defined by ϕ  = sup−r≤ϕ ≤0 |ϕ (ϕ )| for all ϕ ∈ C([−r, 0], Rn ). They may also be determined by the solutions of Volterra integrodifferential equations. In this case, T = R+ , X = C((−∞, 0], Rn ) (the fading memory space) with metric determined by one of several possible fading memory norms. They may also be determined by the solutions of initial-value/boundaryvalue problems determined by several classes of partial differential equations, formulated on appropriate Sobolev spaces, or more generally, by the solutions of differential equations and inclusions on Banach spaces. Further, they may be determined by C0 -semigroups and nonlinear semigroups defined on Banach and Hilbert spaces. Additionally, there are finite dimensional as well as infinite dimensional dynamical systems whose motions cannot be determined by classical equations or inequalities

Stability Theory of Discontinuous Dynamical Systems

175

of the type enumerated above (e.g., certain classes of discrete-event systems whose motions are characterized by Petri nets, Boolean logic elements, and the like). Such systems frequently can not be described on linear spaces, and this is one of the reasons for formulating dynamical systems on metric spaces. Note that any qualitative results obtained for dynamical systems defined on metric space are general enough to be applicable to any of the classes of systems enumerated above. Although the stability analysis of discrete-time dynamical systems and continuous dynamical systems is still a very active area of research, it must be regarded as a mature subject (see, e.g. [Zub64, Hah67, MWH01]). This is not the case for discontinuous dynamical systems (DDSs). DDSs arise in the modeling process of a variety of systems, including hybrid dynamical systems, discrete-event systems, switched systems, intelligent control systems, systems subjected to impulse effects, and the like. Most of the stability results for DDSs that have been established thus far concern finite dimensional systems (e.g. [MWH01], [YMH98]–[LSW05]). However, stability results for general DDSs defined on metric spaces have also been established [YMH98]– [Mic99], as well as results for special classes of infinite dimensional dynamical systems [SMZ05]– [MS06]. The stability theory established in [YMH98] has been applied in the analysis of several important classes of dynamical systems (see, e.g., [HMY97]–[HM01]). Here we give an overview of some of our work on the stability analysis of the DDS. Most of this work involves results concerning uniform stability, uniform asymptotic stability, exponential stability, uniform asymptotic stability in the large, exponential stability in the large, instability and complete instability (in the sense of Lyapunov), as well as uniform boundedness and uniform ultimate boundedness of motions of dynamical systems (Lagrange stability). Due to space limitations, we will address here only results concerning uniform stability, uniform asymptotic stability, and uniform asymptotic stability in the large. For the results concerning the other types of stability and boundedness enumerated above, the reader should refer to [MWH01]– [Mic99]. The remainder of this chapter is organized as follows. In Section 2 we will define uniform stability, uniform asymptotic stability and uniform asymptotic stability in the large and we will state the classical Lyapunov results for these stability types for continuous dynamical systems and discrete-time dynamical systems. In Section 3 we state and prove results for uniform stability, uniform asymptotic stability, and uniform asymptotic stability in the large for DDSs. We note that these results are also applicable to continuous dynamical systems, since such systems may be viewed as special cases of DDSs. In Sections 4 and 5 we show that the classical Lyapunov stability results for continuous dynamical systems (presented in Section 2) are in general more conservative than the corresponding results for the DDS (presented in Section 3). To accomplish this we first show that the classical Lyapunov stability results (of Section 2) will always reduce to the corresponding stability results for DDS (given in Section 3). Next, by using a specific example, we show in Section 5 that converses to these statements are in general not true. In Section 4 we also show that the classical Lyapunov stability results for discrete-time dynamical systems (presented in Section 2) are in general more conservative than the corresponding results

176

Anthony N. Michel and Ling Hou

for the DDS (presented in Section 3). To accomplish this, we associate with every discrete-time dynamical system a DDS, having identical stability properties. The stability results for the DDS presented in Section 3 constitute sufficient conditions. In Section 6 we show that under some additional mild conditions, these conditions also constitute necessary conditions (Converse Theorems) and in particular, we identify conditions under which the involved Lyapunov functions are continuous.

2 Background Material In the present section we first introduce some qualitative characterizations of dynamical systems. Next, we recall some of the existing classical stability results for continuous dynamical systems and discrete-time dynamical systems. 2.1 Some Qualitative Characterizations In the following definitions, T = R+ or T = N. Definition 1. Let {T, X, A, S} be a dynamical system. A set M ⊂ A is said to be invariant with respect to system S (or more compactly, (S, M) is invariant), if a ∈ M implies that p(t, a,t0 ) ∈ M for all t ∈ Ta,t0 , all t0 ∈ T , and all p(·, a,t0 ) ∈ S. If x0 ∈ A and M = {x0 }, we call x0 an equilibrium (point) of the dynamical system. In the following definitions, instead of stating that, e.g., “the set M which is invariant with respect to S is stable,” we will more compactly say that “(S, M) is stable.” Also, recall that the distance between a point a and a set M in a metric space (X, d) is defined as d(a, M) = inf d(a, x). x∈M

Definition 2. (S, M) is said to be stable if for every ε > 0 and every t0 ∈ T , there exists a δ = δ (ε ,t0 ) > 0 such that d(p(t, a,t0 ), M) < ε for all t ∈ Ta,t0 and for all p(·, a,t0 ) ∈ S, whenever d(a, M) < δ . (S, M) is said to be uniformly stable if in the above definition δ is independent of t0 (i.e., δ = δ (ε )). Throughout this paper, when dealing with asymptotic properties of motions, we will assume that for any (a,t0 ) ∈ A × R+ , Ta,t0 = [t0 , ∞) ∩ T and that T ∩ [α , ∞) = 0/ for any α > 0. Definition 3. (S, M) is said to be attractive if there exists an η = η (t0 ) > 0 such that limt→∞ d(p(t, a,t0 ), M) = 0 for all p(·, a,t0 ) ∈ S whenever d(a, M) < η . (S, M) is asymptotically stable if it is stable and attractive. Definition 4. (S, M) is uniformly asymptotically stable if it is (i) uniformly stable, and (ii) uniformly attractive, i.e., for every ε > 0 and every t0 ∈ R+ , there exist a δ > 0 independent of t0 and ε , and a τ = τ (ε ) > 0 independent of t0 , such that d(p(t, a,t0 ), M) < ε for all t ∈ Ta,t0 +τ and all p(·, a,t0 ) ∈ S whenever d(a, M) < δ .

Stability Theory of Discontinuous Dynamical Systems

177

Definition 5. (S, M) is uniformly asymptotically stable in the large if (i) (S, M) is uniformly stable, and (ii) for every α > 0, for every ε > 0, and for every t0 ∈ T , there exists a τ = τ (ε , α ) > 0, such that if d(a, M) < α , then for all p(·, a,t0 ) ∈ S, d(p(t, a,t0 ), M) < ε for all t ∈ Ta,t0 +τ . 2.2 Some Classical Lyapunov Stability Results We now recall the classical Lyapunov stability results concerning uniform stability, uniform asymptotic stability and uniform asymptotic stability in the large for continuous dynamical systems and discrete-time dynamical systems. The proofs of these results may be found in [Zub64]– [MWH01]. In Section 3 we will state and prove corresponding results for the DDS and in Section 4 we will show that the results of the present section reduce to the results of Section 3. (This approach presents an alternative way of proving the results of the present section.) We will make use of functions of class K and class K∞ . A function ϕ ∈ C[[0, r1 ], R+ ] (resp., ϕ ∈ C[R+ , R+ ]) belongs to class K (i.e., ϕ ∈ K ), if ϕ (0) = 0 and if ϕ is strictly increasing on [0, r1 ] (resp., on R+ ). We say that a function ϕ ∈ K defined on R+ belongs to class K∞ if limr→∞ ϕ (r) = +∞. Also, a continuous function σ : [s1 , ∞) → R+ is said to belong to class L if σ is strictly decreasing on [s1 , ∞) and lims→∞ σ (s) = 0, where s1 ∈ R+ . Theorem 1 (Continuous Dynamical Systems). Let {R+ , X , A, S} be a continuous dynamical system, and let M ⊂ A be closed. Assume that there exist a function V : X × R+ → R+ and two functions ϕ1 , ϕ2 ∈ K defined on R+ such that

ϕ1 (d(x, M)) ≤ V (x,t) ≤ ϕ2 (d(x, M))

(2)

for all x ∈ X and t ∈ R+ . Assume that there exists a neighborhood U of M such that for all a ∈ U and all p(·, a,t0 ) ∈ S, V (p(t, a,t0 ),t) is nonincreasing for all t ≥ t0 , t ∈ R+ . Then (S, M) is invariant and uniformly stable. Theorem 2 (Discrete-time Dynamical Systems). Let {N, X , A, S} be a discrete-time dynamical system, and let M ⊂ A be closed. Assume that there exist a function V : X × N → R+ and two functions ϕ1 , ϕ2 ∈ K defined on R+ such that

ϕ1 (d(x, M)) ≤ V (x, n) ≤ ϕ2 (d(x, M))

(3)

for all x ∈ X and n ∈ N. Assume that there exists a neighborhood U of M such that for all a ∈ U and all p(·, a, n0 ) ∈ S, V (p(n, a, n0 ), n) is nonincreasing for all n ≥ n0 , n ∈ N. Then (S, M) is invariant and uniformly stable. Theorem 3 (Continuous Dynamical Systems). If in addition to the assumptions given in Theorem 1, there exists a function ϕ3 ∈ K defined on R+ such that for all a ∈U and all p(·, a,t0 ) ∈ S the upper right-hand Dini derivative D+V ((t, a,t0 ),t) satisfies

178

Anthony N. Michel and Ling Hou

D+V (p(t, a,t0 ),t) ≤ −ϕ3 (d(p(t, a,t0 ), M)) for all t ≥ t0 ,t ∈

R+ ,

(4)

then (S, M) is uniformly asymptotically stable.

Theorem 4 (Discrete-Time Dynamical Systems). If in addition to the assumptions given in Theorem 2 there exists ϕ3 ∈ K defined on R+ , such that for all a ∈ U and all p(·, a, n0 ) ∈ S, V (p(n + 1, a, n0), n + 1) − V(p(n, a, n0 ), n) ≤ −ϕ3 (d(p(n, a, n0 ), M))

(5)

for all n ≥ n0 , n ∈ N, then (S, M) is uniformly asymptotically stable. Theorem 5 (Continuous Dynamical Systems). Let {R+ , X , A, S} be a continuous dynamical system, and let M ⊂ A be closed and bounded. Assume that there exist a function V : X × R+ → R+ and two functions ϕ1 , ϕ2 ∈ K∞ such that

ϕ1 (d(x, M)) ≤ V (x,t) ≤ ϕ2 (d(x, M))

(6)

for all x ∈ X and t ∈ R+ . Assume that there exists a function ϕ3 ∈ K defined on R+ such that for all a ∈ A and all p(·, a,t0 ) ∈ S, V (p(t, a,t0 ),t) is continuous for all t ∈ Rt+0 and the upper righthand Dini derivative D+V ((t, a,t0 ),t) satisfies D+V (p(t, a,t0 ),t) ≤ −ϕ3 (d(p(t, a,t0 ), M))

(7)

for all t ∈ Rt+0 . Then (S, M) is uniformly asymptotically stable in the large. Theorem 6 (Discrete-Time Dynamical Systems). Let {N, X , A, S} be a discretetime dynamical system, and let M ⊂ A be closed and bounded. Assume that there exist a function V : X × R+ → R+ and two functions ϕ1 , ϕ2 ∈ K∞ such that

ϕ1 (d(x, M)) ≤ V (x, n) ≤ ϕ2 (d(x, M))

(8)

for all x ∈ X and n ∈ N. Assume that there exists ϕ3 ∈ K defined on R+ , such that for all a ∈ A and all p(·, a, n0 ) ∈ S, V (p(n + 1, a, n0), n + 1) − V(p(n, a, n0 ), n) ≤ −ϕ3 (d(p(n, a, n0 ), M))

(9)

for all n ≥ n0 , n ∈ N. Then (S, M) is uniformly asymptotically stable in the large.

3 Some Stability Results for DDS In the present section we state and prove results for the DDS concerning uniform stability, uniform asymptotic stability, and uniform asymptotic stability in the large.

Stability Theory of Discontinuous Dynamical Systems

179

These results along with results for exponential stability, exponential stability in the large, uniform boundedness, uniform ultimate boundedness, and instability were originally established in [YMH98]. Theorem 7 (DDS). Let {R+ , X , A, S} be a dynamical system, and let M ⊂ A be closed. Assume that there exist a function V : X × R+ → R+ and two functions ϕ1 , ϕ2 ∈ K defined on R+ such that

ϕ1 (d(x, M)) ≤ V (x,t) ≤ ϕ2 (d(x, M))

(10)

for all x ∈ X and t ∈ R+ . Assume that for any p(·, a,t0 ) ∈ S, V (p(t, a,t0 ),t) is continuous everywhere on Δ

Rt+0 except on an unbounded, discrete subset E = {t1 ,t2 , · · · } of Rt+0 = {t ≥ t0 }, where tn is strictly increasing (E depends on p). Also, assume that there exists a neighborhood U of M such that for all a ∈ U and all p(·, a,t0 ) ∈ S, V (p(tn , a,t0 ),tn ) is nonincreasing for n ∈ N. Furthermore, assume that there exists f ∈ C[R+ , R+ ], independent of p ∈ S, such that f (0) = 0 and such that V (p(t, a,t0 ),t) ≤ f (V (p(tn , a,t0 ),tn ))

(11)

for t ∈ (tn ,tn+1 ), n ∈ N. Then (S, M) is invariant and uniformly stable. Proof. We first prove that (S, M) is invariant. If a ∈ M, then V (p(t0 , a,t0 ),t0 ) = 0 since V (p(t0 , a,t0 ),t0 ) = V (a,t0 ) ≤ ϕ2 (d(a, M)) = 0 and d(a, M) = 0. Therefore we know that V (p(tn , a,t0 ),tn ) = 0 for all n ≥ 0, and furthermore V (p(t, a,t0 ),t) = 0 for all t ≥ t0 since V (p(t, a,t0 ),t) ≤ f (V (p(tn , a,t0 ),tn )). It is then implied that p(t, a,t0 ) ∈ M for all t ≥ t0 . Therefore (S, M) is invariant by definition. Since f is continuous and f (0)= 0, then for any ε > 0 there exists a δ = δ (ε )> 0 such that f (y) < ϕ1 (ε ) as long as 0 ≤ y < δ . We can assume that δ ≤ ϕ1 (ε ). Thus for any motion p(·, a,t0 ) ∈ S, as long as the initial condition d(a, M) < ϕ2−1 (δ ) is satisfied, then V (p(t0 , a,t0 ),t0 ) = V (a,t0 ) ≤ ϕ2 (d(a, M)) < ϕ2 (ϕ2−1 (δ )) = δ and V (p(tn , a,t0 ),tn ) < δ for n = 1, 2, · · · , since V (p(tn , a,t0 ),tn ) is nonincreasing. Furthermore, for any t ∈ (tn ,tn+1 ) we can conclude that V (p(t, a,t0 ),t) ≤ f (V (p(tn , a,t0 ),tn )) < ϕ1 (ε ) and

d(p(t, a,t0 ), M) ≤ ϕ1−1 (V (p(t, a,t0 ),t)) < ϕ1−1 (ϕ1 (ε )).

Therefore, by definition, (S, M) is uniformly stable.

2

180

Anthony N. Michel and Ling Hou

Theorem 8 (DDS). If in addition to the assumptions given in Theorem 7 there exists ϕ3 ∈ K defined on R+ , such that for all a ∈ U, all p(·, a,t0 ) ∈ S, and all n ∈ N, DV (p(tn , a,t0 ),tn ) ≤ −ϕ3 (d(p(tn , a,t0 ), M)),

(12)

where Δ

DV (p(tn , a,t0 ),tn ) =

1 [V (p(tn+1 , a,t0 ),tn+1 ) − V (p(tn , a,t0 ),tn )], tn+1 − tn

(13)

then (S, M) is uniformly asymptotically stable. Proof. For any a ∈ U and any p(·, a,t0 ) ∈ S, letting zn = V (p(tn , a,t0 ),tn ), n ∈ N, we obtain from the assumptions of the theorem that zn+1 − zn ≤ −(tn+1 − tn )ϕ3 ◦ ϕ2−1(zn ), for n ∈ N. If we denote ϕ = ϕ3 ◦ ϕ2−1 , then ϕ ∈ K and the above inequality becomes zn+1 − zn ≤ −(tn+1 − tn )ϕ (zn ). Since {zn } is nonincreasing and ϕ3 ∈ K , it follows that zk+1 − zk ≤ −ϕ (zk )(tk+1 − tk ) ≤ −ϕ (zn )(tk+1 − tk ) for all k ≤ n. We thus obtain that zn+1 − z0 ≤ −(tn+1 − t0 )ϕ (zn ), for all n ≥ 0. It follows that

ϕ (zn ) ≤

z0 − zn+1 z0 ≤ . tn+1 − t0 tn+1 − t0

(14)

Now consider a fixed δ > 0. For any given ε > 0, we can choose a τ > 0 such that

       −1 −1 ϕ2 (δ ) −1 −1 ϕ2 (δ ) max ϕ1 ϕ , ϕ1 f ϕ <ε (15) τ τ since ϕ1 , ϕ2 , ϕ ∈ K and f (0) = 0. For any a ∈ A with d(a, M) < δ and any t0 ∈ R+ 0, we are now able to show that d(p(t, a,t0 ), M) < ε whenever t ≥ t0 + τ . The above statement is true because for any t ≥ t0 + τ , t must belong to some interval [tn ,tn+1 ) for some n ∈ N, i.e., t ∈ [tn ,tn+1 ). Therefore we know that tn+1 −t0 > τ . It follows from (14) that ϕ2 (δ ) ϕ (zn ) ≤ , τ

Stability Theory of Discontinuous Dynamical Systems

181

which implies that V (p(tn , a,t0 )) = zn ≤ ϕ and

−1



 ϕ2 (δ ) , τ

   ϕ2 (δ ) V (p(t, a,t0 )) ≤ f ϕ −1 τ

(16)

(17)

if t ∈ (tn ,tn+1 ). It follows from (16) that d(p(tn , a,t0 ), M) < ε , noticing that (15) holds. In the case when t ∈ (tn ,tn+1 ), we can conclude from (17) that d(p(t, a,t0 ), M) < ε . This proves that (S, M) is uniformly asymptotically stable. 2 Theorem 9 (DDS). Let {R+ , X , A, S} be a dynamical system, and let M ⊂ A be closed and bounded. Assume that there exist a function V : X × R+ → R+ and two functions ϕ1 , ϕ2 ∈ K∞ such that

ϕ1 (d(x, M)) ≤ V (x,t) ≤ ϕ2 (d(x, M))

(18)

for all x ∈ X and t ∈ R+ . Assume that for any p(·, a,t0 ) ∈ S, V (p(t, a,t0 ),t) is continuous everywhere on Rt+0 except on an unbounded, discrete subset E = {t1 ,t2 , · · · } of Rt+0 , where {tn } is strictly increasing and E depends on p. Furthermore, assume that there exists a function f ∈ C[R+ , R+ ] with f (0) = 0 such that for any p(·, a,t0 ) ∈ S, V (p(t, a,t0 ),t) ≤ f (V (p(tn , a,t0 ),tn ))

(19)

for t ∈ (tn ,tn+1 ), n ∈ N. Assume that there exists a function ϕ3 ∈ K defined on R+ , such that for any p(·, a,t0 ) ∈ S, DV (p(tn , a,t0 ),tn ) ≤ −ϕ3 (d(p(tn , a,t0 ), M)), (20) n ∈ N, where DV (p(tn , a,t0 ),tn ) is given in (13). Then (S, M) is uniformly asymptotically stable in the large. Proof. It follows from Theorem 7 that under the present hypotheses, M is an invariant set of S and (S, M) is uniformly stable. We need to show that condition (ii) in Definition 5 is also satisfied. Consider arbitrary α > 0, ε > 0, t0 ∈ R+ , and a ∈ A such that d(a, M) < α . Let zn = V (p(tn , a,t0 ),tn ) and z(t) = V (p(t, a,t0 ),t), and let ϕ = ϕ3 ◦ ϕ2−1 . Using the same argument as that in the proof of Theorem 8, we obtain

ϕ (zn ) ≤

z0 − zn z0 ≤ . tn+1 − t0 tn+1 − t0

Let γ1 = γ1 (ε , α ) = ϕ2 (α )/ϕ (ϕ1 (ε )) > 0 and choose a δ > 0 such that maxr∈[0,δ ] f (r) < ϕ1 (ε ). Let γ2 = ϕ2 (α )/ϕ (δ ) and γ = max{γ1 , γ2 }. For any a ∈ A

182

Anthony N. Michel and Ling Hou

with d(a, M) < α and any t0 ∈ R+ , we are now able to show that d(p(t, a,t0 ), M) < ε whenever t ≥ t0 + γ . The above statement is true because for any t ≥ t0 + γ , t must belong to some interval [tn ,tn+1 ) for some n ∈ N, i.e., t ∈ [tn ,tn+1 ). Therefore we know that tn+1 − t0 > γ and that

ϕ (zn ) ≤

ϕ2 (α ) z0 < , γ γ

which implies that V (p(tn , a,t0 ),tn ) = zn < ϕ −1



ϕ2 (α ) γ



  ≤ min ϕ1 (ε ), δ .

We thus have d(p(tn , a,t0 ), M) < ε and V (p(t, a,t0 ),t) ≤ f (zn ) ≤ ϕ1 (ε ) for all t ∈ (tn ,tn+1 ), and hence, d(p(t, a,t0 ), M) < ε . This proves that (S, M) is uniformly asymptotically stable in the large. 2

4 Relation Between the Classical Lyapunov Stability Results and the DDS Stability Results The results of the present section were first reported in [MH06]–[HM07a]. The section consists of two parts. 4.1 Continuous Dynamical Systems We first prove that if the hypotheses of the classical Lyapunov stability theorems for continuous dynamical systems (given in Subsection 2.2) are satisfied, then the hypotheses of the corresponding stability theorems for the DDS (given in Section 3) are also satisfied. Due to space limitations, we will consider only uniform stability, uniform asymptotic stability, and uniform asymptotic stability in the large. Theorem 10. If the hypotheses of Theorem 1 for continuous dynamical systems are true, then the hypotheses of Theorem 7 for the DDS are satisfied. Therefore Theorem 1 reduces to Theorem 7. Proof. Choose arbitrarily an unbounded and discrete subset E = {t1 ,t2 , · · · } of Rt+0 , where t1 < t2 < · · · . Let f ∈C[R+ , R+ ] be the identity function, i.e., f (r) = r. By assumption, for any a ∈ U and p(·, a,t0 ) ∈ S, V (p(t, a,t0 ),t) is continuous on R+ and V (p(tn , a,t0 ),tn ) is nonincreasing for n ∈ N. Furthermore, V (p(t, a,t0 ),t) ≤ V (p(tn , a,t0 ),tn ) = f (V (p(tn , a,t0 ),tn )) for t ∈ (tn ,tn+1 ), n ∈ N. Hence, all the hypotheses of Theorem 7 are satisfied and thus, (S, M) is invariant and uniformly stable. 2 In the proof of the next theorem, we require the result given below (called a comparison theorem).

Stability Theory of Discontinuous Dynamical Systems

183

Lemma 1 ( [MM81], Lemma 2.8.2). Consider the initial-value problem x(τ ) = ξ ,

x˙ = g(t, x),

(I)

where D ⊂ R2 is a domain and g ∈ C[D, R], and the associated differential inequality D+ x(t) ≤ g(t, x(t)).

(EI)

If x(t) is a continuous solution of (EI) with x(τ ) ≤ ξ , and if ϕM is the maximal solution of (I), then x(t) ≤ ϕM (t) for as long as both functions exist and t ≥ τ . Theorem 11. If the hypotheses of Theorem 3 for continuous dynamical systems are true, then the hypotheses of Theorem 8 for the DDS are satisfied. Therefore Theorem 3 reduces to Theorem 8. Proof. For any a ∈ U and any p(·, a,t0 ) ∈ S, choose E# = {s1 , s2, . . .} recursively in  the following manner. For n ∈ N let s0 = t0 and sn+1 = sn + min 1, αn , where  1 αn = sup τ : V (p(t, a,t0 ),t) ≥ V (p(sn , a,t0 ), sn ) 2

 t ∈ (sn , sn + τ ) ,

i.e., V (p(t, a,t0 ),t) ≥ 12 V (p(sn , a,t0 ), sn ) for all t ∈ (sn , sn+1 ). If E# is unbounded then simply let tn = sn for all n ∈ N. Then the set E = {t1 ,t2 , · · · : t1 < t2 < · · · } is clearly unbounded and discrete. It follows from the assumptions of the theorem and from the choice of tn that for any t ∈ (tn ,tn+1 ), we have ! d(p(t, a,t0 ), M) ≥ ϕ2−1 ◦ V (p(t, a,t0 ),t) 1 ! ≥ ϕ2−1 ◦ V (p(tn , a,t0 ),tn ) 2 ! −1 1 ≥ ϕ2 ◦ ϕ1 (d(p(tn , a,t0 ), M)). (21) 2 Letting g(t, x) = −ϕ3 (d(p(t, a,t0 ), M)), τ = tn , ξ = V (p(tn , a,t0 ),tn ), the (maximal) solution of (I) (in Lemma 1) is simply given by  tn+1 ϕM (t) = V (p(tn , a,t0 ),tn ) − ϕ3 (d(p(t, a,t0 ), M))dt. tn

It follows from Lemma 1 that V (p(tn+1 , a,t0 ),tn+1 ) ≤ V (p(tn , a,t0 ),tn ) −  tn+1

 tn+1

ϕ3 (d(p(t, a,t0 ), M))dt

tn

1 ! ϕ3 ◦ ϕ2−1 ◦ ϕ1 (d(p(tn , a,t0 ), M))dt 2 tn 1 ! = V (p(tn , a,t0 ),tn ) − (tn+1 − tn) ϕ3 ◦ ϕ2−1 ◦ ϕ1 (d(p(tn , a,t0 ), M)). 2 ≤ V (p(tn , a,t0 ),tn ) −

(22)

184

Anthony N. Michel and Ling Hou

It follows readily from the above inequality that for all n ∈ N 1 ! DV (p(tn , a,t0 ),tn ) ≤ − ϕ3 ◦ ϕ2−1 ◦ ϕ1 (d(p(tn , a,t0 ), M)). 2

(23)

Next, we consider the case when E# is bounded, i.e., sup{sn : n ∈ N} = L < ∞. Since sn is strictly increasing, it must be true that L = limn→∞ sn . Therefore there exists an n0 ∈ N such that sn ∈ (L − 1, L) for all n ≥ n0 . Furthermore, it follows from the continuity of V (p(t, a,t0 ),t) that 1 V (p(sn+1 , a,t0 ), sn+1 ) = V (p(sn , a,t0 ), sn ), 2 which yields V (p(L, a,t0 ), L) = limn→∞ V (p(sn , a,t0 ), sn ) = 0. Let tn = sn , if n ≤ n0 , and tn = sn0 + (n − n0 ) if n > n0 . The set E = {t1 ,t2 , · · · : t1 < t2 < · · · } is clearly unbounded and discrete. Similarly as shown above, (23) holds for any n < n0 . For all n > n0 , we have V (p(tn , a,t0 ),tn ) ≤ V (p(L, a,t0 ), L) = 0. Therefore (23) is also satisfied. When n = n0 , we have tn0 +1 = tn0 + 1 > L, V (p(tn0 +1 , a,t0 ),tn0 +1 ) ≤ V (p(L, a,t0 ), L) = 0, and DV (p(tn0 , a,t0 ),tn0 ) = −V (p(tn0 , a,t0 ),tn0 ) ≤ −ϕ1 (d(p(tn0 , a,t0 ), M)).

(24)

!   If we let ϕ#3 defined on R+ be given by ϕ#3 (r) = min ϕ1 (r), ϕ3 ◦ ϕ2−1 ◦ 12 ϕ1 (r) , then ϕ#3 ∈ K . In view of (23) and (24), we have shown that DV (p(tn , ,t0 ),tn ) ≤ −ϕ#3 (d(p(tn , a,t0 ), M))

(25)

for all n ∈ N. Combining with Theorem 10, we have shown that the hypotheses of Theorem 8 are satisfied. Therefore (S, M) is uniformly asymptotically stable. 2 Theorem 12. If the hypotheses of Theorem 5 for continuous dynamical systems are true, then the hypotheses of Theorem 9 for the DDS are satisfied. Therefore Theorem 5 reduces to Theorem 9. Proof. For any a ∈ A and p(·, a,t0 ) ∈ S, choose E = {t1 ,t2 , . . . : t1 < t2 < · · · } in the same manner as in the proof of Theorem 11. Let f ∈ C[R+ , R+ ] be the identity function, i.e., f (r) = r. It follows from (7) and Lemma 1 that t V (p(t, a,t0 ),t) − V (p(tn , a,t0 ),tn ) ≤ − ϕ3 (d(p(s, a,t0 ), M))ds ≤ 0, tn

and thus, V (p(t, a,t0 ),t) ≤ V (p(tn , a,t0 ),tn ) = f (V (p(tn , a,t0 ),tn )) for all t ∈ (tn ,tn+1 ), n ∈ N.

Stability Theory of Discontinuous Dynamical Systems

185

Similarly as in the proof of Theorem 11, we can show that DV (p(tn , a,t0 ),tn ) ≤ −ϕ˜ 3 (d(p(tn , a,t0 ), M)), ! for all n ∈ N, where ϕ˜ 3 ∈ K is given by ϕ˜ 3 (r) = min{ϕ1 (r), ϕ3 ◦ ϕ2−1 ◦ 12 ϕ1 (r)}. Hence, we have shown that the hypotheses of Theorem 9 are satisfied. Therefore (S, M) is uniformly asymptotically stable in the large. 2 We conclude by noting that results for exponential stability, uniform boundedness and uniform ultimate boundedness of motions, exponential stability in the large, instability, and complete instability, which are in the spirit of the above results have also been established [MH06]– [HM07a]. 4.2 Discrete-Time Dynamical Systems We first note that for every discrete-time dynamical system {N, X , A, S} there is a # where DDS, {R+ , X, A, S}, S#= { p#(t, a,t0 = n0 ) : p#(t, a,t0 ) = p(n, a, n0 ), for t ∈ [n, n + 1), n ∈ N, n0 ∈ N}. The motion p#(·, a,t0 ) ∈ S# is continuous with respect to t at all points in Rt+0 except at # the associated DDS the set of points E = {n0 + 1, n0 + 2, · · · }. We call {R+ , X , A, S} of the discrete-time dynamical system {N, X , A, S}. # and the various definitions of Lyapunov staFrom the definition of {R+ , X , A, S} bility and boundedness, it is clear that for M ⊂ A, (S, M) is invariant if and only if # M) is invariant; (S, M) is (uniformly) stable if and only if (S, # M) is (uniformly) (S, stable; and so forth. We summarize this in the following result. Lemma 2. The discrete-time dynamical system, {N, X , A, S}, and the associated # have identical stability properties. DDS, {R+ , X, A, S}, We next prove that when the hypotheses of the various classical principal Lyapunov stability results for a discrete-time dynamical system, enumerated in Subsection 2.2, are satisfied, then the associated discontinuous dynamical system satisfies the hypotheses of the corresponding results for the DDS given in Section 3. Due to space limitations we will consider only uniform stability, uniform asymptotic stability, and uniform asymptotic stability in the large. Theorem 13. If the hypotheses of Theorem 2 for discrete-time dynamical systems are true, then the associated DDS satisfies the hypotheses of Theorem 7. Therefore Theorem 2 reduces to Theorem 7 # be the associated DDS and let V# : X × R+ → R+ be Proof. First, let {R+ , X , A, S} defined as V# (x,t) = V (x, n) for all x ∈ X and t ∈ [n, n + 1), n ∈ N. It follows directly from (3) that ϕ1 (d(x, M)) ≤ V# (x,t) ≤ ϕ2 (d(x, M)) for all x ∈ X and t ∈ R+ .

186

Anthony N. Michel and Ling Hou

For any a ∈ U and p(·, a, n0 ) ∈ S, the associated motion p#(t, a,t0 = n0 ) is continuous everywhere on Rt+0 except possibly on the set E = {t1 = n0 + 1,t2 = n0 + 2, · · ·}. E is clearly unbounded and discrete. Let f ∈ C[R+ , R+ ] be the identity function. It # ( p#(tn , a, n0 ),tn ) is nonincreasing and follows from the assumptions that V # ( p#(tn , a,t0 ), n) = f (V# ( p#(tn , a,t0 ), n)), V# ( p#(t, a,t0 ),t) = V for t ∈ (tn ,tn+1 ), n ∈ N. # and the set M satisfy the hypotheses of Hence the associated DDS {R+ , X , A, S} Theorem 7 and thus, (S, M) is invariant and uniformly stable. 2 Theorem 14. If the hypotheses of Theorem 4 for discrete-time dynamical systems are true, then the associated DDS satisfies the hypotheses of Theorem 8. Therefore Theorem 4 reduces to Theorem 8. # be the associated DDS and let V # : X × R+ → R+ be defined Proof. Let {R+ , X , A, S} as V# (x,t) = V (x, n) for all x ∈ X and t ∈ [n, n + 1), n ∈ N. For any a ∈ U and p(·, a, n0 ) ∈ S, the associated motion p#(t, a,t0 = n0 ) is continuous everywhere on Rt+0 except possibly on the set E = {t1 = n0 + 1,t2 = n0 + 2, · · ·}. E is clearly unbounded and discrete. Noting that tn = n0 + n and tn+1 − tn = 1, along the motion p#(t, a, n0 ) we have # ( p#(tn , a, n0 ),tn ) DV# ( p#(tn , a, n0 ),tn ) = V# ( p#(tn+1 , a, n0 ),tn+1 ) − V = V (p(n0 + n + 1, a, n0 ), n0 + n + 1) − V (p(n0 + n, a, n0), n0 + n) ≤ −ϕ3 (d( p#(tn , a, n0 ), M))

(26)

for all n ∈ N. Combining with Theorem 13, we have shown that the hypotheses of Theorem 8 are satisfied. Therefore (S, M) is uniformly asymptotically stable. 2 Theorem 15. If the hypotheses of Theorem 6 for discrete-time dynamical systems are true, then the associated DDS satisfies the hypotheses of Theorem 9. Therefore Theorem 6 reduces to Theorem 9. ˜ be the associated DDS and let V˜ : X × R+ → R+ be defined Proof. Let {R+ , X , A, S} as V˜ (x,t) = V (x, n) for all x ∈ X and t ∈ [n, n + 1), n ∈ N. For any a ∈ A and p(·, a, n0 ) ∈ S, the associated motion p(t, ˜ a,t0 = n0 ) is continuous everywhere on Rt+0 except possibly on E = {t1 = n0 + 1,t2 = n0 + 2, . . . }. E is clearly unbounded and discrete. Let f ∈ C[R+ , R+ ] be the identity function, i.e., f (r) = r. Similarly as in the proof of Theorem 13, we can show that the associated motions and the function V˜ satisfy (18)–(20). ˜ and the set M satisfy Thus, we have shown that the associated DDS {R+ , X , A, S} the hypotheses of Theorem 9. Therefore (S, M) is uniformly asymptotically stable in the large. 2

Stability Theory of Discontinuous Dynamical Systems

187

We conclude by noting that results for exponential stability, uniform boundedness and uniform ultimate boundedness of motions, exponential stability in the large, instability, and complete instability, which are in the spirit of the above results have also been established [MH06]– [HM07a].

5 An Example In Section 4 it was shown that the classical Lyapunov stability and boundedness results for continuous dynamical systems and discrete-time dynamical systems reduce to corresponding stability and boundedness results for the DDS. By means of a specific example, we now show that for continuous dynamical systems, converse statements to the above assertions are in general not true. The present example was originally reported in [HM07a]. By using a specific example, we can also show that for discrete-time dynamical systems, converse statements to the above assertions are in general also not true [HM07a]. Accordingly, the results of Section 4 and this section show that the DDS results of Section 3 are in general less conservative than the classical Lyapunov stability and boundedness results (for both continuous dynamical systems and discrete-time dynamical systems). The scalar differential equation

(ln2)x, if t ∈ [t0 + 2k,t0 + 2k + 1), x˙ = (27) −(ln4)x, if t ∈ [t0 + 2k + 1,t0 + 2(k + 1)), where k ∈ N, x ∈ R, and t0 ∈ R+ , determines a dynamical system {R+ , X , A, S} with X = A = R and with p(·, a,t0 ) ∈ S determined by the solutions of (27) (obtained by integrating (27)), ⎧ a ⎨ k e(ln2)(t−t0 −2k) , if t ∈ [t0 + 2k,t0 + 2k + 1], p(t, a,t0 ) = 2 a (28) ⎩ e−(ln4)(t−t0 −2k−1) , if t ∈ [t0 + 2k + 1,t0 + 2(k + 1)], k−1 2 for each pair (a,t0 ) ∈ R× R+ and for all k ∈ N and t ≥ t0 . The plot of a typical motion for this system is given in Figure 1. Note that for every (a,t0 ) ∈ R × R+ , there exists a unique p(·, a,t0 ) ∈ S which is defined and continuous for t ≥ t0 and that M = {0} is invariant with respect to S. The block diagram of system (27) is depicted in Figure 2. This system can be viewed as a switched system with switching occurring every unit of time since initial time t0 . In the following we show that (a) if we apply the DDS concepts, then there exists a function V : R × R+ → R+ which satisfies Theorem 9 and, therefore, (S, {0}) is uniformly asymptotically stable; (b) if we use the classical Lyapunov concepts, then there does not exist a Lyapunov function V : R × R+ → R+ which satisfies the hypotheses of Theorem 5 and,

188

Anthony N. Michel and Ling Hou

2.5 2 1.5 1 0.5 0

0

2

4

6

8

10

Fig. 1. Plot of the motion, p(t, 1, 0) ∈ S.

x(t 0 ) ln2 −x(t) Inverting Amplifier

.

x(t)

x(t)

ln4

Integrator Switch

Fig. 2. Block diagram for system (27).

therefore, Theorem 5 cannot be used to prove that (S, {0}) is uniformly asymptotically stable. Proof. Proof of Example. (a) Let V : R → R+ be chosen as V (x) = |x| for all x ∈ R. For any p(·, a,t0 ), choose the set E = {t1 ,t2 , . . . : tk = t0 + 2k, k = 1, 2, . . . }. By (28), V (p(tk , a,t0 )) = |a/2k |, and V (p(t, a,t0 )) ≤ 2V (p(tk , a,t0 )) for all t ∈ [tk ,tk+1 ], k ∈ N. Therefore all hypotheses of Theorem 9 are satisfied and hence, (S, {0}) is uniformly asymptotically stable. (b) For purposes of contradiction, assume that there exist a function V : R × R+ → R+ and two functions ϕ1 , ϕ2 ∈ K defined on R+ such that

ϕ1 (|x|) ≤ V (x,t) ≤ ϕ2 (|x|)

(29)

for all (x,t) ∈ R × R+ , and there exists a neighborhood U of 0 such that for all a ∈ U and all p(·, a,t0 ) ∈ S, V (p(t, a,t0 ),t) is nonincreasing for all t ≥ t0 , t ∈ R+ . Without loss of generality, we assume that 1 ∈ U. 1 We now examine V along a sequence of motions, p(·, 21n , 1), p(·, 2n−1 , 2), · · · , 1 p(·, 2 , n − 1), for any n ∈ N. By (28), p(t0 + 1, a,t0) = 2a for any (a,t0 ) ∈ R × R+ . In particular, since  1      1 1 1 1 p 2, n , 1 = n−1 , p 3, n−1 , 2 = n−2 , · · · , p n, , n − 1 = 1, 2 2 2 2 2

Stability Theory of Discontinuous Dynamical Systems

189

and since V (p(t, a,t0 ),t) is nonincreasing for all p(·, a,t0 ) ∈ S, we have 1    1    1  , 1 ≥ V p 2, , 1 , 2 = V , 2 2n 2n 2n−1  1  (along the motion p ·, n , 1 ); 2  1       1  1 V n−1 , 2 ≥ V p 3, n−1 , 2 , 3 = V n−2 , 3 2 2 2  1  (along the motion p ·, n−1 , 2 ); 2 .. .    1   1 V , n − 1 ≥ V p n, , n − 1 , n = V (1, n) 2 2  1  (along the motion p ·, , n − 1 ). 2   Therefore, V 21n , 1 ≥ V (1, n). On the other hand, V

1 1  1 ≤ V n , 1 ≤ ϕ2 n and ϕ1 (1) ≤ V (1, n) ≤ ϕ2 (1). n 2 2 2   Thus, ϕ2 21n ≥ ϕ1 (1) is true for all n ∈ N, which implies that

ϕ1

ϕ2 (0) = lim ϕ2 n→∞

1 ≥ ϕ1 (1) > 0. 2n

However, since ϕ2 ∈ K , we know that ϕ2 (0) = 0. We have arrived at a contradiction. Therefore, there does not exist a Lyapunov function that satisfies the hypotheses of the classical Lyapunov theorem for uniform asymptotic stability for continuous dynamical systems, Theorem 5. 2 Remark 1. (a) In the case of the classical Lyapunov theorem for asymptotic stability (see Theorems 3 and 5), the Lyapunov functions must decrease monotonically at all time with increasing time, while in the case of the Lyapunov stability theorem for asymptotic stability for DDSs (see Theorems 8 and 9), the Lyapunov functions must decrease monotonically only at an unbounded discrete set of time instants, and between time instants, the Lyapunov functions must be bounded in a certain reasonable way. Among other things, in the case of Theorems 8 and 9, this allows that between time instants, the Lyapunov functions may increase. Accordingly, if as was done in the above example, one uses vector norms as a Lyapunov function, then the norms of the motions may grow between time instants, as long as they decrease monotonically at the indicated unbounded discrete set of time instants. These observations explain why in the above example the DDS results (Theorems 8 and 9) were successful while the corresponding classical Lyapunov stability results (Theorems 3 and 5) failed in the stability analysis. (b) More gen-

190

Anthony N. Michel and Ling Hou

erally, the above discussion explains why in the literature on the stability analysis of switched systems, many investigators resort to a “multi-Lyapunov function approach” in order to achieve reasonable results using the classical Lyapunov stability results (e.g., [DBPL00]– [LSW05]). In the stability analysis of switched systems, using the results in [YMH98], we do not encounter such difficulties (see [HMY97]– [HM00]). Similarly as was done in the preceding example for continuous systems, we can also show that for discrete-time dynamical systems, converse statements to the results of Section 4 are in general not true [HM06a], [HM07a]. We will not pursue this due to space limitations.

6 Converse Theorems The DDS results which we have addressed thus far constitute sufficient conditions. It turns out that under some additional mild assumptions, these results constitute necessary conditions as well, called Converse Theorems. Such results were first established in [YMH98], involving Lyapunov functions which are not necessarily continuous. Subsequently, the results of [YMH98] were refined, requiring continuous Lyapunov functions [HM06b, HM07b]. In the present section we will confine our discussion to converse theorems for DDSs for the case of uniform asymptotic stability, with the understanding that similar results have also been established for other types of stability and boundedness of motions [HM06b, HM07b]. This section consists of three parts. We first present some necessary background material. Next, we establish a Converse Theorem for uniform asymptotic stability involving continuous Lyapunov functions. Finally, we present a specific example in order to clarify some of the assumptions made. 6.1 Background Material In establishing the Converse Theorem for uniform asymptotic stability for the DDS (Theorem 8), we required, in [YMH98], the following assumptions. Assumption 1 Let {R+ , X , A, S} be a DDS and assume that ˜ a1 ,t1 ) ∈ S with a1 = p(t1 , a,t0 ) and t1 > t0 (i) for any p(·, a,t0 ) ∈ S, there exists a p(·, such that p(·, ˜ a1 ,t1 ) = p(·, a,t0 ) for all t ≥ t1 ; and (ii) for any two motions pi (·, ai ,ti ) ∈ S, i = 1, 2, t2 > t1 , if a2 = p1 (t2 , a1 ,t1 ), then there exists a motion p(·, ˆ a1 ,t1 ) ∈ S such that p(t, ˆ a1 ,t1 ) = p1 (t, a1 ,t0 ) for t ∈ [t1 ,t2 ) and p(t, ˆ a1 ,t1 ) = p2 (t, a2 ,t2 ) for t ≥ t2 . In part (i) of Assumption 1, p(·, ˜ a1 ,t1 ) may be viewed as a partial motion of the motion p(·, a,t0 ), and in part (ii), p(·, ˆ a,t1 ) may be viewed as a composition of p1 (·, a1 ,t1 ) and p2 (·, a2 ,t2 ). With this convention, Assumption 1 states that

Stability Theory of Discontinuous Dynamical Systems

191

(a) any partial motion is a motion in S, and (b) any composition of two motions is a motion in S. Assumption 2 Let {R+ , Rn , A, S} be a DDS and assume that every motion p(·, a,t0 ) ∈ S is continuous everywhere on Rt+0 , except possibly on an unbounded, closed, and discrete set E = {t1 ,t2 , · · · : t0 < t1 < t2 < · · · } (recall that in general Δ

E depends on p(·, a,t0 ) ∈ S), lE = inf p∈S {tk+1 − tk : k = 1, 2, · · · } > 0, and that Δ

LE = sup p∈S {tk+1 − tk : k = 0, 1, 2, · · · } < ∞. Assumption 3 For every (x0 ,t0 ) ∈ A × R+ there exists a unique motion p(·, x0 ,t0 ) ∈ S which exists for all t ∈ [t0 , ∞). The next result, which constitutes a Converse Theorem for Theorem 8, involves Lyapunov functions which are not necessarily continuous. For a proof of this result, refer to [YMH98]. Theorem 16 (Converse Theorem). [YMH98] Let {R+ , X , A, S} be a DDS and let M ⊂ A be a closed invariant set, where A is assumed to be a neighborhood of M. Assume that S satisfies Assumptions 1–3. Let (S, M) be uniformly asymptotically stable. Then there exist neighborhoods A1 and X1 of M such that A1 ⊂ X1 ⊂ A, and a mapping V : X1 × R+ → R+ which satisfies the following conditions: (i) there exist functions ψ1 , ψ2 ∈ K (defined on R+ ) such that

ψ1 (d(x, M)) ≤ V (x,t) ≤ ψ2 (d(x, M))

(30)

for all (x,t) ∈ X1 × R+ ; (ii) there exists a function ψ3 ∈ K , defined on R+ , such that for all p(·, a,t0 ) ∈ S, we have DV (p(tk , a,t0 ),tk ) ≤ −ψ3 (d(p(tk , a,t0 ), M)) (31) where a ∈ A1 , k ∈ N, and where DV (p(tk , a,t0 ),tk ) is given in (13); and (iii) there exists a function f ∈ C[R+ , R+ ] such that f (0) = 0 and such that V (p(t, a,t0 ),t) ≤ f (V (p(tk , a,t0 ),tk ))

(32)

for every p(·, a,t0 ) ∈ S and all t ∈ (tk ,tk+1 ), a ∈ A1 and t0 ∈ R+ . 6.2 A Converse Theorem with Continuous Lyapunov Functions In the proof of Theorem 16 [YMH98], the Lyapunov function V is constructed using the unique motion that starts at (x,t0 ) ∈ A × R+ . In the present section, we first show that under some additional very mild assumptions (Assumption 4) the function V given in the converse Theorem 16 is continuous, i.e., V (x0m ,t0m ) approaches V (x0 ,t0 ) as m → ∞ if x0m → x0 and t0m → t0 as m → ∞. Next, we define continuous dependence on the initial conditions for motions of the DDS and we show that Assumption 4 is satisfied when the motions are continuous with respect to initial conditions.

192

Anthony N. Michel and Ling Hou

Assumption 4 Let {R+ , X , A, S} be a DDS. Suppose x0m → x0 and t0m → t0 as m → ∞. The motion starting at (x0m ,t0m ) is denoted by pm (t, x0m ,t0m ) with the discontinuity set E(x0m ,t0m ) = {t1m ,t2m , · · · : t0m < t1m < t2m < · · · }. Denote xkm = pm (tkm , x0m ,t0m ) and xk = p(tk , x0 ,t0 ). Assume that (1) tkm → tk as m → ∞, for all k ∈ N. (2) xkm → xk as m → ∞, for all k ∈ N. In the proof of the converse theorem (Theorem 17), we will require the following preliminary result. Lemma 3. Let β ∈ L be defined on R+ . Then there exists a function α ∈ K defined on R+ such that for any discrete subset {t0 ,t1 , · · · } ⊂ R+ satisfying lE = inf{tn+1 − tn : n = 1, 2, · · · } > 0, it is true that

∞

∑ α (β (t j − t0)) < +∞,

j=0

and

∞

∑ α (β (t j − t0)) < j=k

exp (−(k − 1)lE ) 1 − exp(−lE )

for all k ≥ 1. Proof. We define η ∈ C[(0, ∞), (0, ∞)] as

β (t)/t, t ∈ (0, 1) η (t) = . β (t), t ∈ [1, ∞) Clearly, η (t) is strictly decreasing for all t > 0, limt→0+ η (t) = +∞ and η (t) ≥ β (t) for all t > 0. Furthermore, η is invertible, and η −1 ∈ C[(0, ∞), (0, ∞)] is strictly decreasing, and η −1 (β (τ )) ≥ η −1 (η (τ )) = τ for all τ > 0. We now define α (0) = 0 and

α (t) = exp(−η −1 (t)), t > 0. Then α ∈ K , and

α (β (τ )) = exp(−η −1 (β (τ ))) ≤ exp(−τ ). Since lE ≤ t j+1 − t j , j = 1, 2, · · · , we know that t j − t0 ≥ ( j − 1)lE . Hence it is true that ∞

∞

∞

j=0

j=0

j=1

∑ α (β (t j − t0)) ≤ ∑ exp(−(t j − t0)) ≤ 1 + ∑ exp(−( j − 1)lE ) 1 , = 1+ 1 − exp(−lE )

Stability Theory of Discontinuous Dynamical Systems

193

and ∞

∞

∞

j=k

j=k

j=k

∑ α (β (t j − t0)) ≤ ∑ exp(−(t j − t0)) ≤ ∑ exp(−( j − 1)lE ) exp (−(k − 1)lE ) . = 1 − exp(−lE ) This completes the proof. We are now in a position to prove the following result.

2

Theorem 17. If in addition to the assumptions given in Theorem 16, the motions in S also satisfy Assumption 4, then the Lyapunov function in the Converse Theorem 16 is continuous with respect to initial conditions. Proof. The proof consists of two parts. (a) We first show how the Lyapunov function V is constructed. Since (S, M) is asymptotically stable, by the results established in Problem 3.8.9 in [MWH01], there exists a function ϕ ∈ K , defined on [0, h0 ] for some h0 > 0, and a function σ ∈ L , defined on R+ , such that d(p(t, a,t0 ), M) < ϕ (d(a, M))σ (t − t0 ) (33) for all p(·, a,t0 ) ∈ S and all t ≥ t0 , whenever d(a, M) < h0 . Define X1 = {x ∈ A : d(x, M) < h0 }, and A1 = {a ∈ X1 : d(a, M) < ϕ −1 (h0 )} if ϕ −1 (h0 ) ≤ h0 and A1 = X1 otherwise. Since for any (x,t0 ) ∈ X1 × R+ , there exists a unique motion p(·, x,t0 ) which is continuous everywhere on Rt+0 except on E = {t1 ,t2 , · · · : t1 < t2 < · · · }, we define the Lyapunov function V (x,t0 ) as ∞

V (x,t0 ) =

∑u

! d(p(t j , x,t0 ), M) ,

(34)

j=0

where u ∈ K , defined on R+ , is chosen in such a manner that the above summation will converge. It follows from (33) that for any (x,t0 ) ∈ X1 × R+ , we have ! ! u d(p(t, x,t0 ), M) < u ϕ (d(x, M) σ (t − t0 )) ! 1  ! 1  (35) ≤ u ϕ (d(x, M))σ (0) 2 u ϕ (h0 )σ (t − t0) 2 . Let β (τ ) = ϕ (h0 )σ (τ ). Then β ∈ L . Hence, by!Lemma 3, there exists a function 2 α ∈ K defined on R+ such that ∑∞ i=0 α β (ti − t0 ) < ∞. If we define u(r) = [α (r)] , then it follows that 

! 1 ! ! u ϕ (h0 )σ (t − t0 ) 2 = α ϕ (h0 )σ (t − t0 ) = α β (t − t0 ) .

Hence, we conclude from (34)–(36) that

(36)

194

Anthony N. Michel and Ling Hou ∞

V (x,t0 ) = <

∑u

j=0 ∞ 

∑

! d(p(t j , x,t0 ), M)

! 1  ! 1 u ϕ (d(x, M))σ (0) 2 u ϕ (h0 )σ (t j − t0 ) 2

j=0

 ! 1 = u ϕ (d(x, M))σ (0) 2

∞

∑α

! β (t j − t0 )

j=0

 ! 1   < u ϕ (d(x, M))σ (0) 2 1 + 1/(1 − exp(−lE )) , where lE is the lower bound given in Assumption 2. (b) We now show that V is continuous with respect to the initial conditions. Suppose x0m → x0 and t0m → t0 as m → ∞. We denote pm (tkm , x0m ,t0m ) by xkm . Then ∞

V (x0m ,t0m ) = ∑ u(d(pm (tim , x0m ,t0m ), M)) i=0 ∞

= ∑ u(d(xim , M)).

(37)

i=0

We will show that V (x0m ,t0m ) approaches V (x0 ,t0 ) = ∑∞ i=0 u(d(xi , M)) as m → ∞. It follows from (35), (36), and Lemma 3 that ∞

∞

i=k

i=k

∑ u(d(p(ti , x0 ,t0 ), M)) < ∑ [u(ϕ (d(x0 , M))σ (0))] 2 [u(ϕ (h0)σ (ti − t0))] 2 1

1

≤ [u(ϕ (h0 )σ (0))] 2

1

∞

∑ α (β (ti − t0))

i=k

< [u(ϕ (h0 )σ (0))]

1 2

exp(−(k − 1)lE ) . 1 − exp(−lE )

For every ε > 0, in view of the above inequality, there exists an n0 > 0 such that ∞

∑ u(d(xi , M)) < ε /4

(38)

i=n0

for all x0 ∈ A1 . Similarly, ∞

∑ u(d(xim , M)) < ε /4

(39)

i=n0

for all x0m ∈ A1 . On the other hand, for every k ≤ n0 , there exists a δk > 0 such that |u(r) − u(d(xk , M))| < ε /(2n0 ) whenever |r − d(xk , M)| < δk (since u(·) is continuous everywhere on R+ ). Since xkm → xk as m → ∞, there exists for each k ≤ n0 an mk > 0 such that d(xkm , xk ) < δk is true for all m ≥ mk . Now let mε = maxk≤n0 {mk }. For every

Stability Theory of Discontinuous Dynamical Systems

195

m > mε we have |d(xk , M) − d(xkm , M)| ≤ d(xk , xkm ) < δk and thus   n0 −1  n0 −1    ∑ u(d(xk , M)) − ∑ u(d(xkm , M))  k=0  k=0 ≤

n0 −1

∑ |u(d(xk , M)) − u(d(xkm , M))| < ε /2.

(40)

k=0

Therefore we have shown that |V (x0 ,t0 ) − V (x0m ,t0m )|   ∞  ∞   =  ∑ u(d(xk , M)) − ∑ u(d(xkm , M)) k=0  k=0   n0 −1  n0 −1   ≤  ∑ u(d(xk , M)) − ∑ u(d(xkm , M))  k=0  k=0 +

∞

∞

k=n0

k=n0

∑ u(d(xk , M)) + ∑ u(d(xkm , M))

< ε.

(41)

Therefore, we conclude that V is continuous with respect to initial conditions (x0 ,t0 ). 2 The following concept of continuous dependence on initial conditions for DDSs is motivated by a corresponding concept for ordinary differential equations (see, e.g., [MM81]), and will be used as a sufficient condition for Assumption 4. Definition 6. Suppose {x0m } ⊂ A ⊂ X , {τ0m } ⊂ R+ , x0m → x0 ∈ A and τ0m → τ0 as m → ∞. Assume that the motions are given by p(t, x0 , τ0 ) = pk (t, xk , τk ), t ∈ [τk , τk+1 ), and

pm (t, x0m , τ0m ) = pkm (t, xkm , τkm ), t ∈ [τkm , τ(k+1)m ),

k ∈ N, where pk (t, xk , τk ) and pkm (t, xkm , τkm ) are continuous for all t ∈ R+ with pk (τk , xk , τk ) = p(τk , x0 , τ0 ) = xk and

pkm (τkm , xkm , τkm ) = pm (τkm , x0m , τ0m ) = xkm .

The motions in S are said to be continuous with respect to the initial conditions (x0 , τ0 ) if for any x0m → x0 and any τ0m → τ0 , (1) τkm → τk as m → ∞, for all k ∈ N.

196

Anthony N. Michel and Ling Hou

(2) for every compact set K ⊂ R+ and every ε > 0 there exists L = L(K, ε ) > 0 such that for all t ∈ K and k ∈ N such that K ∩ [τk , τk+1 ) = 0, / d(pkm (t, xkm , τkm ), pk (t, xk , τk )) < ε whenever m > L. An example of the set of continuous functions pk (t, xk ,tk ) is given by ⎧ t < tk , ⎨ xk , k p(t, x ,t ), t ∈ [tk ,tk+1 ), p (t, xk ,tk ) = 0 0 ⎩ − p(tk+1 , x0 ,t0 ), t ≥ tk+1 . Another example of pk (t, xk ,tk ) is given in the next subsection. Theorem 18. If in addition to the assumptions given in Theorem 16, the motions in S are continuous with respect to initial conditions (in the sense of Definition 6), then the Lyapunov function given in (34) is continuous with respect to initial conditions (x0 ,t0 ). Proof. We will show that under the present hypotheses, Assumption 4 is satisfied and hence V is continuous with respect to initial conditions by Theorem 17. Suppose x0m → x0 and τ0m → τ0 as m → ∞. Assumption 4 (1) is the same as in Definition 6. We only need to show that Assumption 4 (2) is satisfied, i.e., xkm → xk as m → ∞ for all k ∈ N. For a fixed k > 0, k ∈ N, let K = [τk − lE /2, τk + lE /2]. For every ε > 0 there exists an L = L(K, ε /2) > 0 such that for all t ∈ K d(pkm (t, xkm , τkm ), pk (t, xk , τk )) < ε /2

(42)

whenever m > L. Since pk (t, xk , τk ) is continuous on R+ , there exists a δ > 0 such that d(pk (t  , xk , τk ), pk (τk , xk , τk )) < ε /2 whenever |t  − τk | < δ . Since τkm → τk as m → ∞, there exists an L1 > 0 such that τkm ∈ K and |τkm − τk | < δ for all m > L1 . Therefore, when m > max{L, L1 }, we have by (42) d(pkm (τkm , xkm , τkm ), pk (τkm , xk , τk )) < ε /2, and by the continuity of pk (t, xk , τk ) d(pk (τkm , xk , τk ), pk (τk , xk , τk )) < ε /2. By the triangle inequality we have d(pkm (τkm , xkm , τkm ), pk (τk , xk , τk )) ≤ d(pkm (τkm , xkm , τkm ), pk (τkm , xk , τk )) + d(pk (τkm , xk , τk ), pk (τk , xk , τk )) < ε. This shows that {xkm = pkm (τkm , xkm , τkm )} → xk as m → ∞. This completes the proof. 2

Stability Theory of Discontinuous Dynamical Systems

197

6.3 An Example We conclude the section by considering a specific example to demonstrate that the assumptions concerning the continuous dependence of the solutions (motions) on initial data are realistic. Consider systems with impulse effects, which are described by equations of the form  dx = f (x,t), t = tk , (43) dt − x(tk ) = g(x(tk )), where x ∈ Rn denotes the state, g ∈ C[Rn , Rn ], and f ∈ C[Rn × R+ , Rn ] satisfies a Lipschitz condition with respect to x which guarantees the existence and uniqueness of solutions of system (43) for given initial conditions. The set E = {t1 ,t2 , · · · : t1 < t2 < · · · } ⊂ R+ denotes the set of times when jumps occur. Assume that E is fixed in the interest of simplicity. A function ϕ : [t0 , ∞) → Rn is said to be a solution of the system with impulse effects (43) if (i) ϕ (t) is left continuous on [t0 , ∞) for some t0 ≥ 0; (ii) ϕ (t) is differentiable and ddtϕ (t) = f (ϕ (t),t) everywhere on (t0 , ∞) except on an unbounded discrete subset E ∩ {t : t > t0 }; and (iii) for any t = tk ∈ E ∩ {t : t > t0 },

ϕ (t + ) =  lim ϕ (t  ) = g(ϕ (t − )). t →t,t >t

Suppose t0 ∈ [tk0 ,tk0 +1 ) for some k0 ∈ N. The motion p(t, x0 ,t0 ) is given by

t ∈ [tk ,tk+1 ), k > k0 p(d) (t, xk ,tk ), p(t, x0 ,t0 ) = − g(p(d) (tk+1 , xk ,tk )), t = tk+1 , and p(t, x0 ,t0 ) = p(d) (t, x0 ,t0 ), t ∈ [t0 ,tk0 +1 ), where xk = p(tk , x0 ,t0 ), and where p(d) (t, xk ,tk ), t ∈ R+ is the solution of the following ordinary differential equation: dx = f (x,t), dt

with p(d) (tk , xk ,tk ) = xk .

Suppose x0m → x0 and t0m → t0 as m → ∞. Without loss of generality, we may assume that t0 < t1 ∈ E. By the assumption that E is fixed it follows that for sufficiently large m, the discontinuity set is {tkm = tk }, for all k > 0. From the continuous dependence on initial conditions of ordinary differential equations, we know that p(d) (t, x0m ,t0m ) → p(d) (t, x0 ,t0 ) for t in any compact set of R+ as m → ∞. Since g(·) is continuous, we have x1m = g(p(d) (t1− , x0m ,t0m )) → x1 = g(p(d) (t1− , x0 ,t0 )) as m → ∞. In turn, we have p(d) (t, x1m ,t1 ) → p(d) (t, x1 ,t1 ) for t in any compact set of R+ as m → ∞ and thus, x2m = g(p(d) (t2− , x1m ,t1 )) → x2 = g(p(d) (t2− , x1 ,t1 )) as m → ∞. By induction, we can show that xkm → xk as m → ∞ for all k > 0. Therefore we have shown that the motions of (43) are continuous with respect to initial conditions.

198

Anthony N. Michel and Ling Hou

7 Concluding Remarks In providing an overview of some of our results concerning the stability analysis of discontinuous dynamical systems (DDSs), we first presented sample results from [YMH98] where we developed a general stability theory for DDS described on metric spaces. Next, we proved that no matter what kind of dynamical system is being considered (continuous time or discrete time and finite dimensional or infinite dimensional) and no matter what kind of Lyapunov function is being used, whenever the hypotheses of a classical Lyapunov stability and boundedness result are satisfied for a given Lyapunov function, then the hypotheses of the corresponding stability and boundedness result for DDSs (given in [YMH98]) are also satisfied for the same Lyapunov function [MH06]– [HM07a]. Moreover, by using a specific example, we showed that converse statements to these assertions are in general not true [MH06]– [HM07a]. Next, we presented a sample converse stability theorem (for the uniform asymptotic stability) for DDSs, involving continuous Lyapunov functions [HM06b, HM07b]. Finally, we note that stability results for DDSs of the type reported herein have been applied extensively (e.g., [HMY97]– [HM01]).

References [Zub64]

V. I. Zubov, Methods of A.M. Lyapunov and their Applications, Groningen, The Netherlands: P. Noordhoff, Ltd., 1964. [Hah67] W. Hahn, Stability of Motion, Berlin, Germany: Springer-Verlag, 1967. [MWH01] A. N. Michel, K. Wang and B. Hu, Qualitative Analysis of Dynamical Systems, 2nd Ed., New York: Marcel Dekker, 2001. [YMH98] H. Ye, A. N. Michel and L. Hou, Stability theory for hybrid dynamical systems, IEEE Transactions on Automatic Control, vol. 43, no. 4, pp. 461–474, April 1998. [MH99] A. N. Michel and B. Hu, Towards a stability theory of general hybrid dynamical systems, Automatica, Vol. 35, pp. 371–384, April 1999. [Mic99] A. N. Michel, Recent trends in the stability analysis of hybrid dynamical systems, IEEE Transactions on Circuits and Systems-I: Fundamental Theory and Applications, Vol. 46, No. 1, pp. 120–134, January 1999. [DBPL00] R. DeCarlo, M. Branicky, S. Pettersson and B. Lannertson, Perspectives and results on the stability and stabilizability of hybrid systems, Proceedings of the IEEE, Vol. 88, No. 7, pp. 1069–1082, 2000. [LM99] D. Liberzon and A. S. Morse, Basic problems in stability and design of switched systems, IEEE Control Systems Magazine, Vol. 19, No. 5, pp. 59–70, 1999. [BS89] D. D. Bainov and P. S. Simeonov, Systems with Impulse Effects: Stability Theory and Applications, New York: Halsted Press, 1989. [LSW05] Z. Li, Y. Soh, and C. Wen, Switched Impulsive Systems: Analysis, Design, and Applications, Berlin, Germany: Springer-Verlag, 2005. [SMZ05] Y. Sun, A. N. Michel and G. Zhai, Stability of discontinuous retarded functional differential equations with applications, IEEE Transactions on Automatic Control, Vol. 50, No. 8, pp. 1090–1105, August 2005. [MSM05] A. N. Michel, Y. Sun, and A. P. Molchanov Stability analysis of discontinuous dynamical systems determined by semigroups, IEEE Transactions on Automatic Control, Vol. 50, No. 9, pp. 1277–1290, September 2005.

Stability Theory of Discontinuous Dynamical Systems [MS06] [HMY97] [HM99] [HM00] [HM01] [MH06]

[HM06a]

[HM07a]

[MM81] [HM06b]

[HM07b]

199

A. N. Michel and Y. Sun, Stability analysis of discontinuous dynamical systems in Banach space, Nonlinear Analysis, Vol. 65, pp. 1805–1832, 2006. L. Hou, A. N. Michel and H. Ye, Some qualitative properties of sampled-data control systems, IEEE Trans. on Automatic Control, vol. 42, pp. 1721–1725, 1997. B. Hu and A. N. Michel, Some qualitative properties of multi-rate digital control systems, IEEE Trans. on Automatic Control, vol. 44, pp. 765–770, 1999. B. Hu and A. N. Michel, Stability analysis of digital feedback control systems with time-varying sampling periods, Automatica, vol. 36, pp. 897–905, 2000. L. Hou and A. N. Michel, Stability analysis of pulse-width-modulated feedback systems, Automatica, vol. 37, pp. 1335–1349, 2001. A. N. Michel and L. Hou, Stability of continuous, discontinuous and discretetime dynamical systems: unifying local results, Proceedings of the 2006 American Control Conference, Minneapolis, MN, June 2006, pp. 2418–2423. L. Hou and A. N. Michel, Stability of continuous, discontinuous and discretetime dynamical systems: unifying global results, Proceedings of the 17th International Symposium on Mathematical Theory of Networks and Systems, Kyoto, Japan, August 2006, pp. 2685–2693. L. Hou and A. N. Michel, Unifying theory for stability of continuous, discontinuous, and discrete-time dynamical system, Nonlinear Analysis: Hybrid Systems, vol. 1, Issue 2, June 2007, pp. 154–172. R. K. Miller and A. N. Michel, Ordinary Differential Equations, New York: Academic Press, 1981. L. Hou and A. N. Michel, On the continuity of the Lyapunov functions in the converse stability theorems for discontinuous dynamical systems, Proceedings of the 45th IEEE Conference on Decision and Control, San Diego, CA, December 2006, pp. 5097–5101. L. Hou and A. N. Michel, Converse stability theorems for discontinuous dynamical systems: improved results, Proceedings of the 2007 European Control Conference, Kos, Greece, July 2007.

Direct Adaptive Optimal Control: Biologically Inspired Feedback Control Draguna Vrabie and Frank Lewis Automation and Robotics Research Institute, The University of Texas at Arlington, 7300 Jack Newell Blvd. S, Ft. Worth, Texas 76118 USA Tel/Fax: 817-272-5938; E-mail: (dvrabie,lewis)@uta.edu Summary. Control system theory has been based on certain well understood and accepted techniques such as transfer function-based methods, adaptive control, robust control, nonlinear systems theory and state-space methods. Besides these classical techniques, in recent decades, many successful results have been obtained by incorporating artificial neural networks in classical control structures. Due to their universal approximation property, neural network structures are the perfect candidates for designing controllers for complex nonlinear systems. These successful results have caused a number of control engineers to focus their interest on the results and algorithms of the machine learning and computational intelligence community and, at the same time, to find new inspiration in the biological neural structures of living organisms in their most evolved and complex form: the human brain. In this chapter we discuss two algorithms that were developed, based on a biologically inspired structure, with the purpose of learning the optimal state feedback controller for a linear system, while at the same time performing continuous-time online control for the system at hand. Moreover, since the algorithms are related to the reinforcement learning techniques in which an agent tries to maximize the total amount of reward received while interacting with an unknown environment, the optimal controller will be obtained while only making use of the input-to-state system dynamics. Mathematically speaking, the solution of the algebraic Riccati equation underlying the optimal control problem will be obtained without making use of any knowledge of the system internal dynamics. The two algorithms are built on iteration between the policy evaluation and policy update steps until updating the control policy no longer improves the system performance. Both algorithms can be characterized as direct adaptive optimal control types since the optimal control solution is determined without using an explicit, a priori obtained, model of the system internal dynamics. The effectiveness of the algorithms is shown and their performances compared while finding the optimal state feedback dynamics of an F-16 autopilot.

1 Introduction In classical control theory the first question that the control engineer had to answer was: “How does a system respond to certain external stimuli (i.e. control signals)?”. Thus, the first step in designing a control system consists in determining a model that will describe the behavior of the system at hand, a mapping between the input and C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 10, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

202

Draguna Vrabie and Frank Lewis

the output of the system. Once a model is available and the system behavior can be classified, a number of techniques can be employed to determine the most suitable control strategy for obtaining the specified system performances. Another question that was asked, and was then followed by a new approach to controller design, was: “What would be the control signal that will make the system respond in the desired specified manner?”. The answer to this question is equivalent to determining the inverse model behavior of the system. The result is what is now addressed as model reference control or adaptive inverse control [WW95], a mature research field whose methodologies can be applied to a variety of plant types (e.g. linear or nonlinear, minimum or nonminimum phase, stable or unstable). For this approach the desired performances are specified by a reference signal to be followed by the system output. Starting with a different formulation for the desired performances of the control system, another class of methods was developed to solve the control problem such that a certain optimality criterion is achieved. The optimal control problem includes a cost (value) functional which is a function of the system state and control input. The optimal control input needs to determine the minimum (maximum) value for the specified cost (value) functional. The solution to this problem is generally calculated using the dynamic programming method, a backward-in-time approach based on Bellman’s optimality principle [Be03]. Having noted that solving for the optimal controller using the dynamic programming approach is most of the times intractable, due to the “curse of dimensionality,” a number of researchers in the computational intelligence society proposed a new approach to approximate dynamic programming. The result was a class of reinforcement learning techniques called adaptive critic techniques. The reinforcement learning approach to optimal control answers the question: “How do we map situations to actions so as to maximize the reward value?” [Be03]. In control engineering terms, answering this question translates into directly finding the optimal controller for the system at hand (i.e. direct adaptive optimal control). The adaptive critics approach to reinforcement learning assumes the existence of two interconnected parametric structures in the control system: • the Actor, which implements the control strategy in the feedback system, and • the Critic, which approximates the cost function with the purpose of evaluating the controller performance and taking corrective action on the control strategy. The algorithms, known as approximate dynamic programming (ADP) [W92] (or neuro-dynamic programming (NDP) [BT96], since the Actor and Critic structures are often modeled as neural networks), approach the optimal solution of the control problem in a forward-in-time fashion. The Actor-Critic architecture is presented in Figure 1. Initially the corrective action from the Critic had the form of a reinforcement signal indicating if the present control strategy was either successful or not. This can be appreciated as a “trial and error” approach to learning. The most successful approaches are not the ones based on trial and error but the ones which guarantee that the series of controller improvement steps will result in a sequence of performance improvement steps [AAL07, AL07, MCLS02]. The acceptance of these algorithms

Direct Adaptive Optimal Control

203

Fig. 1. Adaptive critic structure (Actor-Critic architecture).

in the industry will be strongly motivated by the mathematical proofs that offer the convergence guarantee, under certain regular assumptions, of these algorithms to the optimal control strategy. It has been pointed out in works such as [W92, WS92] that there exists a close relation between the various intelligent control strategies (i.e. control methods which involve, at some level, the process of cognition) and the way in which the nervous system works. In this chapter will be briefly outlined the relation between a few control strategies and the brain structure and mode of operation, with a focus on the approximate dynamic programming methods for continuous-time systems. In Section 3, we present two adaptive critic solutions for the continuous-time infinite horizon optimal control problem for linear systems. Details regarding the implementation of the algorithms are presented in Section 4. It will be shown that the two adaptive critic algorithms are in fact direct optimal adaptive control methods and that the resulting controllers can be viewed as dynamic regulators. Simulation results obtained for the design of an F-16 autopilot will be presented and conclusions will be drawn.

2 Control Engineering Solutions in Relation to the Brain Structure As was briefly outlined in the Introduction, the control systems theory was gradually developed, while answering different questions, and presently includes increasingly more intelligent approaches to control (controllers are hierarchically structured and are capable of adjusting the system behavior in response to the constant changes in the environment). In the same manner the human nervous system, and especially the brain, is organized in a hierarchy of multiple parallel loops [Bu06]. The information from the sensors will come through the spinal cord and pass through the thalamus which is under the control of the neocortex. The hippocampus is a relatively random synaptic space believed to be responsible for packaging the sensorial information that will be memorized [ATD06]. In the cerebellum and the basal ganglia there are strictly parallel loops. At the same time, in the cerebral cortex there exist a number

204

Draguna Vrabie and Frank Lewis

of long-range direct connections between the different loops. This physical organization clearly indicates a layered functionality; each level in the brain structure is performing a specific task that is integrated in the complex behavior of the individual. The main pathways in the nervous system are hardwired, but the strength of these connections can be changed based on the new sensorial information resulting from the interaction with the environment. It will not come as a surprise that all the types of control that have been mentioned in the previous section, and many other approaches that have not been specified, can be distinguished when one analyzes the mode of operation of the human nervous system. For example, the control loops at the reflex level can be viewed as hardwired proportional controllers. Also, the complex movements that can be performed by the human body require a learning process. The various types of learning mechanisms that appear in different regions of the brain have been discussed in [D99, D00a]. It appears that the cerebellum is specialized in supervised learning based on the error information coming from the inferior olive in the brain stem; this connectivity structure appears to be involved in motor learning. In relation to the control theory this type of structure is mostly related to the adaptive control techniques: the controller parameters are adapted to minimize the error difference between the desired output and the actual system output. The basal ganglia are specialized in reinforcement learning, an approach which was discussed in more detail in the previous section. The cerebral cortex, which learns based on the statistical properties of the information from sensors, is specialized in unsupervised learning and basically acts as a memory for the relevant actions (mistakes or successes) that were taken with time. The most important property of the mode of operation of the biological neural structures is that resulting solutions in response to external stimuli need to be, in some sense, characterized by optimality [LE97]. However, the brain does not learn how to compute optimal control strategies after acquiring complete knowledge of a system model. Instead it involves a trial-and-error learning process characterized by gradual improvement of a performance model and of the behavior associated with it; in control engineering terms this behavior is described as (direct) adaptive optimal control. The existence of a structure that should learn to describe optimality of behavior and use this information while performing complex tasks is thus required. These observations argue the connection between the Actor-Critic structure presented in Figure 1 and certain structures in the human brain (e.g. the basal ganglia). Oscillation is a fundamental property of neural tissue. Evidence regarding the oscillatory behavior naturally characterizing biological neural systems is presented in a comprehensive manner in [LBS00]. In fact the research supports the existence of multiple adaptive clocks in the brain [Bu06]. These neural oscillators operate on different frequencies in connection with the way in which the outside information from the sensors is processed by the central nervous system. An intelligent control structure must take fast actions in response to the outside world stimuli while at the same time it must observe and assess the results associated with a chosen behavior strategy. A step in the learning process consists in an adjustment of the control strategy for performing a required task. It is thus natural to conclude that, similarly with the way in which different brain structures process information at different frequencies, the

Direct Adaptive Optimal Control

205

Actor and Critic entities in an Actor-Critic control structure also need to operate at different frequencies due to the different nature of the tasks that they are performing. Following the intuition that comes from the above-mentioned observations we present here a continuous-time approach to adaptive critic. Two related algorithms, [VALW07, VPL07], based on the adaptive critic approach will be revisited. Both algorithms obtain online the optimal state feedback controller for a linear system without making use of a model describing the internal dynamics of the system. Thus we obtain adaptive control schemes for partially unknown linear systems. Moreover, the algorithm presented in [VALW07] does not require an initial stabilizing controller. In this formulation the Actor will perform in continuous time while the Critic will take actions to adapt the control policy, in the sense of performance optimization, at discrete moments in time. It will be shown that the new adaptive critic-based control scheme is in fact a dynamic controller with the state given by the cost or value function.

3 Continuous-Time Adaptive Critic Solutions Based on Policy Iteration for the Infinite Horizon Optimal Control Problem In this section are presented two iterative algorithms that were developed for directly solving the continuous-time linear-quadratic-regulator (LQR) problem without using knowledge regarding the system internal dynamics. 3.1 Problem Formulation Consider the linear time-invariant dynamical system described by x(t) ˙ = Ax(t) + Bu(t)

(1)

with x(t) ∈ Rn , u(t) ∈ Rm and (A, B) controllable, subject to the optimal control problem u∗ (t) = arg min V (t0 , x(t0 ), u(t)), (2) u(t) t0 ≤ t ≤ ∞

where the infinite horizon quadratic cost function is ∞

V (x(t0 ),t0 ) = (x(τ )T Qx(τ ) + u(τ )T Ru(τ ))d τ

(3)

t0

with Q ≥ 0, R > 0. The solution of this optimal control problem, determined by Bellman’s optimality principle [LS95], is given by u(t) = −Kx(t) with K = R−1 BT P,

(4)

206

Draguna Vrabie and Frank Lewis

where the matrix P is the unique positive definite solution of the algebraic Riccati equation (ARE) AT P + PA − PBR−1BT P + Q = 0. (5) It is known that the solution of the infinite horizon optimization problem can be obtained using the dynamic programming method and amounts to solving a finite horizon optimization problem backwards in time while extending the horizon to infinity. The following Riccati differential equation has to be solved: −P˙ = AT P + PA − PBR−1BT P + Q P(t f ) = Pt f ,

(6)

the solution of which will converge to the solution of the ARE for t f → ∞. But it is important to see that, in order to solve equation (6), complete knowledge of the model of the system is needed, i.e. both the system matrix A and control input matrix B must be known. However, the information regarding the system dynamics is regularly not available and a system identification procedure is required prior to solving the optimal control problem; a procedure which most often ends with finding an approximate model of the system. Thus, from the control systems point of view, we want to develop algorithms that will converge to the solution of the optimization problem without the need for prior system identification and the use of explicit models of the system dynamics. 3.2 Policy Iteration Approach to Direct Optimal Control 3.2.1 Background The solution of the optimal stabilization problem for linear systems (i.e. the unique positive definite solution of the ARE) can be obtained as a result of a Newton-type iteration procedure. The convergence guarantee of this iterative technique to the optimal controller (i.e. the solution of the LQR problem) was given in [K68] (for continuous-time formulation) and in [H71] (for the discrete-time case). The method requires repetitive solution of Lyapunov equations, and thus requires knowledge of the full system dynamics (i.e. the plant input and system matrices). For the purpose of obtaining optimal controllers without making use of a model of the system to be controlled, a new class of iterative methods, called policy iteration, was introduced for control applications in [BYB94, MCLS02]. These algorithms are often employed as an alternative to finding the solution of the optimal control problem by directly solving Bellman’s equation for the optimal cost, and then computing the optimal control policy (i.e. the feedback gain for linear systems). The policy iteration algorithm is built on the previously presented Actor-Critic structure and consists of incremental updates of the parameters of these two parametric structures in the sense of correct evaluation of the cost (Critic update: policy evaluation) and improvement of the system behavior (Actor update: policy improvement). The method starts by evaluating the cost associated with a given initial stabilizing

Direct Adaptive Optimal Control

207

control policy and then uses this information to obtain a new improved control policy. The two steps are repeated until the policy improvement step no longer changes the actual policy. The policy iteration technique was first formulated in the framework of stochastic decision theory [Ho60], and it has been extensively studied and employed for finding the optimal control solution for Markov decision problems of all sorts. [WS92, BT96] give a comprehensive overview of the research status in this field. For feedback control purposes, the policy iteration technique was used for finding the solution of the optimal control problem for continuous state linear systems in [BYB94, MCLS02]. These methods require initialization of the recursive algorithms with a stabilizing controller, the controller remaining stabilizing at every step of the iteration. Using iterative algorithms to solve for the state feedback optimal control policy, while working with linear systems, is particularly affordable since a sweep of the entire state space, required when solving Markov decision problems, is no longer necessary. In this case, the cost associated with a control policy can be easily determined using data along a single state trajectory. In [BYB94] a policy iterations algorithm was formulated that converges to the optimal solution of the discretetime LQR problem using Q-functions. By using the Q-functions [Wa89, W89], the algorithm does not require any knowledge of the system dynamics. For continuoustime systems, in [MCLS02] two partially model-free policy iteration algorithms were presented. The model-free quality of the approach was achieved either by evaluating the infinite horizon cost associated with a control policy along the entire stable state trajectory, or by using measurements of the state derivatives. The recently proposed [VPL07] continuous-time policy iteration formulation for linear time-invariant systems is given next. The equivalence with iterating on underlying Lyapunov equations is shown, as the policy iteration technique is in fact a Newton method for solving the Riccati equation. The update of the Critic structure results in calculating the infinite horizon cost associated with the use of a given stabilizing controller (i.e. it is equivalent to solving a Lyapunov equation). The Actor parameters (i.e. the controller feedback gain) are then updated in the sense of reducing the cost compared to the present control policy. 3.2.2 Policy Iteration Algorithm Let K be a stabilizing gain for (1), such that x˙ = (A − BK)x is a stable closed loop system. Then the corresponding infinite horizon quadratic cost is given by ∞

V (t, x(t)) = xT (τ )(Q + K T RK)x(τ )d τ =

,

(7)

t xT (t)Px(t)

where P is the real symmetric positive definite solution of the Lyapunov matrix equation (A − BK)T P + P(A − BK) = −(K T RK + Q) (8)

208

Draguna Vrabie and Frank Lewis

and V (x(t)) serves as a Lyapunov function for (1) with controller gain. The cost function (7) can be written as t+T 

V (x(t)) =

X T (τ )(Q + K T RK)X (τ )d τ + V (x(t + T )).

(9)

t

Based on (9), with the cost parameterized as V (x(t)) = xT (t)Px(t), considering an initial stabilizing control gain K0 , the following policy iteration scheme can be implemented online: t+T  T

x (t)Pi x(t) =

xT (τ )(Q + KiT RKi )x(τ )d τ + xT (t + T )Pi x(t + T )

(10)

t

Ki+1 = R−1 BT Pi .

(11)

Equations (10) and (11) formulate a new policy iteration algorithm motivated by the work of Murray et al. in [MCLS02]. Note that implementing this algorithm does not involve the plant matrix A. 3.2.3 Convergence Discussion The following results will establish the convergence of the proposed algorithm. Lemma 1. Solving for Pi in equation (10) is equivalent to finding the solution of the underlying Lyapunov equation ATi Pi + Pi Ai = −(KiT RKi + Q),

(12)

where Ai = A − BKi is stable. Proof. Since Ai is a stable matrix and KiT RKi + Q > 0 then there exists a unique solution of the Lyapunov equation (12), Pi > 0. Also, since xT Pi x is a Lyapunov function for the system x˙ = Ai x and d(xT (t)Pi x(t)) = xT (t)(ATi Pi + Pi Ai )x(t) = −xT (t)(KiT RKi + Q)x(t), dt

(13)

then the unique solution of the Lyapunov equation satisfies t+T 

t

xT (τ )(Q + KiT RKi )x(τ )d τ = −

t+T 

d(x(τ )T Pi x(τ )) dτ dτ

t

= xT (t)Pi x(t) − xT (t + T )Pi x(t + T )

i.e. equation (10). That is, provided that the closed Coop system is asymptotically stable, the solution of (10) is the unique solution of (12). 2

Direct Adaptive Optimal Control

209

Remark 1. Although the same solution is obtained for either equation (12) or (10), solving equation (10) does not require any knowledge of the system matrix A. From Lemma 1 it follows that the algorithm (10) and (11) is equivalent to iterating between (12) and (11), without using knowledge of the system internal dynamics. Let Ric(Pi ) be the matrix-valued function defined as Ric(Pi ) = AT Pi + PiA + Q − PiBR−1 BT Pi

(14)

and let RicPi denote the Fr´echet derivative of Ric(Pi ) taken with respect to Pi . The matrix function RicPi evaluated at a given matrix M will thus be RicPi (M) = (A − BR−1 BT Pi )T M + M(A − BR−1BT Pi ). Lemma 2. The iteration between (10) and (11) is equivalent to Newton’s method, Pi = Pi−1 − (RicPi−1 )−1 Ric(Pi−1 ).

(15)

Proof. Equations (12) and (11) are compactly written as ATi Pi + PiAi = −(Pi−1BR−1 BT Pi−1 + Q).

(16)

Subtracting ATi Pi−1 + Pi−1Ai on both sides gives and making use of the introduced notation Ric(Pi ) and RicPi , one obtains the Newton method formulation (15). 2 Theorem 1 (Convergence). The policy iteration (10) and (11) converges to the optimal control solution given by (4) where the matrix P satisfies the ARE (5). Proof. In [BT96] it was shown that using Newton’s method, conditioned by an initial stabilizing policy K0 , all the subsequent control policies will be stabilizing and the iteration between (12) and (11) will converge to the solution of the ARE. Since the equivalence between (12) and (11), and between (10) and (11) was shown, we can conclude that the proposed new policy iteration algorithm will converge to the solution of the optimal control problem (2) with the infinite horizon quadratic cost (3)—without knowledge of the internal dynamics of the controlled system (1). 2 3.3 Approximate Dynamic Programming Approach to Direct Optimal Control 3.3.1 Background ADP combines reinforcement learning Actor-Critic designs with dynamic programming to determine the solution of the optimal control problem using a forward-intime computation. Each iteration step consists of an update of the value function estimate, based on the current control policy, followed by a greedy update of the control policy based on the new value function estimation. However, compared with the policy iteration, in this approximate approach the value function that is determined at each step it is not necessarily the value function which corresponds to the present control policy but merely a heuristic approximation of it. Initially developed

210

Draguna Vrabie and Frank Lewis

for systems with finite state and action spaces the ADP methods were based on Sutton’s temporal difference method [S88], Watkins’s Q-learning [Wa89] and Werbos’s heuristic dynamic programming (HDP) [W92]. For the case of discrete-time systems with continuous state and action spaces different adaptive critic architectures were reported, with successful implementations and rigorous proofs; an incomplete list being [FS02,L97,LB00]. For the continuous-time case a dynamic programmingbased reinforcement learning scheme, formulated using the advantage function, was introduced in [B94]. Doya proposed in [D00b] reinforcement learning techniques based on the temporal difference method. The current status of work in ADP is given in [SBPW04]. In the works [L97] and [AAL07] there have been introduced greedy policy iteration algorithms which converge to the discrete-time H2 and H-infinity optimal state feedback control solution without the need for the stringent requirement for the controller to be stabilizing at each iteration step. In the following we present an adaptive optimal controller based on an Actor-Critic structure that allows solving the infinite horizon LQR problem (2), without any knowledge about the plant internal dynamics (matrix B is required) and without the requirement of an initial stabilizing policy. 3.3.2 Continuous-Time ADP Algorithm In this approach the infinite horizon cost of a policy is approximated as the summation of the observed reward while using a given control policy over a finite interval [t,t + T ] and an approximation of the infinite horizon cost from time t + T to ∞. This approximation depends only on the observed new state x(t + T ) and will be denoted by W (x(t + T )), t+T 

V (x(t)) =

(xT Qx + uT Ru)d τ + W (x(t + T )).

(17)

t

Equation (17) describes in fact the mechanism by which an intelligent structure learns: the performance associated with a given control policy is the sum of the observed reward over a period of time and the approximation of the future reward: the approximation was made based on the history of the system. This formalism is closely related to the rollout methods in [B05] and hence to receding horizon control. Yet in those methods the approximation W (x(t + T )) is not updated and, for this reason, restrictive conditions need to be imposed on this term (e.g. it needs to be a control lyapunov function). In contrast, in the ADP algorithm the W (x(t + T )) term is updated based on increased experience, along the philosophy presented in [SB98], thus no restrictions need to be imposed. In fact this term can simply be initialized as being equal to zero. Based on equation (17) the following greedy iteration scheme may be implemented online:

Direct Adaptive Optimal Control t+T 

Vi+1 (x(t)) =

(xT Qx + uTi Rui )d τ + Vi (x(t + T ))

211

(18)

t

ui = −R−1 BT Pi x = Ki x with the V -function parameterized as Vi (x) = will be written as: t+T  T

x (t)Pi+1 x(t) =

xT Pi x.

(19)

Explicitly these two equations

xT (τ )(Q + KiT RKi )x(τ )d τ + xT (t + T )Pi x(t + T )

(20)

t

Ki = R−1 BT Pi .

(21)

Both the policy iteration algorithm and the ADP method share the controller update strategy described in equations (11) and (21). The difference between the two algorithms is related to the way in which the Critic parameterization is updated— compare equations (10) and (20). In the policy iteration case the Critic exactly estimates the cost function associated with a given control policy, while in the case of the ADP algorithm the cost function corresponding to the present control policy is a heuristic approximation and not the real value. Since in the policy iteration case a Lyapunov equation needs to be solved, the control strategy needs to be stabilizing at each iteration step. However, for the ADP algorithm this stringent requirement is not necessary. 3.3.3 Convergence Discussion We now present a brief analysis of the ADP algorithm to provide a mathematical formulation and place it into the context of control system theory. While proceeding with the analysis it will become clear that this ADP method is in fact closely related to the well-known Newton method of searching the minimum in a convex setting. Lemma 3. The ADP iteration between (20) and (21) is equivalent (proof is given in [VALW07]) to the quasi-Newton method,   !T Pi+1 = Pi − (RicPi )−1 Ric(Pi ) − eAi T Ric(Pi )eAi T . (22) Comparing equation (22) with the formulation of Newton’s method, (15), for finding the solution of the matrix ARE, one will note that the only difference is given by the last term appearing in equation (22). This quasi-Newton algorithm will give successful results while being initialized in a less restrictive setting than the classical Newton method (i.e. the restriction of an initial stabilizing controller is no longer valid since the iteration is no longer performed on Lyapunov equations as in [K68]). It seems that the last term, appearing in the quasi-Newton formulation of the algorithm, plays an important role in canceling this requirement.

212

Draguna Vrabie and Frank Lewis

Lemma 4. The iteration between (20) and (21) is equivalent (proof in [VALW07]) to T Pi+1 = Pi + eAitT Ric(Pi )eAit dt.

(23)

0

Lemma 5. Let the ADP algorithm converge so that Pi → P∗ . Then P∗ satisfies Ric(P∗ ) = 0, i.e. P∗ is the solution of the continuous-time ARE. The proof follows directly from the result in Lemma 4.

4 Online Implementation of the Adaptive Optimal Control Algorithms Without Knowledge of the System Internal Dynamics For the implementation of the iteration scheme given by (10) and (11) (the policy iteration algorithm) as well as the iteration given by equations (20) and (21) one only needs to have knowledge of the B matrix as it explicitly appears in the policy update. The information regarding the system A matrix is embedded in the states x(t) and x(t + T ) which are observed online, and thus the system matrix is not required for the computation of either of the two steps of the policy iteration scheme. The details regarding the online implementation of the algorithm are discussed next. Simulation results obtained while finding the optimal controller dynamics for an F-16 autopilot are then presented. 4.1 Online Implementation of the Adaptive Algorithm Based on Policy Iteration 4.1.1 Mathematical Setup for Solving the Critic Update To find the parameters (matrix Pi ) of the cost function for the policy Ki in (10) and (20), the term xT (t)Pi x(t) is written as xT (t)Pi x(t) = p¯Ti x(t), ¯

(24)

where x(t) ¯ is the Kronecker product quadratic polynomial basis vector with the elements {xi (t)x j (t)}i=1,n; j=i,n and p¯ = ν (P) with ν (.) a vector-valued matrix function that acts on n × n symmetric matrices and gives a column vector by stacking the elements of the diagonal and upper triangular part of the symmetric matrix into a vector where the off-diagonal elements are taken as 2Pi j [Br78]. Using (24) equation (10) is rewritten as t+T 

p¯Ti (x(t) ¯ − x(t ¯ + T ))

= t

xT (τ )(Q + KiT RKi )x(τ )d τ .

(25)

Direct Adaptive Optimal Control

213

In equation (25), p¯i is the vector of unknown parameters and x(t) ¯ − x(t ¯ + T ) acts as a regression vector. The right-hand side target function, denoted d(x(t), ¯ Ki ), t+T 

d(x(t), ¯ Ki ) ≡

xT (τ )(Q + KiT RKi )x(τ )d τ ,

t

is measured based on the system states over the time interval [t,t + T ]. In the same manner, equation (20) in the ADP algorithm will be written as t+T 

¯ = p¯Ti+1 x(t)

xT (τ )(Q + Pi BR−1 BT Pi )x(τ )d τ + p¯ Ti x(t ¯ + T ).

(26)

t

¯ acts as a In equation (26), p¯i+1 is the vector of unknown parameters and x(t) regression vector. The right-hand side target function can be calculated based on the measurements of the system states. Considering V˙ (t) = xT (t)Qx(t) + uT (t)Ru(t) as a definition for a new state, augmenting the system (1), the value of the integral in d(x(t), ¯ Ki ) can simply be measured by taking two measurements of this newly t+T  introduced system state since xT (τ )(Q + Pi BR−1 BT Pi )x(τ )d τ = V (t + T ) −V (t). t

At each iteration step, after a sufficient number of state trajectory points are collected using the same control policy Ki , a least-squares method can be employed to solve for the V -function parameters (i.e. the Critic). The parameter vector p¯i is found by minimizing, in the least-squares sense, the error between the target function (i.e. the right-hand side of the equations (25) respectively (26)), and the parameterized left-hand side of (25) and (26). Evaluating the right-hand side of (25) at N ≥ n(n + 1)/2 (the number of independent elements in the matrix P) points x¯i in the state space, over the same time interval T , the least-squares solution is obtained as p¯i = (XX T )−1 XY,

(27)

where X = [ x¯1Δ x¯2Δ . . . x¯NΔ ] x¯iΔ = x¯i (t) − x¯i (t + T ) Y = [d(x¯1 , Ki ) d(x¯2 , Ki ) . . . d(x¯N , Ki )]T . The least-squares problem can be solved in real time after a sufficient number of data points are collected along a single state trajectory. Similarly, for the ADP Critic update the solution will be given by p¯i+1 = (XX T )−1 XY,

(28)

214

Draguna Vrabie and Frank Lewis

where X = [ x¯1 x¯2 ... x¯N ] Y = [ d(x¯1 , Ki ) d(x¯2 , Ki ) ... d(x¯N , Ki ) ]T and d(x, ¯ Ki ) is the right-hand side of equation (26). Alternatively, the solution given by (27) or (28) can also be obtained using recursive estimation algorithms (e.g. gradient descent algorithms, recursive least-squares algorithm) in which case a persistence of excitation condition is required. Implementation of both the algorithms requires only measurements of the states at discrete moments in time, t and t + T , as well as knowledge of the observed cost over the time interval [t,t + T ], which is d(x(t), ¯ Ki ). Therefore there is no required knowledge about the system A matrix for the evaluation of the cost or the update of the control policy. However, the B matrix is required for the update of the control policy, using (11), and this makes the tuning algorithm only partially model free. 4.1.2 Structure of the Direct Adaptive Controller The structure of the system with the adaptive controller is presented in Figure 2. Most important is that the system was augmented with an extra state V (t), which satisfies V˙ = xT Qx + uT Ru, in order to extract the information regarding the cost associated with the given policy. This newly introduced system dynamics is part of the adaptive critic based controller. Thus the control scheme is actually a dynamic controller with the state given by the cost function V . It is shown that having little information about the system states, x, and the augmented system state, V (controller dynamics), extracted from the system only at specific time values (i.e. the algorithm uses only the data samples x(t), x(t + T ) and

Critic ZOH T

T

T

V V = xT Qx + uT Ru

Actor −K

u

System

x

x = Ax + Bu; x0

Fig. 2. Structure of the system with adaptive controller

Direct Adaptive Optimal Control

215

V (t + T ) − V (t)), the critic is able to evaluate the performance of the system associated with a given control policy. Then the policy is improved at time t + T , after observing the state x(t + T ), and it is used for controlling the system during the time interval [t + T ,t + 2T ]; thus the algorithm is suitable for online implementation from the control theory point of view. In this way, over a single state trajectory in which several policy evaluations and updates have taken place, the algorithm can converge to the optimal control policy. The Critic will stop updating the control policy when the error between the performance of the system evaluated at two consecutive steps crosses below a designerspecified threshold, i.e. the algorithm has converged to the optimal controller. Also if this error is bigger than the above-mentioned threshold the Critic will again take the decision to start tuning the actor parameters. It is observed that the update of both the Actor and the Critic is performed at discrete moments in time. However, the control action is a full-fledged continuoustime control, except that its constant gain is updated only at certain points in time. Figure 3 are presents the control policy, Ki , and control signal, u(t), for the case of a single-input and single-state system. The Critic update is based on the observations of the continuous-time cost over a finite sample interval. As a result, the algorithm converges to the solution of the continuous-time optimal control problem, as was shown in Section 3.

Control Policy update

Ki

0

1

2

3

4

5

i

Control signal u i (t ) = Ki x(t )

0

1

2

3

4

5

t

Fig. 3. Continuous-time control with discrete-time controller gain updates (sample periods need not be the same).

216

Draguna Vrabie and Frank Lewis

4.2 Online Design of an F-16 Autopilot In this section we present the simulation results obtained for finding the optimal controller for the short period dynamics of an F-16 aircraft. We consider the linear model given in [SL03]. The system state vector is x = [ α q δe ], where α denotes the angle of attack, q is the pitch rate and δe is the elevator deflection angle. The control input is the elevator actuator voltage. The matrices that describe the linearized system dynamics are ⎤ ⎡ −1.01887 0.90506 −0.00215 A = ⎣ 0.82225 −1.07741 −0.1755 ⎦ 0 0 −20.2 ⎤ 0 B = ⎣ 0 ⎦. 20.2 ⎡

The simulations were conducted using data obtained from the system every 0.05 s. The cost function parameters, namely the Q and R matrices, were chosen to be identity matrices of appropriate dimensions. For the purpose of demonstrating the algorithm the initial state of the system is taken to be different than zero. 4.2.1 Policy Iteration Result Since the system to be controlled is stable, the algorithm is initialized without a controller (i.e. K0 = 0). In order to solve online for the values of the P matrix which parameterizes the cost function, before each iteration step one needs to set up a least-squares problem of the sort described in Section 4.1.1 with the solution given by (27). Since there are six independent elements in the symmetric matrix P we set up the least-squares problem by measuring the cost function associated with the given control policy over six time intervals T = 0.05 s, the initial state and the system state at the end of each time interval. In this way, at each 0.3 s, enough data is collected from the system to solve for the value of the matrix P and perform a policy update. The result of applying the algorithm for the F-16 system is presented in Figure 4. The simulation experiment was performed along the state trajectory presented in Figure 5. From Figure 4 it is clear that the system controller comes close to the optimal controller after only two iteration steps are performed. The update of the controller was not performed after the third iteration since the difference between the measured cost and the expected cost went below the specified threshold of 0.00001. The value of the obtained P matrix obtained after the third iteration is ⎤ ⎡ L1.4117 1.1540 −0.0072 P = ⎣ 1.1540 1.4191 −0.0087 ⎦ , −0.0072 −0.0087 0.0206

Direct Adaptive Optimal Control P matrix parameters 2

1.5

1

P(1,2) P(2,2) P(3,3) P(1,2)-optimal P(2,2)-optimal P(3,3)-optimal

0.5

0 0

0.5

1 Time (s)

1.5

2

Fig. 4. Parameters of the P matrix converging to the optimal values. System states

8

X(1) X(2) X(3)

6

4

2

0

–2

–4

0

0.5

1 Time (s)

Fig. 5. System state trajectories.

1.5

2

217

218

Draguna Vrabie and Frank Lewis

while the value of the optimal P matrix, i.e. the solution of the ARE, is ⎤ ⎡ L1.4116 1.1539 −0.0072 P = ⎣ 1.1539 1.4191 −0.0087 ⎦ . −0.0072 −0.0087 0.0206 Thus, after 0.9 s, the system will be controlled in an optimal fashion with the optimal controller which was obtained online without any knowledge about the system’s internal dynamics. In practice, the convergence of the algorithm is considered to be achieved when the difference between the measured cost and the expected cost crosses below a designer-specified threshold value. It is important to note that after the convergence to the optimal controller is attained, the algorithm need not continue to run and subsequent updates of the controller need not be performed. 4.2.2 Approximate Dynamic Programming Result Since this ADP algorithm does not require initialization with a specific controller it was conveniently initialized with a zero controller (i.e. K0 = 0). In this case, to solve for the parameters of the cost function at each iteration step a least-squares problem was set up with the solution given by (28). In order to improve the numerical stability of the simulated algorithm, the time interval that was used in this case was T = 0.1 s. Thus, at each 0.6 s, enough data is collected from the system to solve for the six unknown parameters of the matrix P and perform a policy update. The convergence result of applying the algorithm for the F-16 system is presented in Figure 6 and the state trajectory of the system is presented in Figure 7. From Figure 6 one can observe that convergence of the cost function was obtained after 70 seconds of simulation. Although this is a relatively short time interval, it is significantly longer than that in the policy iteration case. Comparing the two results, it is clear that when the policy iteration algorithm is used (i.e. the Newton method) the first iteration is very important to bring the Critic and Actor parameters close to the real optimal values, while for the case in which the ADP algorithm is used we can observe a gradual improvement in the values of the two parametric structures. However, even if the policy iteration approach seems to have a faster convergence rate, the fact that it must always be initialized with a stabilizing controller is a stringent requirement.

5 Conclusions In this chapter we revisited two iterative techniques based on an adaptive critic scheme to solve online the continuous-time LQR problem without using knowledge about the system’s internal dynamics (system matrix A). The Actor structure performs continuous-time control while the Critic incrementally corrects the actor’s behavior at discrete moments. The Critic formulates the Actor performance in a

Direct Adaptive Optimal Control

P matrix parameters 1.5

1 P(1,2) P(2,2) P(3,3) P(1,2)-optimal P(2,2)-optimal P(3,3)-optimal

0.5

0

0

10

20

30

40 Time (s)

50

60

70

Fig. 6. Parameters of the P matrix converging to the optimal values.

System states

8

6

4

2

0

–2

–4

0

2

4

6

8

Time (s) Fig. 7. System state trajectories for the first 10s of the simulation.

10

219

220

Draguna Vrabie and Frank Lewis

parameterized form. Based on the Critic’s evaluation the Actor’s policy is updated to achieve better control performance. Both the algorithms effectively provide a solution to the ARE associated with the optimal control problem without using knowledge of the system matrix A. The algorithms have been proven to be equivalent with Newton’s method (the policy iteration algorithm) and a quasi-Newton method (for the ADP algorithm) of searching for the minimum in a convex setting. At the same time, we offered some evidence which supports the existence of a close connection between the brain structure and functionality and the control structures and methods used in an engineering environment. The existence of this relation might prove to be advantageous for both control engineering researchers, who can find new inspiration for developing intelligent controllers able to adapt to changes in the environment (the brain is the most efficient and robust control system), as well for brain researchers in their attempt to understand and express mathematically the mechanisms involved in the cognition process. From the control engineering point of view, these algorithms are a first step in developing a new approach to determining optimal controllers for nonlinear plants in a less restrictive setup (e.g. no initial stabilizing controller required, no model of the nonlinear system to be controlled necessary); in this case the Actor and Critic structures will probably be represented as artificial neural networks.

Acknowledgments The authors wish to acknowledge Dr. Murad Abu-Khalaf and Prof. Octavian Pastravanu for their ideas, criticism and support during the research that led to the results presented here. This work was supported by the National Science Foundation ECS0501451 and the Army Research Office W91NF-05-1-0314.

References [AAL07]

[AL07]

[ATD06] [B94]

[Be03]

A. Al-Tamimi, M. Abu-Khalaf, and F. L. Lewis, Model-free Q-learning designs for discrete-time zero-sum games with application to H-infinity control, Automatica, Vol 43, No. 3, pp. 473–482, 2007. A. Al-Tamimi and F. Lewis, Discrete-time nonlinear HJB solution using approximate dynamic programming: Convergence proof, Proc. of IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007, pp. 38–43, April 2007. A. Al-Chalabi, M. R. Turner, and R. S. Delamond, The Brain—A Beginner’s Guide, Oneworld Publications, Oxford, England, 2006. L. Baird, Reinforcement learning in continuous time: Advantage updating, Proceedings of the International Conference on Neural Networks, Orlando, FL, June 1994. R. Bellman, Dynamic Programming, Dover Publications, Mincola, NY, 2003.

Direct Adaptive Optimal Control [B05]

221

D. P. Bertsekas, Dynamic programming and suboptimal control: A survey from ADP to MPC, Proceedings of CDC’05, 2005. [BT96] D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, Belmont, and MA, 1996. [BYB94] S. J. Bradtke, B. E. Ydestie, and A. G. Barto, Adaptive linear quadratic control using policy iteration, Proceedings of the American Control Conference, pp. 3475–3476, Baltimore, MD, June 1994. [Br78] J. W. Brewer, Kronecker products and matrix calculus in system theory, IEEE Trans. on Circuits and Systems, Vol. 25, No. 9, 1978. [Bu06] G. Buzsaki, Rhythms of the Brain, Oxford University Press, London, 2006. [D99] K. Doya, What are the computations of the cerebellum, the basal ganglia and the cerebral cortex, Neural Networks, Vol. 12, pp. 961–974, 1999. [D00a] K. Doya, Complementary roles of basal ganglia and cerebellum in learning and motor control, Current Opinion in Neurobiology, Vol. 10, pp. 732–739, 2000. [D00b] K. Doya, Reinforcement learning in continuous time and space, Neural Computation, vol. 12, pp. 219–245, MIT Press, Cambridge, MA, 2000. [DKK01] K. Doya, H. Kimura, and M. Kawato, Neural mechanisms of learning and control, IEEE Control Systems Magazine, pp. 42–54, Aug. 2001. [FS02] S. Ferrari, and R. Stengel, An adaptive critic global controller, Proceedings of the American Control Conference, pp. 2665–2670, Anchorage, AK, 2002. [H71] G. Hewer, An iterative technique for the computation of the steady state gains for the discrete optimal regulator, IEEE Trans. on Automatic Control, Vol. 16, pp. 382–384, 1971. [Ho60] R. A. Howard, Dynamic Programming and Markov Processes, MIT Press, Cambridge, MA, 1960. [K68] D. Kleinman, On an Iterative technique for Riccati equation computations, IEEE Trans. on Automatic Control, Vol. 13, pp. 114–115, 1968. [L97] T. Landelius, Reinforcement learning and distributed local model synthesis, PhD Dissertation, Linkoping University, Sweden, 1997. [LE97] D. S. Levine and W. R. Elsberry, eds., Optimality in Biological and Artificial Networks?, Lawrence Erlbaum Assoc., Mahwah, NJ, 1997. [LBS00] D. S. Levine, V. R. Brown, and V. T. Shirey, eds., Oscillations in Neural Systems, Lawrence Erlbaum Assoc., Mahwah, NJ, 2000. [LS95] F. L. Lewis and V. L. Syrmos, Optimal Control, John Wiley, New York, 1995. [LB00] X. Liu and S. N. Balakrishnan, Convergence analysis of adaptive critic based optimal control, Proceedings of the American Control Conference, pp. 1929–1933, Chicago, IL, 2000. [MCLS02] J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, Adaptive dynamic programming, IEEE Trans. on Systems, Man and Cybernetics, Vol. 32, No. 2, pp. 140– 153, 2002. [SBPW04] J. Si, A. Barto, W. Powel, and D. Wunch, Handbook of Learning and Approximate Dynamic Programming, John Wiley, Hoboken, NJ, 2004. [SL03] B. L. Stevens and F. L. Lewis, Aircraft Control and Simulation, Wiley, Hoboken, NJ, 2nd Edition, 2003. [SB98] R. S. Sutton and A. G. Barto, Reinforcement Learning–An Introduction, MIT Press, Cambridge, MA, 1998. [S88] R. Sutton, Learning to predict by the method of temporal differences, Machine Learning, Vol. 3, pp. 9–44, 1988.

222

Draguna Vrabie and Frank Lewis

[VALW07] D. Vrabie, M. Abu-Khalaf, F. Lewis, and Y. Wang, Continuous-time ADP for linear systems with unknown dynamics, Proc. of IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, ADPRL 2007, pp. 247– 253, Hawaii, April 2007. [VPL07] D. Vrabie, O. Pastravanu, and F. Lewis, Policy iteration for continuous-time systems with unknown internal dynamics, Proc. of Mediterranean Conference on Control and Automation, MED’07, Athens, June 2007. [Wa89] C. J. C. H. Watkins, Learning from delayed rewards. PhD Thesis, University of Cambridge, England, 1989. [W89] P. Werbos, Neural networks for control and system identification, IEEE Proc. CDC89, IEEE, 1989. [W92] P. J. Werbos, Approximate dynamic programming for real-time control and neural modeling, Handbook of Intelligent Control, D. A. White and D. A. Sofge eds., Van Nostrand, New York, 1992. [WS92] D.A. White and D.A. Sofge, eds., Handbook of Intelligent Control, Van Nostrand Reinhold, New York, 1992. [WW95] B. Widrow and E. Walach, Adaptive Inverse Control, Prentice-Hall, Englewood Cliffs, NJ, 1995.

Characterization and Calculation of Approximate Decentralized Fixed Modes (ADFMs) Edward J. Davison1 and Amir G. Aghdam2 1 2

Dept. of Elect. and Comp. Eng, University of Toronto, Toronto, Ontario M5S 1A4, Canada. [email protected] Dept. of Elect. and Comp Eng., Concordia University, Montreal, Quebec H3G 1M8, Canada. [email protected]

Summary. It is well known that a linear time-invariant (LTI) system can be stabilized using decentralized LTI control if and only if the system does not possess any unstable decentralized fixed modes (DFMs). However, in industrial system application studies, it often is the case that a system has no DFMs, but may have approximate DFMs (ADFMs), which are modes that are not DFMs, but are “close” to being DFMs; in particular, such ADFMs can be divided into two types: “structured” ADFMs and “unstructured” ADFMs. In general ADFMs have the property that, although they are not fixed, they may require a “huge control energy” to shift the modes to desirable regions of the complex plane, which may be impossible to obtain. It is thus important to be able to characterize and determine what modes of a system, if any, are ADFMs, and this is the focus of the chapter. A number of industrial application problems will be used to demonstrate the effectiveness of the proposed algorithms for the calculation of ADFMs and to illustrate the properties of these ADFMs.

1 Introduction This chapter is concerned with different quantitative measures for approximate decentralized fixed modes (ADFMs); i.e., those modes of decentralized control systems which are not fixed, but are close to being fixed. In centralized control systems, the Hankel norm is often used to characterize the closeness of modes to uncontrollability and/or unobservability, which in turn indicates how close the mode is to being fixed. Note that the Hankel norm is defined only for stable systems, and is derived from the controllability and observability Gramians, which are not defined for decentralized systems. Decentralized fixed modes (DFMs) were introduced by Wang and Davison in [WaD73] to characterize those modes of linear time-invariant (LTI) systems which cannot be shifted by means of LTI decentralized controllers. Several methods are

C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 11, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

224

Edward J. Davison and Amir G. Aghdam

proposed in the literature to characterize DFMs, e.g. see [DaC90], [LaA07]. The notion of a structurally fixed mode was introduced in [SeS81] to identify those DFMs that result from the structure of the state-space matrices, rather than from the perfect matching of the nonzero parameters of the system. Quotient fixed modes (QFMs) were later introduced in [GoA97] to characterize those DFMs which are also fixed with respect to any type of decentralized control law, i.e. a nonlinear time-varying controller. The notion of a DFM is essential in determining whether or not a system is stabilizable with respect to the class of LTI decentralized controllers. The question arises: in the case when the mode is not fixed, how difficult is it (in terms of the required control effort and output performance) to shift the mode? This motivates the importance of developing a quantitative measure for the closeness of a mode to being fixed. The notion of an approximate decentralized fixed mode was introduced in [VaD89] to characterize the “flexibility” of modes in a decentralized control system. As a result, an unstable mode which is not “flexibile” or, equivalently, is close to being fixed, may require a huge control energy to be shifted to the stability region by means of LTI decentralized controllers. This idea was later extended to nonlinear time-varying controllers in [AgD08] by defining approximate quotient fixed modes (AQFMs). It is to be noted that for centralized control systems, approximate centralized fixed modes can be characterized by using a Hankel norm. A mode which has a large Hankel norm has a large controllability and observability Gramian, which implies that the mode is far from being fixed. Such modes can be easily shifted by employing a proper centralized controller. The quantitative measure used in [VaD89] for the characterization of ADFMs is useful in comparing different modes of a system in terms of their “flexibility.” However, there is no concrete result available to identify how “closeness to being fixed” constitutes an ADFM of large magnitude. In this chapter, different measures for the quantitative characterization of ADFMs will be introduced. These measures are then compared analytically, and thence are applied to various industrial multiinput, multi-output (MIMO) systems to demonstrate the efficacy of the proposed analysis. This chapter is organized as follows. In Section 2, a brief description of DFMs based on the transmission zeros of a set of matrices as defined in [VaD89] is used to describe the problem under investigation. In Section 3, three different measures for characterization and calculation of ADFMs are given, based on the condition numbers, singular values, and the distance of the modes from the transmission zeros. Section 4 considers the special case of a centralized control system and presents an alternative technique to evaluate approximate centralized fixed modes in this case, using the results obtained for characterizing ADFMs. Simulation results are presented in Section 5 to elucidate the proposed methods and compare the results. Some concluding remarks are given in Section 6.

Characterization and Calculation of ADFMs

225

2 Problem Statement Consider an LTI system with ν local control stations given by ⎡ ⎤ u1 (t) ⎢  ⎥ x(t) ˙ = Ax(t) + B1 · · · Bν ⎣ ... ⎦ uν (t) ⎡ ⎤⎡ ⎤ ⎡ ⎤ ⎤, y1 (t) D11 · · · D1ν u1 (t) C1 ⎢ .. ⎥ ⎢ .. ⎥ ⎢ . . ⎥⎢ . ⎥ ⎣ . ⎦ = ⎣ . ⎦ x(t) + ⎣ .. . . . .. ⎦ ⎣ .. ⎦ Cν Dν 1 · · · Dνν yν (t) uν (t) ⎡

(1)

where x(t) ∈ Rn is the state vector, ui (t) ∈ Rmi and yi (t) ∈ Rri are the input and output, respectively, of the ith control station (i = 1, . . . , v), and where A ∈ Rn×n , Bi ∈ Rn×mi , Ci ∈ Rri ×n , and Di j ∈ Rri ×m j (i, j = 1, . . . , ν ) are the system matrices. For simplicity, assume that all control stations are single-input, single-output (SISO) (i.e., mi = ri = 1, i = 1, . . . , ν ); in this case the matrices Bi and Ci are denoted by bi and ci , respectively, to emphasize the fact that they are column and row vectors. The following result is obtained from [DaC90]. Lemma 1. Given the system (1) with mi = ri = 1, i = 1, . . . , v, then λ ∈ sp(A) is a decentralized fixed mode of (1) with respect to the diagonal information flow K given below: ⎡ ⎤ K1 0 ⎢ ⎥ .. K := {K ∈ Rν ×ν |K = ⎣ ⎦, . 0

Kν

Ki ∈ R , i = 1, . . . , ν , det(I − DK) = 0},

(2)

⎡

⎤ D11 · · · D1ν ⎢ ⎥ D := ⎣ ... . . . ... ⎦ Dν 1 · · · Dνν

where

(3)

if and only if λ is a transmission zero of all of the following systems [DaC90]: (ci , A, bi )

(4)

i = 1, 2, . . . , ν 

     ci 0 di j , A, bi b j , cj d ji 0

i = 1, 2, . . . , ν − 1; j = i + 1, i + 2, . . . , ν

(5)

226

Edward J. Davison and Amir G. Aghdam

⎡ ⎤⎞ ⎤ 0 di j dik ci   ⎝⎣ c j ⎦ , A, bi b j bk , ⎣ d ji 0 d jk ⎦⎠ ck dki dk j 0 ⎛⎡

(6)

i = 1, 2, . . . , ν − 2; j = i + 1, i + 2, . . . , ν − 1; k = j + 1, j + 2, . . . , ν .. . ⎛⎡

⎡ ⎤ c1 0 ⎢ d21 ⎜⎢ c2 ⎥   ⎢ ⎜⎢ ⎥ ⎜⎢ .. ⎥ , A, b1 b2 · · · bν , ⎢ .. ⎣ . ⎝⎣ . ⎦ cν

d12 0 .. .

··· ··· .. .

⎤⎞ d 1ν ⎟ d 2ν ⎥ ⎥⎟ ⎥ .. ⎟ , . ⎦⎠

(7)

d ν 1 dν 2 · · · 0

and it is shown in [WaD73] that the DFMs of the system (1) remain fixed with respect to any dynamic LTI controller (note however that these modes may not be fixed with respect to a nonlinear or time-varying controller). Remark 1. Note that the total number of matrices in (4) to (7) is given by       ν ν ν + + ···+ . ν¯ = ν 1 2 It can be easily verified that ν¯ = 2ν − 1. Remark 2. For the case when the subsystems (Ci , A, Bi ), i = 1, . . . , ν are not SISO or has an arbitrary structure, the information flow matrix (2) is block diagonal (as opposed to diagonal), one can use the expansion technique [VaD89] to convert the system to an expanded model (with some redundant inputs and outputs), where all inputs and outputs as well as the elements of the information flow matrix (2) are scalar. This will be clarified later through an example. In order to characterize those modes of the system which remain fixed with respect to any type of decentralized control law (i.e., nonlinear or time-varying), the concept of DFM was used in [GoA97] to define quotient fixed modes (QFMs). A brief description of QFM as defined in [GoA97] will be presented here. Consider the system (1), and for simplicity assume that the system is strictly proper (Di j = 0, i, j = 1, . . . , ν . The digraph of this system consists of a set of ν nodes and a set of directed branches which connect these nodes. Each node represents a component of the system to which a control agent is assigned, and each directed branch represents a connection between a pair of nodes. If the transfer function from input j to output i is nonzero (i.e., Ci (sI − A)−1B j = 0), then there exists a directed branch from node j to node i (i, j = 1, 2, . . . , ν ). Consider a subsystem of (1) consisting of a subset of the nodes (1, 2, . . . , ν ) with the property that for each distinct pair of nodes i, j in that subset, there exists a directed path from node i to node j and also a path from node j to node i (a directed path consists of one or more directed branches). This subsystem

Characterization and Calculation of ADFMs

227

is called a strongly connected subsystem of (1) [KoY82] (usually the term “strongly connected” refers to the digraph of a system instead of the system itself). A system can always be decomposed uniquely into a number of strongly connected subsystems, which have the property that if one more node from the graph is added to any of the strongly connected subsystems, then that subsystem will no longer be strongly connected. Corresponding to the decomposition of a system into the above mentioned unique strongly connected subsystems, one can define a new decentralized system with a control agent assigned to each subsystem, which is in fact the quotient system for (1) [KoY82]. As an illustration of this definition, consider a decentralized control system with the digraph of Figure 1. This system consists of three subsystems, and only two of these subsystems (subsystems 1 and 2) are strongly connected. The corresponding quotient system is depicted in Figure 2. Given the system (1), the DFMs of the corresponding quotient system are called quotient fixed modes (QFMs) of the system (1). It is shown in [GoA97] that a DFM of the system (1) cannot be shifted using a time-varying or nonlinear decentralized controller if and only if the DFM is a QFM. From the definitions of DFM and QFM, one can conclude that these notions play an essential role in determining if an LTI plant can be stabilized by applying a decentralized LTI or a non-LTI controller. In particular, an LTI plant can be stabilized using a decentralized LTI (non-LTI) controller if and only if the plant has no unstable DFMs (QFMs). While the notions of DFM and QFM are theoretically appealing, in most practical applications it is often the case that a system has no DFMs (or QFMs), but may have some modes which are “close” to being DFMs (or QFMs). Such modes are referred to as approximate decentralized fixed modes (ADFMs) or approximate

1

2

3

Controller 1

Controller 2

Controller 3

Fig. 1. A decentralized control system

1

2

Controller 1’

3

Controller 2’

Fig. 2. A quotient system

228

Edward J. Davison and Amir G. Aghdam

quotient fixed modes (AQFMs) [VaD89], [AgD08]. Usually, systems with an ADFM or an AQFM result from a slight perturbation of the parameters of a system which has a DFM or QFM. A system with an unstable ADFM can in principle be stabilized by a proper LTI decentralized controller. However, the resultant control performance will be poor; in such cases, it can turn out that a nonlinear or time-varying decentralized controller may outperform its LTI counterpart [AgD08] depending on whether the ADFM is an AQFM or not. One can analyze the subsystems (4)–(7) to obtain a quantitative measure of “closeness” of a mode to being a DFM. Different approaches can be employed for this purpose, and these approaches will be studied in the next section.

3 Approximate Decentralized Fixed Modes (ADFMs) Consider the system (1), and corresponding to the set of related subsystems given in (4)–(7), define the following matrices:   A − λ I bi Mi (λ ) := (8) ci 0 i = 1, 2, . . . , ν ⎡

⎤ A − λ I bi b j 0 di j ⎦ Mi, j (λ ) := ⎣ ci c j d ji 0

(9)

i = 1, 2, . . . , ν − 1; j = i + 1, i + 2, . . . , ν ⎡

A−λI ⎢ ci Mi, j,k (λ ) := ⎢ ⎣ cj ck

bi 0 d ji dki

bj di j 0 dk j

⎤ bk dik ⎥ ⎥ d jk ⎦ 0

(10)

i = 1, 2, . . . , ν − 2; j = i + 1, i + 2, . . . , ν − 1; k = j + 1, j + 2, . . . , ν .. . ⎡

A−λI ⎢ c1 ⎢ ⎢ M1, 2, ...,ν (λ ) := ⎢ c2 ⎢ .. ⎣ . cν

b1 0 d21 .. .

b2 d12 0 .. .

··· ··· ··· .. .

⎤ bν d 1ν ⎥ ⎥ d 2ν ⎥ ⎥. .. ⎥ . ⎦

(11)

dν 1 dν 2 · · · 0

Three different approaches for evaluating ADFMs using the above set of matrices will be provided next.

Characterization and Calculation of ADFMs

229

3.1 Condition Number One can obtain the condition number of the respective matrices in (8)–(11) for a given mode of the system, and determine the minimum condition number among all of these matrices as a quantitative measure for that mode. Note that if this minimum condition number is infinity, the corresponding mode is, in fact, an exact DFM. Definition 1. Consider the system (1). The mode λ ∈ sp(A) is said to be a mode of magnitude Measure1 (λ ), where Measure1 (λ ) := min{{cond(Mi(λ )) , i = 1 , . . . , ν }, {cond(Mi, j (λ )) , i = 1 , . . . , ν − 1; j = i + 1 , . . . , ν } , {cond(Mi, j,k (λ )) , i = 1 , . . . , ν − 2; j = i + 1 , . . . , ν − 1; k = j + 1 , . . . , ν } , . . . , {cond(M1, 2, ...,ν ) } } , (12) and where cond(.) represents the condition number (which is the ratio of the largest singular value of a matrix to the smallest). In this case, the larger Measure1 (λ ) is, the closer λ is to being a DFM (i.e., λ is a DFM if and only if Measure1 (λ ) = ∞). If Measure1 (λ ) is “large,” then the system is said to have an ADFM of magnitude Measure1 (λ ) at λ . A similar definition of ADFM can be made when the information flow matrix is not diagonal (in which case, one can use the expansion method [VaD89] mentioned before), and also when the inputs and outputs of a control station are multivariable. 3.2 Singular Value Alternatively, the singular values of the respective matrices in (8)–(11) for a given mode of the system can be used as a quantitative measure for the closeness of that mode to being a DFM. If zero is a singular value of all of the matrices in (8)–(11) for a given mode, the corresponding mode is, in fact, an exact DFM. Definition 2. Consider the system (1). The mode λ ∈ sp(A) is said to be a mode of magnitude Measure2 (λ ), where Measure2 (λ ) := min{{1/ρ (Mi (λ )) , i = 1 , . . . , ν }, {1/ρ (Mi, j (λ )) , i = 1 , . . . , ν − 1; j = i + 1 , . . . , ν } , {1/ρ (Mi, j,k (λ )) , i = 1 , . . . , ν − 2; j = i + 1 , . . . , ν − 1; k = j + 1 , . . . , ν } , . . . , {1/ρ (M1, 2, ...,ν ) } } , (13) and where ρ (.) represents the minimum singular value. Similar to the previous case, the larger Measure2 (λ ) is, the closer λ is to being a DFM (again, λ is a DFM if and only if Measure2 (λ ) = ∞). 3.3 Distance from Transmission Zeros The third quantitative measure proposed here to evaluate the closeness of a mode to being a DFM is the minimum relative distance of the mode from the transmission zeros of the subsystems given in (4)–(7).

230

Edward J. Davison and Amir G. Aghdam

Definition 3. Given the system (1), let λ ∈ sp(A). Furthermore, let the transmission ˆ li zeros of the subsystem (ci , A, bi) given in (4) be denoted  by λ i , i = 1 , 2, . . . , ν and li = 1 , 2, . . . , nˆ i . Define σi :=

   l   min λ −λˆ ii  , li =1, 2,..., nˆ i max{ |λ |, 1}

if these subsystems have at

i ≥ 1); otherwise, let σ = ∞. Similarly, let the least one transmission zero (i.e., if nˆ    i   0 di j ci given in (5) be , A, bi b j , transmission zeros of the subsystem cj d ji 0 l denoted by λˆ i,i,jj , i = 1, 2, . . . , ν − 1; j = i + 1, i + 2, . . . , ν and li, j = 1 , 2, . . . , nˆ i, j .  ⎤ ⎡  l 

Define σi, j := ⎣

i, j min λ −λˆ i, j  , li, j =1, 2,..., nˆ i, j

max{ |λ |, 1}

⎦ if these subsystems have at least one

transmission zero (i.e., if nˆ i, j ≥ 1); otherwise, let σi, j = ∞. Finally, let the transmission zeros of the system given in (7) be denoted by  ˆ l | , l=1, 2,..., nˆ } min λ − λ {| if this system has at least λˆ l , l = 1 , 2, . . . , n. ˆ Define σ := max{ |λ |, 1} one transmission zero (i.e., nˆ ≥ 1); otherwise, let σ = ∞. Then the mode λ ∈ sp(A) is said to be a mode of magnitude Measure3 (λ ), where

ν¯

Measure3 (λ ) := ⎛ ⎜ ⎝

∑

i=1 , 2,... , ν

σi +

∑

i=1 , 2,... , ν −1 j=i+1, i+2, ..., ν

⎞

(14)

⎟ σi, j + · · · + σ ⎠

and where ν¯ is equal to 2ν − 1, as pointed out in Remark 1. Similar to the previous two cases, the larger Measure3 (λ ) is, the closer λ is to being a DFM, and in particular λ is a DFM if and only if Measure3 (λ ) = ∞. Remark 3. One can apply the three measures introduced in this work to the discretetime equivalent model of the system to find the ADFMs of the discrete-time model. In this case, if an ADFM of large magnitude for a continuous-time system turns out to be an ADFM of small magnitude for the discrete-time equivalent model, a discrete-time decentralized controller will potentially outperform its continuous-time counterpart [AgD08].

4 A Special Case: Centralized Control Systems For the special case of a centralized control system, one can use the expansion technique introduced in [VaD89] to convert the system to a decentralized one. Consider the system (1), and again, for the sake of simplicity, assume that all control stations are SISO and the system is strictly proper so that the system is described by x(t) ˙ = Ax(t) + Bu(t) y(t) = Cx(t),

Characterization and Calculation of ADFMs

231

where B ∈ Rn×ν , C ∈ Rν ×n . The control gain for this centralized control structure is given by ⎡ ⎤ k11 · · · k1ν ⎢ .. ⎥}. K := {K ∈ Rν ×ν |K = ⎣ ... . ⎦ kν 1 · · · kνν By expanding A + BKC in terms of the rows of C and columns of B we will have A + BKC = A + b1k 11 c1 + · · · + b1k1ν cν + · · · + bν k ν 1 c1 + · · · + bν kν ν cν , where ci and bi represent the ith row and ith column of the matrices C and B, respectively (i ∈ {1 , . . . , ν }). Note that ki j (i, j ∈ {1 , . . . , ν }) are all scalars. This implies that (C, A, B) with the centralized information flow K given above is equivalent to ¯ A, B) ¯ where ¯ with diagonal information flow K, (C, ⎡ ⎤ ⎫ c1 ⎪ ⎬ ⎢ .. ⎥ ⎢ . ⎥ ⎢ ⎥ ⎪ ⎢ c1 ⎥ ⎭ ⎢ ⎥   ⎢ ⎥ C¯ = ⎢ ... ⎥ , B¯ = B · · · B ⎢ ⎥ ⎫ ⎢ cν ⎥ ⎪ ⎢ ⎥ ⎬ ⎢ . ⎥ ⎣ .. ⎦ ⎪ ⎭ cν ¯ is the set of all ν 2 × ν 2 diagonal matrices. This means that λ ∈ sp(A) is a and K centralized fixed mode (CFM) of the system (C, A, B) if and only if it is a DFM of ¯ A, B) ¯ Note that the ¯ with respect to the diagonal information flow K. the system (C, ¯ ¯ matrices C and B can be obtained from C and B by using the Kronecker product, as will presently be shown. Consider the system (1) in the general proper form (nonzero matrix D). Then this system can be written in the following form: ν∗

x(t) ˙ = Ax(t) + ∑ bi u∗i (t) y∗i (t) = ci x(t) +

ν∗

∑

j=1

i=1

di j u∗j (t),

i = 1, . . . , ν ∗ ,

where u∗i (t) is a scalar input, y∗i (t) is a scalar output, ν ∗ = ν 2 , and   b1 · · · bν ∗ := α (ν ) ⊗ B ⎤ c1 ⎢ .. ⎥ ⎣ . ⎦ := C ⊗ α (ν ) cν ∗ ⎡

(15)

232

Edward J. Davison and Amir G. Aghdam

⎡

⎤ d11 · · · d1ν ∗ ⎢ .. . . . ⎥  ⎣ . . .. ⎦ := α (ν ) ⊗ D ⊗ α (ν ) , dν ∗ 1 · · · dν ∗ ν ∗   where α (p) = 1 · · · 1 ∈ R p , and ⊗ represents the Kronecker product. The representation (15) is called a scalar expanded f orm of system (1) [LaA08]. The three measures introduced in the previous section can now be used to quantify the ADFMs of system (15); these ADFMs are, in fact, approximate centralized fixed modes of the system (1). Alternatively, one can use the controllability and observability Gramians of a given centralized system as a measure of the degree of controllability and observability of each mode of the system, and these measures can directly be used as a measure of the closeness of a mode to being fixed. This will now be studied in detail. Consider the system (1), and let the matrices B and C be defined as ⎡ ⎤ C1   ⎢ .. ⎥ B := B1 · · · Bν , C := ⎣ . ⎦ . Cν The controllability Gramian is defined as follows [KoY82]:  tf  Wc (0,t f ) := eAτ BB eA τ d τ . 0

It can be shown that this matrix is invertible at any finite time t f , if and only if the pair (C, A) is controllable, and if the system is asymptotically stable, then as t f → ∞ the matrix Wc := Wc (0, ∞) can be computed by solving the following Lyapunov equation: AWc + Wc A + BB = 0. Similarly, the observability Gramian is defined as follows [KoY82]:  tf  Wo (0,t f ) := eA τ CCeAτ d τ 0

and it can be shown that this matrix is invertible at any finite time t f , if and only if the pair (A, B) is observable. If the system is asymptotically stable, then as t f → ∞ the matrix Wo := Wo (0, ∞) can be computed by solving the following Lyapunov equation: AWo + Wo A + CC = 0. For the physical interpretation of a controllability Gramian, assume that it is desired to transfer the state of the system (2) from the origin at time t = −T (x(−T ) = 0) to a given state x0 at t = 0 (x(0) = x0 ), such that the input energy is minimized. In other words, it is desired to solve the following problem:

Characterization and Calculation of ADFMs

0 minimize J =

−T

u (t)u(t)dt,

x(−T ) = 0,

233

x(0) = x0 .

Note that the relationship between the input and the state is given by the state equation (1). It can be shown that the optimal input for the controllable system (1) is given by 

u∗ (t) = B e−A t [Wc (0, T )]−1 x0 and that the corresponding energy (minimum energy) will be equal to J ∗ = x0 [Wc (0, T )]−1 x0 . If the system is asymptotically stable, then as T → ∞ we will have J ∗ = x0Wc−1 x0 .

(16)

Note that if the state xo is an eigenvector of A corresponding to a mode which is close to being uncontrollable, then J ∗ will be “large,” which means that it requires a large amount of energy to reach this state. In other words, such states are “less controllable.” This implies that if Wc is close to being singular, then the pair (A, B) is close to being uncontrollable. Similarly, for a physical interpretation of the observability Gramian, consider the system (1) and assume that u(t) = 0,t ≥ 0. Then the output is CeAt x(0),

t≥0

where x(0) is the initial state. The output energy is equal to T

E= y (t)y(t)dt 0 T  = x (0)eA t CCeAt x(0)dt 0

= x (0)Wo (0, T )x(0) , and if the system is asymptotically stable, then as T → ∞ we will have E = x0Wo x0 .

(17)

Note that if the state xo is an eigenvector of A corresponding to a mode which is close to being unobservable, then the output energy E will be “small.” This means that the effect of this initial state on the output is small. In other words, such states are “less observable.” This implies that if W0 is close to being singular, then the pair (C, A) is close to being unobservable. Note that from the controllability and observability matrices it can only be determined if a system is controllable or observable, but from the controllability and observability Gramians, as discussed above, one can determine “how controllable”

234

Edward J. Davison and Amir G. Aghdam

and “how observable” a system is. Thus, one can use the cost functions (16) and (17) and define the following measure for the closeness of a mode to being uncontrollable and/or unobservable: Measure4 (λ ) = (xλ Wc−1 xλ )/(xλ Wo xλ ),

(18)

where xλ denotes the normalized eigenvector corresponding to the eigenvalue λ of the matrix A in (1). Note that if a mode is close to being a CFM, then the measure given by (18) will be large; thus, this measure can be directly used to quantify an approximate fixed mode of a centralized system.

5 Numerical Examples Example 1. Consider a state-space model with the following parameters: ⎤ ⎤ ⎤ ⎤ ⎡ ⎡ ⎡ ⎡ 0.3915 0 0 100 A = ⎣ 0 2 0 ⎦ , b1 = ⎣ 0 ⎦ , b2 = ⎣ 0.1690 ⎦ , b3 = ⎣ 0 ⎦ 0 0 0.3870 003       c1 = 0 0.3832 0 , c2 = 0.1 × 10−3 0 0.8736 , c3 = 0 0 0.7641 ⎤ 000 D = ⎣0 0 0⎦ 000 ⎡

The digraph of this system is given in Figure 3. From this figure, it can be easily seen that the quotient system has the information flow structure given by the following matrix: ⎤ ⎡ ×× 0 (19) K = ⎣ × × 0 ⎦. 0 0 × It is desired to find⎡the ADFMs with ⎤ of this system ⎤ respect to the two information ⎡ × 0 0 ×× 0 flow matrices K1 = ⎣ 0 × 0 ⎦ and K2 = ⎣ × × 0 ⎦, using Measure3 . From the dis0 0 × 0 0 × cussion above, the ADFMs of this system with respect to K2 are, in fact, the AQFMs of the system.

1

2

3

Fig. 3. The digraph of the system in Example 1.

Characterization and Calculation of ADFMs

235

To determine the ADFMs of the system with respect to K1 using Measure3 , the distance between the transmission zeros of the following systems: (c1 , A, b1 ) , (c2 , A, b2 ) , (c3 , A, b3 ) ,                c1 c1 c2 , A, b1 b2 , A, b1 b3 , A, b2 b3 , , , c2 c3 c3 ⎛⎡ ⎤ ⎞ c1   ⎝⎣ c2 ⎦ , A, b1 b2 b3 ⎠ c3 and each mode of the system (λ = 1, λ = 2, and λ = 3) must be calculated. It can be verified that Measure3 (1) = 7.0 × 10−8, Measure3 (2) = 7.0 × 10−8, Measure3 (3) = 7.0 × 10−8. For the case of the information flow structure given by K2 , one can apply the technique introduced in [VaD89] to obtain a diagonal information flow matrix with scalar diagonal terms. This leads to the following systems for investigating ADFMs: 

(c1 , A, b1 ) , (c2 , A, b2 ) , (c3 , A, b3 ) , (c1 , A, b2 ) , (c2 , A, b1 ) ,               c1 c1 c2 , A, b1 b2 , A, b1 b3 , A, b2 b3 , , , c2 c3 c3







c1 , A, b2 b3 c3





 ,





c2 , A, b1 b3 c3

 

⎛⎡

⎤ ⎞ c1   , ⎝⎣ c2 ⎦ , A, b1 b2 b3 ⎠ . c3

The following results are obtained: Measure3 (1) = 7.0 × 10−8, Measure3 (2) = 7.0 × 10−8, Measure3 (3) = 7.0 × 10−8. These results imply that all three modes are “small” AQFMs and also “small” ADFMs and thus can be easily “shifted” using an LTI decentralized controller. Note that in order to stabilize the system, all three modes need to be shifted to the left-half complex plane in this example. Example 2. Consider a system with the following state-space model: ⎤ ⎡ ⎤ ⎡ ⎡ ⎤ 1 −1 0 −3 0 x(t) ˙ = ⎣ 0 −2 0 ⎦ x(t) + ⎣ 0 ⎦ u1 (t) + ⎣ 1 ⎦ u2 (t) , 1 0 0 −3 1   y1 (t) = 0 1 0 x(t) ,   y2 (t) = 1 α −2 x(t) .

236

Edward J. Davison and Amir G. Aghdam

One can easily verify that the mode λ = −2 is a DFM of this system if and only if α = 0. It is now desired to compare the proposed measures obtained for λ = −2 as an ADFM for different values of α . See Figures 4 through 6.

1000

900

800

700

Measure

1

600

500

400

300

200

100

0

0

0.1

0.2

0.3

0.4

0.5 α

0.6

0.7

0.8

0.9

1

Fig. 4. The magnitude of ADFM λ = −2 for Example 2, using Measure1 .

300

250

Measure

2

200

150

100

50

0

0

0.1

0.2

0.3

0.4

0.5 α

0.6

0.7

0.8

0.9

1

Fig. 5. The magnitude of ADFM λ = −2 for Example 2, using Measure2 .

Characterization and Calculation of ADFMs

237

300

250

Measure

3

200

150

100

50

0

0

0.1

0.2

0.3

0.4

0.5 α

0.6

0.7

0.8

0.9

1

Fig. 6. The magnitude of ADFM λ = −2 for Example 2, using Measure3 .

Example 3. Let the matrices b2 and c2 in Example 2 be replaced by ⎡ ⎤ 0   b2 = ⎣ α ⎦ , c2 = 1 1 −2 . 1 Similar to the previous example, the mode λ = −2 is a DFM of this system if and only if α = 0. However, the main difference between the two examples is that, unlike Example 2, here the mode λ = −2 is, in fact, a centralized fixed mode (CFM) for α = 0 which means it is unobservable and/or uncontrollable. Thus, for small values of α the mode λ = −2 will be close to being a CFM. Since the open-loop system is stable, this means that one can use Measure4 given in (18), which is the most precise quantitative measure in this case, and compare it to Measure1 , Measure2 , and Measure3 after expanding the two-input, two-output centralized system to obtain a four-input, four-output decentralized model as described in [VaD89]. The results obtained for different values of α between 0 and 1 are shown in Figures 7 through 10. Example 4. The state-space model of a drum boiler system is given by the following matrices: ⎤ ⎡ −3.93 −3.15 × 10−3 0 0 0 4.03 × 10−5 0 0 0 3.68 × 102

⎢ 2.74 × 101−2 ⎢ −6.47 × 10 3.85 × 103 A=⎢ ⎢ 2.24 × 104 ⎣ 0 0 −2.20

−3.05 3.03 0 0 7.87 × 10−2 −5.96 × 10−2 0 0 −5 −1 −5.20 × 10 0 −2.55 × 10 −3.35 × 10−6 1.73 × 101 −1.28 × 101 −1.26 × 104 −2.91 1.80 × 101 0 −3.56 × 101 −1.04 × 10−4 0 2.34 × 10−3 0 0 0 0 −1.27 1 × 10−3 −3 −1.77 × 10 0 −8.44 −1.11 × 10−4

−3.77 × 10−3 0 0 0 −2.81 × 10−4 0 0 0 −7 −5 −4 3.60 × 10 6.33 × 10 1.94 × 10 0 −1 1 1 −1.05 × 10 1.27 × 10 4.31 × 10 0 −4.14 × 10−1 9.00 × 101 5.69 × 101 0 2.22 × 10−4 −2.03 × 10−1 0 0 7.86 × 10−5 0 −7.17 × 10−2 0 1.38 × 10−5 1.49 × 10−3 6.02 × 10−3 −1 × 10−10

⎥ ⎥ ⎥, ⎥ ⎦

Edward J. Davison and Amir G. Aghdam 1000

900

800

700

Measure

1

600

500

400

300

200

100

0

0

0.1

0.2

0.3

0.4

0.5 α

0.6

0.7

0.8

0.9

1

Fig. 7. The magnitude of ADFM λ = −2 for Example 3, using Measure1 .

300

250

2

200

Measure

238

150

100

50

0

0

0.1

0.2

0.3

0.4

0.5 α

0.6

0.7

0.8

0.9

1

Fig. 8. The magnitude of ADFM λ = −2 for Example 3, using Measure2 .

Characterization and Calculation of ADFMs 300

250

Measure

3

200

150

100

50

0

0

0.1

0.2

0.3

0.4

0.5 α

0.6

0.7

0.8

0.9

1

Fig. 9. The magnitude of ADFM λ = −2 for Example 3, using Measure3 .

5

2

x 10

1.8

1.6

1.4

Measure

4

1.2

1

0.8

0.6

0.4

0.2

0

0

0.1

0.2

0.3

0.4

0.5 α

0.6

0.7

0.8

0.9

1

Fig. 10. The magnitude of ADFM λ = −2 for Example 3, using Measure4 .

239

240

Edward J. Davison and Amir G. Aghdam

⎤ 0 0 ⎥ ⎢ 0 0 ⎥ ⎢ ⎥ ⎢ 1.56 0 ⎥ ⎢ ⎢ 0 −5.13 × 10−6 ⎥   ⎥ ⎢ ⎥ , C= 000001000 . 8.28 −1.50 B=⎢ ⎥ ⎢ 000000001 ⎥ ⎢ 0 1.78 ⎥ ⎢ ⎥ ⎢ 2.33 0 ⎥ ⎢ ⎣ 0 −2.45 × 10−2 ⎦ 0 2.94 × 10−5 ⎡

Measure3 (which has been observed through several experiments to be more consistent compared to other measures proposed in this work for identifying ADFMs) is now applied to the above system and the results obtained are summarized in Table 1. These results show that the mode λ = −9.13 × 10−3 can be considered as an ADFM with “large” magnitude. Similar results for the ADFMs of the discrete-time equivalent model obtained by using a zero-order hold with a sampling period of T = 1 sec are given in Table 2. These results show that the ADFM pointed out above is also an ADFM of “large” magnitude for the sampled system; i.e., it is also difficult to “shift” this mode by using a discrete-time LTI controller. One can verify that this mode is also an AQFM of the system (ADFM of the corresponding quotient system) which means that Table 1. Measure3 for ADFMs of the boiler system of Example 4. Mode −1.00 × 10−10 −7.84 × 10−3 −9.13 × 10−3 ∗ ∗ ∗ −9.85 × 10−2 −2.37 × 10−1 −3.28 × 10−1 −2.94 −3.64 + j9.27 × 10−1 −3.64 − j9.27 × 10−1

Measure3 1.85 × 102 7.39 × 102 1.62 × 103 ∗ ∗ ∗ 1.40 × 101 1.21 × 101 6.22 2.24 1.37 1.37

Table 2. Measure3 for ADFMs of the boiler system of Example 4 (T = 1 sec). Mode −1.00 × 10−10 −7.84 × 10−3 −9.13 × 10−3 ∗ ∗ ∗ −9.85 × 10−2 −2.37 × 10−1 −3.28 × 10−1 −2.94 −3.64 + j9.27 × 10−1 −3.64 − j9.27 × 10−1

Measure3 1.86 × 10−2 7.45 × 102 1.63 × 103 ∗ ∗ ∗ 1.48 × 101 1.38 × 101 7.63 6.95 × 101 5.77 × 101 5.77 × 101

Characterization and Calculation of ADFMs

241

it cannot be “shifted” easily by using any type of nonlinear and/or time-varying decentralized controller. Example 5. The state-space matrices of a distillation column are as follows: ⎡

0 0 0 0 0 0 0 −1.40 × 10−2 4.30 × 10−3 ⎢ 9.50 × 10−3 −1.38 × 10−2 4.60 × 10−3 0 0 0 0 0 0 ⎢ ⎢ 0 9.50 × 10−3 −1.41 × 10−2 6.30 × 10−3 0 0 0 0 0 ⎢ ⎢ 0 0 9.50 × 10−3 −1.58 × 10−2 1.10 × 10−2 0 0 0 0 ⎢ ⎢ ⎢ 0 0 0 9.50 × 10−3 −3.12 × 10−2 1.50 × 10−2 0 0 0 ⎢ A=⎢ 0 0 0 0 2.02 × 10−2 −3.52 × 10−2 2.20 × 10−2 0 0 ⎢ ⎢ −2 −2 −2 0 0 0 0 0 2.02 × 10 −4.22 × 10 2.80 × 10 0 ⎢ ⎢ −2 −2 0 0 0 0 0 0 2.02 × 10 −4.82 × 10 3.70 × 10−2 ⎢ ⎢ −2 ⎢ 0 0 0 0 0 0 0 2.02 × 10 −5.72 × 10−2 ⎢ ⎣ 0 0 0 0 0 0 0 0 2.02 × 10−2 −2.55 × 10−2 0 0 0 0 0 0 0 0 ⎤ 0 −4 ⎥ 0 5.00 × 10 ⎥ 0 2.00 × 10−4 ⎥ ⎥ ⎥ 0 0 ⎥ ⎥ ⎥ 0 0 ⎥ ⎥, 0 0 ⎥ ⎥ 0 0 ⎥ ⎥ 0 2.00 × 10−4 ⎥ −2 −4 ⎥ ⎥ 4.20 × 10 5.00 × 10 ⎥ −4.83 × 10−2 5.00 × 10−4 ⎦ −2 −2 2.55 × 10 −1.85 × 10

⎤ 0 0 −6 −3 ⎢ 5.00 × 10 2.50 × 10 ⎥ ⎥ ⎢ ⎢ 2.00 × 10−6 5.00 × 10−3 ⎥ ⎥ ⎢ ⎢ 1.00 × 10−6 5.00 × 10−3 ⎥ ⎥ ⎢ ⎢   0 5.00 × 10−3 ⎥ ⎥ ⎢ 00000000010 −3 ⎥ 0 5.00 × 10 . , C = B=⎢ ⎥ ⎢ 10000000000 ⎢ −5.00 × 10−6 5.00 × 10−3 ⎥ ⎥ ⎢ ⎢ −1.00 × 10−5 5.00 × 10−3 ⎥ ⎥ ⎢ ⎢ −4.00 × 10−5 2.50 × 10−3 ⎥ ⎥ ⎢ ⎣ −2.00 × 10−5 2.50 × 10−3 ⎦ 4.60 × 10−4 0 ⎡

As is similar to Example 4, Measure3 is calculated in Tables 3 and 4, respectively, with a sampling period of T = 1, for each mode of this system and the corresponding discrete-time equivalent model. The results given in these tables show that the modes −7.01 × 10−2 and −1.82 × 10−2 can be considered ADFMs of “large” magnitude for both the continuous-time system and the corresponding discrete-time model. In other words, it is difficult to “shift” these modes by using an LTI continuous-time controller, or an LTI discrete-time controller. Example 6. The state-space model of a two-input, two-output thermal system is given below: ⎡ ⎢ ⎢ ⎢ A=⎢ ⎢ ⎣

−2.77 −3.07 2.98 0 0 1.00 0 0 2.68 × 101 −6.15 × 101 5.24 × 10−1 0 0 1.76 × 10−1 0 −9.23 × 10−2 3.00 × 101 −1.55 × 101 −3.22 × 101 0 0 0 0 0 0 0 0 −2.77 × 101 0 0 0 −8.28 × 10−2 0 4.40 × 101 0 −8.98 × 101 −1.00 × 102 0 0 0 0 0 0 0 −3.88 × 103 −1.00 × 102 0 0 0 0 0 0 0 0 −3.33 0 0 −2.23 × 102 0 −4.78 × 101 0 0 5.54 × 101 −3.50 × 10−1 0 0 0 0 0 0 0 1.00

−5.99 × 10−1 −3.21 × 101 −1.56 × 101 −5.32 0 0 0 −2.22 × 102 0

⎤ ⎥ ⎥ ⎥ ⎥, ⎥ ⎦

242

Edward J. Davison and Amir G. Aghdam

⎡

0 ⎢ 0 ⎢ ⎢ 0 ⎢ ⎢ 0 ⎢ 0 B=⎢ ⎢ ⎢ 2.5 × 101 ⎢ ⎢ 0 ⎢ ⎣ 0 0

⎤ 0 0 ⎥ ⎥ 0 ⎥ ⎥   0 ⎥ ⎥ 000000010 0 ⎥ . , C = ⎥ 000000001 0 ⎥ ⎥ 3.33 ⎥ ⎥ 0 ⎦ 0

As is similar to the last two examples, Measure3 is calculated for each mode of this system and the corresponding discrete-time equivalent model with a sampling period of T = 0.01 sec (see Tables 5 and 6, respectively. These results show that this system has no ADFM of “large” magnitude associated with the continuous-time system or the sampled model. In fact, from the magnitudes of Measure3 provided for each mode in the two tables, one can conclude that a continuous-time LTI controller is more suitable for decentralized control of this system. Table 3. Measure3 for ADFMs of the distillation system of Example 5. Mode −9.60 × 10−2 −7.01 × 10−2 ∗ ∗ ∗ −5.05 × 10−2 −3.39 × 10−2 −1.22 × 10−4 −3.24 × 10−3 −7.76 × 10−3 −2.46 × 10−2 −1.99 × 10−2 −1.82 × 10−2 ∗ ∗ ∗ −1.42 × 10−2

Measure3 1.50 × 102 1.08 × 104 ∗ ∗ ∗ 4.98 × 102 2.77 × 102 2.97 × 102 4.38 × 102 8.48 × 102 3.94 × 102 6.94 × 102 2.46 × 103 ∗ ∗ ∗ 3.23 × 102

Table 4. Measure3 for ADFMs of the distillation system of Example 5 (T = 1 sec). Mode −9.60 × 10−2 −7.01 × 10−2 ∗ ∗ ∗ −5.05 × 10−2 −3.39 × 10−2 −1.22 × 10−4 −3.24 × 10−3 −7.76 × 10−3 −2.46 × 10−2 −1.99 × 10−2 −1.82 × 10−2 ∗ ∗ ∗ −1.42 × 10−2

Measure3 1.66 × 102 1.16 × 104 ∗ ∗ ∗ 5.23 × 102 2.86 × 102 2.98 × 102 4.40 × 102 8.54 × 102 4.03 × 102 7.07 × 102 2.50 × 103 ∗ ∗ ∗ 3.28 × 102

Characterization and Calculation of ADFMs

243

Table 5. Measure3 for ADFMs of the thermal system of Example 6. Mode −3.33 −1.01 × 10−1 + j1.05 × 101 −1.01 × 10−1 − j1.05 × 101 −1.35 × 101 −2.96 × 101 −3.39 × 101 + j10.2 × 101 −3.39 × 101 − j10.2 × 101 −1.07 × 102 + j1.59 × 101 −1.07 × 102 − j1.59 × 101

Measure3 3.26 × 10−1 1.12 × 10−1 1.12 × 10−1 2.67 × 10−1 7.79 × 10−1 2.21 × 10−1 2.21 × 10−1 3.35 × 10−2 3.35 × 10−2

Table 6. Measure3 for ADFMs of the sampled system of Example 6 (T = 0.01 sec). Mode −3.33 −1.01 × 10−1 + j1.05 × 101 −1.01 × 10−1 − j1.05 × 101 −1.35 × 101 −2.96 × 101 −3.39 × 101 + j10.2 × 101 −3.39 × 101 − j10.2 × 101 −1.07 × 102 + j1.59 × 101 −1.07 × 102 − j1.59 × 101

Measure3 2.55 × 101 9.26 9.26 1.53 × 101 6.92 × 101 1.75 × 101 1.75 × 101 2.46 × 101 2.46 × 101

6 Conclusion In this chapter, different quantitative measures have been proposed to evaluate the closeness of a mode to being fixed in decentralized control systems. The proposed measures use the transmission zero technique given in [DaC90] for characterizing the decentralized fixed modes (DFMs) of the system. The closeness of a mode to being fixed determines how easily the mode can be “shifted” in the complex plane. A proper measure for characterizing the approximate decentralized fixed modes (ADFMs) is important in the sense that it enables the designer to determine if a desired performance can be achieved by using an LTI decentralized controller. The method is also extended to characterize approximate quotient fixed modes (AQFMs), which are the approximate decentralized fixed modes of the quotient system, corresponding to the strongly connected subsystems. Note that an ADFM which is not an AQFM can be easily “shifted” by employing a proper time-varying decentralized controller. However, it will require a huge amount of input energy using a decentralized controller of any type (nonlinear and/or time varying) to displace a mode which is an AQFM. The proposed measures are applied to a number of numerical and practical examples. The authors have observed that the third measure, which is based on the minimum distances between the mode and the transmission zeros of certain systems, is more consistent in characterizing ADFMs. Furthermore, in all

244

Edward J. Davison and Amir G. Aghdam

three industrial examples (and many others) that have been checked, it has always been the case that an ADFM of the continuous-time system is also an ADFM of the discrete-time equivalent model. This implies that a continuous-time LTI system which is difficult to control by applying a decentralized LTI controller due to the existence of an ADFM of large magnitude will in general also be difficult to control by using a discrete-time decentralized controller or any nonlinear or time-varying decentralized controller.

References [WaD73] S. H. Wang and E. J. Davison, On the stabilization of decentralized control systems, IEEE Transactions Automatic Control, vol. AC-18, no. 5, pp. 473–478, Oct. 1973. [DaC90] E. J. Davison and T. N. Chang, Decentralized stabilization and pole assignment for general proper systems, IEEE Transactions Automatic Control, vol. AC-35, no. 6, pp. 652–664, June 1990. [LaA07] J. Lavaei and A. G. Aghdam, A graph theoretic method to find decentralized fixed modes of LTI systems, to appear in Automatica, Vol. 43, no. 2, pp. 2129–2133, Dec 2007. [SeS81] M. E. Sezer and D. D. Siljak, Structurally fixed modes, Systems and Control Letters, vol. 1, no. 1, pp. 60–64, 1981. [GoA97] Z. Gong and M. Aldeen, Stabilization of decentralized control systems, Journal of Mathematical Systems, Estimation, and Control, vol. 7, no. 1, pp. 1–16, 1997. [VaD89] A. F. Vaz and E. J. Davison, On the quantitative characterization of approximate decentralized fixed modes using transmission zeros, Mathematics of Control Signal, and Systems, vol. 2, pp. 287–302, 1989. [AgD08] A. G. Aghdam and E. J. Davison, Discrete-time control of continuous systems with approximate decentralized fixed modes, Automatica, no. 1, pp. 75–87, Jan 2008. [KoY82] H. Kobayashi and T. Yoshikawa, Graph-theoretic approach to controllability and localizability of decentralized control system, IEEE Transactions Automatic Control, vol. AC-27, no. 5, pp. 1096–1108, May 1982. [Moo81] B. Moore, Principal component analysis in linear systems: controllability, observability, and model reduction, IEEE Transactions Automatic Control, vol. AC-26, no. 1, pp. 17–31, Feb 1981. [LaA08] J. Lavaei and A. G. Aghdam, Control of continuous-time LTI systems by means of structurally constrained controllers, Automatica, vol. 44, no. 1, pp. 141–148, Jan 2008.

Some New Nonlinear and Symbol Manipulation Techniques to Mitigate Adverse Effects of High PAPR in OFDM Wireless Communications Byung Moo Lee1 and Rui J.P. de Figueiredo2 1 2

Infra Laboratory, KT, 17 Woomyeon-dong, Seocho-gu, Seoul, 137-792, South Korea, [email protected] California Institute for Telecommunications and Information Technology, University of California, Irvine, CA 92697-2800, USA, [email protected]

This chapter is presented in honor of Professor Michael K. Sain on the occasion of his seventieth birthday celebration. Summary. Orthogonal frequency division multiplexing (OFDM) modulation has several attributes which favor it for high speed wireless communications. But its high peak-to-average power ratio (PAPR) seriously limits the linear range, and hence the power efficiency, of the transmitter’s high power amplifier (HPA). We present an overview of two complementary approaches to the solution of this problem. Details are provided in our papers listed in the References. The first approach uses an adaptive nonlinear pre-distorter (PD) to compensate for the nonlinearity in the HPA. The analytical expressions used for the HPA and the corresponding PD lead to satisfactory overall system operation up to the saturation region of the HPA, under rapidly time-varying conditions. The second approach reduces the PAPR to an acceptable range by means of several recently proposed PAPR reduction techniques. These techniques include (1) an enhanced version (denoted EIF-PTS) of the Cimini/Sollenberger iterative flipping procedure for implementation of the Muller/Huber partial transmit sequence (PTS) algorithm; (2) a decision-oriented tree-structured modification (denoted T-PTS) of the PTS algorithm, which seeks the best complexity/performance trade-off in the implementation of the resulting simplified PTS algorithm; (3) a combination of clipping and selected mapping techniques for fading channels; and (4) an extension of some of the underlying PAPR reduction concepts to multiple input, multiple output OFDM-based wireless communication systems.

1 Introduction Orthogonal frequency division multiplexing (OFDM) has several desirable attributes, such as high immunity to inter-symbol interference, robustness with respect to multipath fading, and the ability to deliver high data rates, causing OFDM to be incorporated in several wireless standards. However, one of the major problems posed C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 12, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

246

Byung Moo Lee and Rui J.P. de Figueiredo

by OFDM is its high peak-to-average power ratio (PAPR), which seriously limits the power efficiency of the transmitter’s high power amplifier (HPA). This occurs because the PAPR forces the HPA to operate beyond its linear range with a consequent nonlinear distortion in the transmitted signal. This distortion, in turn, causes significant errors in the transmitted signals but also undesirable out-of-band radiation (OBR). The distortion is viewed as a major impediment to progress by the radio frequency (RF) system design community. Clever signal processing techniques are necessary to deal with this problem. In this chapter, we overview some of our recent techniques based on two complementary approaches to the solution of the above problem. In the first approach, we provide a new mixed computational/analytical approach for adaptive pre-compensation of this nonlinear distortion for cases in which the HPA is a traveling wave tube amplifier (TWTA) or a solid state power amplifier (SSPA), and thus we increase the linear dynamic range up to the saturation region of the HPA. However, these pre-distortion techniques only work in the limited range that extends up to the saturation region of the HPA. In order to mitigate this problem, as a second and complementary approach, we develop several novel PAPR reduction techniques which pull down the PAPR of the OFDM signal to an acceptable range. Even though various PAPR reduction techniques have been proposed by many researchers, due to the practical importance of this problem, there is still considerable interest in further developing new and combining existing PAPR reduction techniques to mitigate nonlinear distortion and clipping in the HPA under ever-increasing throughput demands from the system. In what follows, we briefly describe the OFDM signal and PAPR in Section 2. Section 3 discusses the digital pre-distortion approach, a powerful approach used to mitigate the effect of the high PAPR of the OFDM signal. Simple simulation results are also shown in Section 3. We briefly describe our techniques using the alternate approach to PAPR reduction based on symbol manipulation in Section 4. Due to, space limitations, we provide only the basic ideas of our PAPR reduction techniques. Section 5 presents our concluding remarks.

2 OFDM Signal and PAPR An OFDM signal of N subcarriers can be represented as 1 N−1 x(t) = √ ∑ X [k]e j2π fkt , 0 ≤ t ≤ Ts , N k=0

(1)

where Ts is the duration of the OFDM signal and fk = Tks . The high PAPR of the OFDM signal arises from the summation in the above inverse discrete Fourier transform (IDFT) expression. The PAPR of the OFDM signal in the analog domain can be represented as

Some New Nonlinear and Symbol Manipulation Techniques

PAPRc =

max0≤t≤Ts |x(t)|2 E(|x(t)|2 )

.

247

(2)

Nonlinear distortion in the HPA occurs in the analog domain, but most of the signal processing for PAPR reduction is performed in the digital domain. The PAPR in the digital domain is not necessarily the same as the PAPR in the analog domain. However, in some literature [T99] [OL95] [WT05], it is shown that one can closely approximate the PAPR in the analog domain by oversampling the signal in the digital domain. Usually, an oversampling factor L = 4 is sufficient to satisfactorily approximate the PAPR in the analog domain. For these reasons, we express the PAPR of the OFDM signal as follows: PAPR =

max0≤n≤LN |x(n)|2 E(|x(n)|2 )

.

(3)

3 Digital Pre-Distortion Approach As stated earlier, our first approach to mitigate the nonlinear distortion caused by the high PAPR of the OFDM signal is based on passing it through a pre-distorter (PD) prior to its entry into the HPA [Ld06] [Ld05] [dL04], as depicted in Figure 1. First, the OFDM baseband signal goes through the PD to pre-compensate the nonlinear distortion of the HPA. Then it goes into the DAC to convert the digital signal to an analog signal. After that the RF signal passes the HPA and goes through the channel. Except in the case of Brajal and Chouly [BC94], previous PD-based approaches have been based on (1) using a look-up table (LUT) and updating the table via least mean square (LMS) error estimation [HH00] [JCC97]; (2) two-stage estimation, using Wiener-type system modeling for the HPA, and Hammerstein system modeling for the PD [KCY99]; (3) simplified Volterra-based modeling for compensation of the HPA nonlinearity [CPC00] [CP01]; and (4) polynomial approximation of this nonlinearity [GC02]. We note that all these techniques are based on a general approximation form for the nonlinear system, rather than on exploiting specific forms gleaned from physical device considerations. In our approach, on the other hand, we resort to the closedform expressions for the inverse of the HPA characteristic in two important cases: Saleh’s TWTA model [S81] and Rapp’s SSPA model [R91] for the HPA. Specifically, from the expressions for the forward characteristics, we first obtain the closed

Fig. 1. Simplified OFDM transmitter with PD and HPA.

248

Byung Moo Lee and Rui J.P. de Figueiredo

form for their inverses using only a few parameters [Ld06]. This avoids the larger number of parameters that a generic approximation expression (like the polynomial approximation) would require for accurate representation of the characteristic. We have combined the closed-form expression for the inverse of the HPA characteristic with a sequential nonlinear parameter estimation algorithm, which allows sparse implementation of the PD and accurate and rapid tracking of (adaptation to) the timevarying behavior of the HPA. Let us explain this approach for the case in which the HPA is an SSPA. Specifically, we use the normalized Rapp SSPA model, for which we assume that the AM/PM conversion is small enough to be negligible. Then, the AM/AM and AM/PM conversions for the SSPA may be represented by [R91] u[r] =

r [1 + (r/A0)2p ]1/2p

Φ [r] ≈ 0,

(4) (5)

where r denotes the amplitude of the input to the SSPA, u is the amplitude response, Φ is the phase response, A0 is the maximum output amplitude, and p is a parameter which affects the smoothness of the transition. Since we can neglect AM/PM conversion in the SSPA model, we mainly focus on the compensation of the amplitude distortion. In order to compensate the nonlinearity of the HPA with a PD, the output u of the PD/HPA system is targeted to be equal to k · r where, for simplicity, we assume that k = 1. Thus we have q = r, [1 + (q/A0)2p ]1/2p

(6)

where q is the output amplitude of the PD. From the above, we derive the analytical expression for the PD (with system gain normalized to 1 (k = 1)): q[r] =

r , r < A0 . [1 − (r/A0)2p ]1/2p

(7)

The compensation effect of the above analysis is shown in Figure 2. When r ≥ A0 , Figure 2 has no solution. In this case, we clip the input signal as in Figure 2. In the time-varying case, the time-varying behavior of the SSPA is captured by the change in time of the two parameters A0 and p. Then we can track the variation of A0 and p using the LMS algorithm. Simulation results with and without the PD for a typical SSPA OFDM communication system are shown in Figure 3 and Figure 4. In these simulations, we use 16QAM-OFDM with 128 subcarriers and IBO (input backoff) = 6 dB (Figure 3) and 10 dB (Figure 4). We assume an additive white Gaussian noise (AWGN) channel to clearly show the effect of nonlinear distortion and compensation. As we can see from

Some New Nonlinear and Symbol Manipulation Techniques

249

1.5

Normalized Output

PD Compensated SSPA model

1

0.5

0

0

0.5

1

1.5

Normalized Input

Fig. 2. Compensation and clipping effect of SSPA with PD.

1.5

1.5

1

1

0.5

0.5

Q

Q

the two simulation results, using the PD, we can significantly reduce both in-band distortion and OBR. By using the same analysis, pre-distortion for the TWTA shows similar effects to those for pre-distortion for the SSPA case. Additional details for systems that use SSPA and TWTA are provided in [Ld06].

0

0

−0.5

−0.5

−1

−1

−1.5 −1.5

−1

−0.5

0

I

0.5

1

1.5

−1.5 −1.5

−1

−0.5

0

0.5

1

1.5

I

Fig. 3. Received signal constellations of OFDM, without (left) and with (right) PD, IBO = Eb = 20 dB. 6 dB, N 0

250

Byung Moo Lee and Rui J.P. de Figueiredo

0

−10 Without PD

Normailized PSD

−20

−30

−40

−50

−60

−70 −0.5

With PD

0 Normalized Frequency

0.5

Fig. 4. Power spectral density analysis, IBO = 10 dB.

4 PAPR Reduction Approach by Signal Manipulation Even though the digital pre-distortion technique may significantly reduce nonlinear distortion in an OFDM system and thus allow a signal with a high PAPR to go through the HPA, it only works in a limited peak power range, that is, up to the saturation region of the HPA. However, in practical situations, the OFDM signal has a peak power exceeding this range. Thus, even using the digital pre-distortion technique, a high IBO is needed to avoid the significant in-band distortion and OBR that would otherwise occur at the output of the HPA. This high IBO, in turn, reduces the transmission power efficiency. For this reason, as a complementary approach to the OFDM signal with a given PAPR through the HPA using digital pre-distortion, several techniques have been and are being developed to appropriately manipulate the OFDM signal so that the transmitted signal has a lower PAPR than the original one. These techniques may be partitioned into the following nine categories: (1) clipping and filtering technique [LC98] [J02], (2) block coding technique [JWB94] [WJ95] [TJ00] [PT00], (3) partial transmit sequence (PTS) technique [MH97], (4) selected mapping (SLM) technique [BFH96], (5) interleaving technique [JTR00], (6) tone reservation/injection technique [T99], (7) active constellation extension technique [KJ03], (8) companding technique [HLZLG04], and (9) other techniques. In the remaining part of this section, we briefly describe the new PAPR reduction techniques that we have developed.

Some New Nonlinear and Symbol Manipulation Techniques

251

In [LdE08], we propose an enhanced iterative flipping algorithm. The PTS technique, developed by Muller and Huber [MH97], is a very promising technique since it does not cause any signal distortion. The PTS technique divides one signal block into several sub-blocks and each sub-block has its own IDFT. By using these IDFTs, each sub-block is transferred to the time domain. After that, it iteratively searches the optimum phase factor for each sub-block to reduce the PAPR of the OFDM signal. Usually the number of phase factors is limited to a certain number (practically 4). However, the main problem with the PTS technique is its high complexity which makes it difficult for use in practical systems, because iterative searching requires a large computational power. To combat this difficulty, Cimini and Sollenberger developed a low complexity iterative flipping PTS algorithm (denoted by IF-PTS) [CS00]. the IF-PTS algorithm tries to optimize the PAPR for each sub-block, not for all subblocks, thus it is a kind of greedy algorithm. Even though the IF-PTS algorithm is simple, there is still a significant performance gap between IF-PTS and the ordinary PTS algorithm. For these reasons, we have developed an enhanced version of the iterative flipping algorithm which we call an enhanced iterative flipping algorithm. In this algorithm, there is a new adjustable parameter which is chosen according to a performance/complexity trade-off. The main idea of this algorithm is that we make more possibilities and delay the final decision at the end of the sub-block. In [LdL06], we present a new decision-oriented, tree-structure-based modification of the PTS technique, denoted henceforth by T(Tree)-PTS. T-PTS has two adjustable parameters, T and S, to reduce PAPR. These can be adjusted to allow one to choose a trade-off between complexity and performance depending on circumstances. If these parameters are increased to their highest possible values, T-PTS becomes the same as the ordinary PTS technique. If they are reduced to their lowest values, T-PTS reduces to Cimini and Sollenberger’s IF-PTS. This means that T-PTS is a generalization of PTS techniques and one can choose almost any intermediate level of performance/complexity between that of PTS and IF-PTS. In [LdD08], design criteria for clipping with adaptive symbol selection for PAPR reduction of the OFDM signal are introduced. To reduce the PAPR of the OFDM signal with minimum distortion and/or complexity, clever signal processing is necessary. This can be done more easily if we combine two PAPR reduction techniques. As a first PAPR reduction technique, we choose the clipping technique since it is one of the most widely used and simplest PAPR reduction techniques. As a second one, we choose an adaptive symbol selection technique, like the SLM technique. A combination of the clipping and SLM techniques was originally proposed in [OH00] for an AWGN channel. In [OH00], the authors used a Nyquist rate sampled signal, and showed a bit error rate performance of the OFDM-QPSK signal in the AWGN channel for high clipping ratios. [LdD08] is different from [OH00] with respect to several points. First, we note that oversampling is important for the PAPR reduction technique. Most of the signal processing for PAPR reduction is performed in the digital domain. However, nonlinear distortion occurs in the analog domain. Without oversampling, the PAPR of the digital signal can underestimate the PAPR of the analog signal. For this reason, we perform 4-times oversampling by using trigonometric interpolation. Second, we place an ideal PD and HPA at the end of the transmitter to

252

Byung Moo Lee and Rui J.P. de Figueiredo

measure spectral leakage. For this purpose, we use a soft envelope limiter as a combination of an ideal PD and HPA. Third, we simulate and analyze the PAPR reduction technique under fading channel conditions. Since OFDM is widely used in wireless channels, we believe that the fading channel analysis is more valuable than AWGN channel analysis. And last, we propose design criteria which depend on simulation results and analysis, rather than just a simple combination of both techniques. We also provide performance of clipping with SLM technique in fading channels based on the analysis of [OH00] and [LdP08]. In [Ld07], we study PAPR reduction techniques for multiple input, multiple output (MIMO)-OFDM based on the V-BLAST (Vertical Bell Laboratories Layered Space-Time) system [S98] [TSC98] [TJC99] [J05] [Te99] [F96] [FG98] [WFGV98]. As we mentioned above, among various PAPR reduction techniques, we chose SLM. The SLM technique has been receiving much attention since it does not give rise to any signal distortion. One important issue with the use of the SLM technique is the transmission of side information (SI). Incorrect detection of SI causes burst error. For this reason, SI needs to be carefully protected. Protecting SI by turbo code [OH00] or using a diversity technique [LYJPS03] has been proposed by other researchers. These techniques require redundancy and/or computational complexity. We propose an SI power allocation technique for MIMO-OFDM systems. In our approach, we allocate more power to the SI to protect it from the hostile wireless channel, with slight power loss of other subcarriers. This technique is quite simple and effective, and it does not need any redundancy. Moreover, we get more of a benefit if we use more transmit antennas and/or reserve more symbols as SI. Simulation results show that the proposed technique shows a performance close to the case of perfect side information at the receiver.

5 Conclusions In this chapter, we presented two approaches for mitigating PAPR in OFDM systems. One approach is based on adaptive pre-distortion of the base-band signal being transmitted so as to compensate the nonlinear distortion that would otherwise occur in the HPA at high peak power values. This approach is valid up to the saturation region of the HPA. For higher values of the allowable signal power, as a complementary approach, we have presented PAPR reduction techniques. Various novel PAPR reduction techniques have been only briefly introduced due to space limitations. More interested readers should refer to the cited papers.

References [T99] [OL95]

Tellado, J.: Peak to average power reduction for multicarrier modulation, Ph.D dissertation, Stanford University, Palo Altc, CA (1999) O’Neil, R., Lopes, L.N.: Envelope variations and spectral splatter in clipped multicarrier signals, in Proc. of PIMRC’95, 71–75, Sept (1995)

Some New Nonlinear and Symbol Manipulation Techniques [WT05]

[Ld06]

[Ld05]

[dL04]

[BC94]

[HH00] [JCC97]

[KCY99]

[CPC00]

[CP01]

[GC02]

[S81]

[R91]

[LC98] [J02]

[JWB94]

[WJ95]

253

Wang, L., Tellambura, C.: A simplified clipping and filtering technique for PAR reduction in OFDM systems, IEEE Signal Processing Letters, vol. 12, no. 6, 453–456, June (2005) Lee, B.M., de Figueiredo, R.J.P.: Adaptive pre-distorters for linearization of high power amplifiers in OFDM wireless communications, Circuits, Systems & Signal Processing, Birkh¨auser Boston, vol. 25, no. 1, 55–80, (2006) Lee, B.M., de Figueiredo, R.J.P.: A tunable pre-distorter for linearization of solid state power amplifier in mobile wireless OFDM, IEEE 7th Emerging Technologies Workshop, 84–87, St. Petersburg, Russia, June 23–24, (2005) de Figueiredo, R.J.P., Lee, B.M.: A new pre-distortion approach to TWTA compensation for wireless OFDM systems, 2nd. IEEE International Conference on Circuits and Systems for Communications, ICCSC-2004 (Invited Plenary Lecture), Moscow, Russia, no. 130, June 30–July 2, (2004) Brajal, A., Chouly, A.: Compensation of nonlinear distortions for orthogonal multicarrier schemes using predistortion, GLOBECOM 1994, San Francisco, CA, vol. 3, 1909–1914, Nov. (1994) Han D., Hwang, T.: An adaptive pre-distorter for the compensation of HPA nonlinearity, IEEE Transactions on Broadcasting, vol. 46, 152–157, June (2000) Jeon, W.G., Chang, K.H., Cho, Y.S.: An adaptive data predistorter for compensation of nonlinear distortion in OFDM systems, IEEE Transactions on Communications, vol. 45, no 10, 1167–1171, Oct. (1997) Kang, H.K., Cho, Y.S., Youn, D.H.: On compensation nonlinear distortions of an OFDM system using an efficient adaptive predistorter, IEEE Transactions on Communications, vol. 47, no. 4, 522–526, April (1999) Chang, S., Powers, E.J., Chung, J.: A compensation scheme for nonlinear distortion in OFDM systems, Global Telecommunications Conference, IEEE GLOBECOM 2000, vol. 2, 736–740, Dec. (2000) Chang, S., Powers, E.J.: A simplified predistorter for compensation of nonlinear distortion in OFDM systems, Global Telecommunications Conference, IEEE GLOBECOM 2001, vol. 5, 3080–3084, Nov. (2001) Guo, Y., Cavallaro, J.R.: Enhanced power efficiency of mobile OFDM radio using predistortion and post-compensation, IEEE 56th Vehicular Technology Conference, Proceedings. VTC 2002-Fall. 2002, vol. 1, 214–218, Sept. (2002) Saleh, A.A.M.: Frequency-independent and frequency-dependent nonlinear models of TWT amplifiers, IEEE Transactions on Communications, vol. 29, no. 11, 1715–1720, Nov. (1981) Rapp, C.: Effect of HPA-nonlinearity on 4-DPSK/OFDM-signal for a digital sound broadcasting system, in proceedings of the second European conference on satellite communications, Liege, Belgium, Oct. 22–24, 179–184, (1991) Li,X., Cimini, L.J.: Effect of Clipping and Filtering on the performance of OFDM, IEEE Communication Letters, vol. 2 no. 5, 131-133, May (1998) Armstrong, J.: Peak-to-average power reduction for OFDM by repeated clipping and frequency domain filtering, Electron. Lett., vol. 38, 246–247, Feb. (2002) Jones, A.E., Wilkinson, T.A., Barton, S.K.: Block coding scheme for reduction of peak to mean envelope power ratio of multicarrier transmission scheme, Electron. Lett., vol. 30, 2098–2099, Dec. (1994) Wilkinson,T.A., Jones, A.E.: Minimization of the peak to mean envelope power ratio in multicarrier transmission schemes by block coding, in Proc. IEEE Vehicular Technology Conf., Chicago, IL, 825–831, July (1995)

254 [TJ00]

Byung Moo Lee and Rui J.P. de Figueiredo

Tarokh, V., Jafarkhani, H.: On the computation and reduction of the peak-toaverage power ratio in multicarrier communications, IEEE Trans. Commun., vol. 48, 37–44, Jan. (2000) [PT00] Paterson, K.G., Tarokh, V.: On the existence and construction of good codes with low peak-to-average power ratios, IEEE Trans. Info. Theory, vol. 46, no. 6, 1974–1987, Sept. (2000) [MH97] Muller, S.H., Huber, H.B.: OFDM with reduced peak-to-mean power ratio by optimum combination of partial transmit sequences, Electron. Lett., vol. 33, 368–369, Feb. (1997) [BFH96] Bauml, R.W., Fischer, R.F.H., Huber, J.B.: Reducing the peak-to-average power ratio of multicarrier modulation by selected mapping, Electron. Lett., vol. 32, 2056–2057, Oct. (1996) [JTR00] Jayalath, A.D.S., Tellambura, C.: Reducing the peak-to-average power ratio of orthogonal frequency division multiplexing signal through bit or symbol interleaving, Elect. Lett., vol. 36, no. 13, 1161–1163, June (2000) [KJ03] Krongold, B.S., Jones, D.L.: PAR reduction in OFDM via active constellation extension, IEEE Trans. Broadcast., vol. 49, no. 3, 258–268, Sept. (2003) [HLZLG04] Huang, X., Lu, J., Zheng J., Letaief, K.B., Gu, J.: Companding transform for reduction in peak-to-average power ratio of OFDM signals, IEEE Trans. Wireless Comuunications, vol. 03, No. 6, 2030–2038, Nov. (2004) [LdE08] Lee B.M., de Figueiredo, R.J.P.: An enhanced iterative flipping algorithm for PAPR reduction of OFDM signals, to be submitted. [CS00] Cimini Jr., L.J., Sollenberger, N.R.: Peak-to-average power ratio reduction of an OFDM signal using partial transmit sequences, IEEE Communication Letters, vol. 4, 86–88, Mar. (2000) [LdL06] Lee, B.M., de Figueiredo, R.J.P.: A low complexity tree algorithm for PTSbased PAPR reduction in wireless OFDM, 2006 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2006 Proceedings, vol. 4, 301–304, 14-19 May (2006) [LdD08] Lee B.M., de Figueiredo, R.J.P.: Design of the clipping with adaptive symbol selection for PAPR reduction of OFDM signal in flat and frequency selective fading channels, to be submitted. [OH00] Ochiai, H., Imai, H.: Performance of the deliberate clipping with adaptive symbol selection for strictly band-limited OFDM systems, IEEE J. Select. Areas Commun., vol. 18, 2270–2277, Nov. (2000) [LdP08] Lee B.M., de Figueiredo, R.J.P.: Performance analysis of the clipping with adaptive symbol selection for PAPR reduction of OFDM signal in flat and frequency selective fading channels, to be submitted. [Ld07] Lee B.M., de Figueiredo, R.J.P.: Side information power allocation for MIMOOFDM PAPR reduction by selected mapping, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2007, vol. 3, 361– 364, 15–20 April (2007) [S98] Alamouti, S.M.: A simple transmit diversity technique for wireless communications, IEEE J. Sel. Areas Commun., vol. 16, no. 8, 1451–1458, Oct. (1998) [TSC98] Tarokh, V., Seshadri, N., Calderbank, A.R.: Space-time codes for high data rate wireless communications: Performance criterion and code construction, IEEE Trans. Inf. Theory, vol. 44, 744–765, Mar. (1998) [TJC99] Tarokh, V., Jafarkhani, H., Calderbank, A.R.: Space-time block codes from orthogonal design, IEEE Trans. Inf. Theory, vol. 45, no. 5, 1456–1567, Jul. (1999)

Some New Nonlinear and Symbol Manipulation Techniques [J05] [Te99] [F96]

[FG98]

[WFGV98]

[LYJPS03]

255

Jafarkhani, H.: Space-time coding: Theory and practice, Cambridge University Press, London (2005) Telatar, E.: Capacity of multi-antenna Gaussian channels, European Trans. Telecomm., vol. 10, no. 6, 585–596, Nov. (1999) Foschini, G.J.: Layered space-time architecture for wireless communication in a fading environment when using multi-element antennas, Bell Labs Technical J., vol. 1, no. 2, 41–59, (1996) Foschini, G.J., Gans, M.J.: On limits of wireless communications in a fading environment when using multiple antennas, Wireless Personal Commun., vol. 6, 311–335, (1998) Wolniasky, P.W., Foschini, G.J., Golden, G.D., Valenzuela, R.: V-BLAST: An architecture for realizing very high data rates over the rich scattering wireless channel, in Proc. ISSSE 98, Pisa, Italy, 295–300, (1998) Lee, Y., You, Y., Jeon, W., Paik J., Song, H.: Peak-to-average power ratio in MIMO-OFDM systems using slective mapping, IEEE Comm. Lett., vol. 7, no. 12, 575–577, Dec. (2003)

Homogeneous Domination and the Decentralized Control Problem for Nonlinear System Stabilization Jason Polendo,1 Chunjiang Qian,2 and Cheryl B. Schrader3 1 2 3

Southwest Research Institute,‡ 6220 Culebra Rd., San Antonio, TX 78229, USA [email protected] Dept. of Electrical & Computer Engineering, University of Texas at San Antonio,§ One UTSA Circle, San Antonio, TX 78249, USA [email protected] Boise State University, 1910 University Dr., Boise, ID 83725, USA [email protected]

Summary. This chapter gives an overview of constructive techniques for nonlinear dynamic system stabilization in the face of uncertainties and limited information from sensors. Such techniques have ultimately allowed for a loosening of the restrictions on the classes of nonlinear systems that could be systematically stabilized. The schemes discussed herein have the distinct commonality of dominating and thereby cancelling the undesirable effects of additive perturbations. Output feedback stabilization of highly nonlinear systems is discussed first, with the specific decentralized control problem setting then considered in addition to the output feedback issue. The schemes outlined in this chapter can be found in their entirety in several papers by the authors, with specific publications cited where applicable.

1 Introduction Nonlinear control theory has seen major advances in the past few decades, via descriptive differential geometric methods [Is95] and the advent of recursive design procedures [KKK95], which allow for unprecedented control over a wide variety of nonlinear phenomena. However, to say that this is a complete theory would be an obvious oversight, especially with the vast research effort currently evidenced in control theory research journals and conference proceedings. Nevertheless, as the theory has matured it has become apparent that in designing a controller for a given nonlinear system model, such as (1) below, one must incorporate a combination of tools in the development process, depending upon the design objective. ‡ Jason

Polendo’s research was supported in part by the NASA-Harriett G. Jenkins Predoctoral Fellowship Program. § Chunjiang Qian’s research was supported in part by the NSF under grant ECS-0239105. C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 13, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

258

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

x˙1 = x2p1 + f1 (t, x, u) x˙2 = x3p2 + f2 (t, x, u) .. . x˙n = u + fn (t, x, u) y = x1 .

(1)

This is a stark contrast to linear control theory, where a plethora of tools such as the root locus, Bode plots, Linear Quadratic optimal control, eigenstructure assignment, H∞ , μ -synthesis, linear matrix inequalities, etc. are available to achieve the desired closed-loop system performance. Each of these methods can be used to achieve stabilization, regulation and tracking, and disturbance attenuation among similar design objectives. A particular realm of interest for nonlinear control theory is when the state information available (i.e. feedback) is limited to the output of the system, as y = x1 . This type of problem is predominantly more difficult than its linear counterpart since the heralded separation principle, where the state estimates are deduced independently of the controller u and then utilized as the true states, does not generally hold. Such results have been slower in realization than in full-state feedback methods, but there has been some promising work accomplished recently [Qi05] that gives credence to the idea of pursuing these topics. Additional complexities are seen when working with nonlinear large-scale systems, as in the nonlinear decentralized control problem setting, where again only the output of each subsystem may be available for measurements. Therefore, the subsystems require stabilization in the face of the highly nonlinear and uncertain interconnections, which is a particularly arduous task considering these interconnecting perturbations depend nonlinearly on unmeasurable states. Initial developments in the control of nonlinear dynamic systems focused on describing the dynamics and the system properties, such as stability and uncertainty concepts, rather than designing systems that contain these properties. Note that we are not concerned with stability analysis per se, such as in [Kh92], but on the systematic schemes that answer the “How to” concerning nonlinear system stabilization. Therefore rather than study stability analysis, we focus our attention on the synthesis problem. The current state of the output feedback stabilization research for nonlinear systems is reviewed in Section 3.1. Section 3.2 reviews the initial incarnation of the homogeneous domination approach originally published in [Qi05] for a linear system perturbed by a nonlinear vector field. Section 3.3 then details a systematic method for the stabilization of a class of inherently nonlinear systems (where a nonlinear chain of integrators, rather than a linear chain, is assumed) with higher-order nonlinear perturbation functions, itself published originally in [Po06,PQ07b]. This design scheme generalizes and extends current methods for power integrator output feedback stabilization. Section 4 then considers the complex problem setting of large-scale systems and decentralized control by output feedback where the unmeasured states are allowed to couple in a polynomial fashion between subsystems. First, however, the

Nonlinear System Decentralized Output Feedback Control

259

next section introduces some mathematical concepts which are integral to the subsequent technical discussion.

2 Mathematical Preliminaries In this section, we collect some useful definitions and lemmas that play very important roles in proving the main results of this chapter. 2.1 Homogeneous Systems The innovative idea of homogeneity was introduced for the stability analysis of a nonlinear system [Ha67] and has led to a number of interesting results (see [He91, BR01]). It also opened the door for a solution to the difficult problem of controllability and controller design for nonlinear systems [Ka89, DMK90, Ka90, He91, Day92, BR01]. We recall the definitions of homogeneous systems with weighted dilation. The interested reader can refer to [Ka90, He91, BR01] for more details. Weighted Homogeneity: For fixed coordinates (x1 , . . . , xn ) ∈ Rn and real numbers ri > 0, i = 1, . . . , n, – the dilation Δε (x) is defined by

Δε (x) = (ε r1 x1 , . . . , ε rn xn ),

∀ε > 0,

with ri being called the weights of the coordinates (for simplicity of notation, we define dilation weight Δ = (r1 , . . . , rn )). – a function V ∈ C(Rn , R) is said to be homogeneous of degree τ if there is a real number τ ∈ R such that ∀x ∈ Rn \ {0}, ε > 0, V (Δε (x)) = ε τ V (x1 , · · · , xn ). – a vector field f ∈ C(Rn , Rn ) is said to be homogeneous of degree τ if there is a real number τ ∈ R such that ∀x ∈ Rn \ {0}, ε > 0, fi (Δε (x)) = ε τ +ri fi (x1 , · · · , xn ),

i = 1, . . . , n.

– a homogeneous p-norm is defined as xΔ ,p =

n

∑ |xi |

1/p p/ri

,

∀x ∈ Rn ,

i=1

for a constant p ≥ 1. For simplicity, in this chapter, we choose p = 2 and write xΔ for xΔ ,2.

260

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

The introduction of homogeneity has given us a very powerful tool for the stability analysis of a nonlinear system. Extensive research has been done on homogeneous systems in the past two decades and a number of interesting results have been achieved. For instance, it has been proven that local stability is equivalent to global stability for a homogeneous system. Another important result is that any homogeneous system that is asymptotically stable admits a homogeneous Lyapunov function. Interested readers can refer to book [BR01] and the references therein for more results on homogeneous systems. In what follows, we list some useful properties of homogeneous functions which will be frequently used throughout the chapter. Lemma 1. Given a dilation weight Δ = (r1 , · · · , rn ), suppose V1 (x) and V2 (x) are homogenous functions of degree τ1 and τ2 , respectively. Then V1 (x)V2 (x) is also homogeneous with respect to the same dilation weight Δ . Moreover, the new homogeneous degree of V1 (x)V2 (x) is τ1 + τ2 . Lemma 1 can be easily proven using the homogeneity definition and therefore its proof will be omitted here. The next lemma shows that a positive definite homogeneous function shares some properties analogous to those of a quadratic Lyapunov function used in linear control theory. Lemma 2. Suppose V : Rn → R is a homogeneous function of degree τ with respect to the dilation weight Δ . Then the following holds: 1)

∂V is still homogeneous of degree τ − ri with ri being the homogeneous weight ∂ xi of xi .

2) There is a constant c such that V (x) ≤ cxΔτ .

(2)

Moreover, if V (x) is positive definite, cxΔτ ≤ V (x),

(3)

for a positive constant c > 0. The proof of Lemma 2 is quite straightforward and can be found in most of the literature on homogeneous systems theory such as the book [BR01]. Remark 1. Apparently, when the dilation weight ri = 1 and τ = 2, the homogeneous function V (x) reduces to the quadratic function xT Px. In that case, properties 1) and 2) described in Lemma 2 reduces to the following well-known properties: a)

∂ xT Px = xT P, ∂x

and b) c1 x2 ≤ xT Px ≤ c2 x2 .

Nonlinear System Decentralized Output Feedback Control

261

Similar to the roles of properties a) and b) in linear system theory, the generalized properties 1) and 2) are also extremely important for the design procedure in this chapter. 2.2 Useful Inequality This next lemma is a modified version of Young’s inequality. A detailed proof can be found in [QL01], among other places. Lemma 3. Let c, d be positive constants. Given any positive number γ > 0, the following inequality holds: |x|c |y|d ≤

c d − c c+d γ |x|c+d + γ d |y| . c+d c+d

(4)

3 Stabilization of Nonlinear Systems by Output Feedback 3.1 Problem Background A formidable problem in the nonlinear control literature is the global stabilization of a nonlinear dynamic system by output feedback. Such a problem formulation is inherently practical in that only partial sensing is necessary to feed back state information, an efficient and cost-effective solution in many applications and a necessity in others. Unfortunately, the majority of the existing output feedback stabilization schemes have been very limited in what types of nonlinearities could be handled, an issue made more complex due to the lack of a true “separation principle” for nonlinear systems. In this work, we investigate more generic systems than those previously studied, such as (1), restated below, x˙1 = x2p1 + φ1 (t, x, u) x˙2 = x3p2 + φ2 (t, x, u) .. . x˙n = u + φn(t, x, u) y = x1 ,

(5)

where x = (x1 , . . . , xn )T ∈ Rn , u ∈ R, and y ∈ R are the system state, input, and output, respectively. When pi = 1, the results on global output feedback stabilization of system (1) are based on quite restrictive conditions imposed on the nonlinear terms φi (·), mainly attributed to finite escape time phenomena [MPD94]. These stabilization results include systems where the nonlinear function is only dependent on the output [BZ83, KI83, KR85, KX02, MT91], i.e.,

φi (·) = φi (y),

262

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

or is Lipschitz or linear in the unmeasurable states [Be98, GHO92, KS87, Ts91], i.e., |φi (·)| ≤ c (|x1 | + |x2 | + · · · + |xi |) , with a constant growth rate c. Later in [PJ03,CH04], the linear growth condition was extended to the case when the growth rate is a polynomial of the output, i.e., |φi (·)| ≤ c(y) (|x1 | + |x2| + · · · + |xi |) . Until recently, there was no systematic way of dealing with systems whose dynamics are highly nonlinear in the unmeasured states. In the work of [Qi05], however, the nonlinearity restriction was relaxed to globally stabilize more general systems such as (1) by output feedback under a much less restrictive polynomial growth condition. This result has made it possible to achieve global output feedback stabilization of systems such as [Qi05], x˙1 = x2 x˙2 = x3 x˙3 = x4 + d(t)x3 n(1 + x23) x˙4 = u + d(t)x32 + x22 sin x4 y = x1 , with a bounded disturbance d(t). To accomplish this in general, the following assumption was made. Assumption 1 [Qi05] Given τ ≥ 0, for (1) with pi = 1, i = 1, . . . , n,   iτ +1 iτ +1 |φi (t, x, u)| ≤ c |x1 |iτ +1 + |x2 | 2τ +1 + · · · + |xi | (i−1)τ +1 ,

(6)

where c > 0 is a constant. However, when pi are of higher order, that is, the system has uncontrollable/unobservable linearization, the global output feedback stabilization solutions are few. Current output feedback stabilization results for these higher-order systems, on the other hand, are contained in the works [QL04] for different pi and [YL04], which necessitates that the pi are all the same odd integer. In [QL04], the nonlinear function, φi (·), is allowed a slow growth rate in lower-triangular form, stated formally as follows. Assumption 2 [QL04] For (1) with pi ≥ 1 odd integers, i = 1, . . . , n,   1 1 1 p1 ···pi−1 p2 ···pi−1 pi−1 |φi (t, x, u)| ≤ c |x1 | + |x2 | + · · · + |xi−1| + |xi | , where c > 0 is a constant.

Nonlinear System Decentralized Output Feedback Control

263

While in [YL04], φi (·) is restricted to be Lipschitz-like, again in lower-triangular form, and can be described by the following assumption. Assumption 3 [YL04] For (1) there is a constant odd integer p ≥ 1 such that for pi = p, i = 1, . . . , n, |φi (t, x, u)| ≤ c (|x1 | p + |x2| p + · · · + |xi | p ) , with a constant c > 0. Note that the methods in [QL04] and [YL04] are quite different from each other and their results do not overlap, not even when considering the exact same system such as x˙1 = x32 , x˙2 = u. An interestingly unresolved problem is whether or not we can find a generalized framework to unify these two solutions, simultaneously covering both Assumptions 2 and 3, while also containing Assumption 1 when pi = 1. To handle this issue, we employ the concept of homogeneous domination [Qi05] to cover a larger class of inherently nonlinear systems. In doing so, we allow pi to be any odd real number such that pi ≥ 1 and the pi ’s are not necessarily equivalent. This formalism will allow for more complex nonlinearities than those seen in [Qi05,QL04,YL04], and in fact, generalizes the homogeneous domination approach introduced in [Qi05]. This design methodology will allow for the stabilization of systems such as x˙1 = x32 x˙2 = u + d(t)xq2 y = x1 ,

(7)

with a bounded disturbance d(t). For q = 1, the system (7) was stabilized via output feedback by the techniques described in [QL04], and for q = 3, output feedback stabilization of (7) was dealt with in [QL02b]. However, when q = 2, the global stabilization of (7) has been an open problem. Nevertheless, we will show that (7) can now be controlled via output feedback for, although not limited to, q = 2 by methods described in Section 3.3. 3.2 The Homogeneous Domination Approach for Global Output Feedback Stabilization of Nonlinear Systems In this section, we briefly review an output feedback stabilization result presented in [Qi05]. Based on homogeneous systems theory, the result provides a systematic design tool for the construction of dynamic compensators, and is essential in solving our decentralized control problem. Consider the linear system z˙ j = z j+1 , j = 1, . . . , n − 1, z˙n = v, y = z1 ,

(8)

where v is the input and y is the output. For system (8), one can easily design a linear observer plus a linear feedback controller to globally stabilize the system.

264

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

This method has been extended to nonlinear systems with a linear growth condition on the nonlinear vector field [QL02a]. However, the linear nature of this type of design makes it inapplicable to inherently nonlinear systems. For instance, when the nonlinear vector field has higher-order growth terms, a linear dynamic output feedback controller fails to globally stabilize the system. For the output feedback design of inherently nonlinear systems, a genuinely nonlinear observer design method is needed. The recently developed nonlinear homogeneous observer in [Qi05] provides such a tool for handling inherently nonlinear systems. According to [Qi05], one can construct a homogeneous observer for system (8) as follows:

η˙ 2 = fn+1 (z1 , η2 ) = −1 zˆ2 , with zˆ2 = (η2 + 1 z1 )r2 /r1 η˙ 3 = fn+2 (z1 , η2 , η3 ) = −2zˆ3 , with zˆ3 = (η3 + 2 zˆ2 )r3 /r2 .. .

(9)

η˙ n = f2n−1 (z1 , η2 , . . . , ηn ) = −n−1zˆn , with zˆn = (ηn + n−1zˆn−1 )rn /rn−1 , where r j = ( j − 1)τ + 1, j = 1, . . . , n are the homogeneous dilations and the constants  j > 0, j = 1, . . . , n − 1 are observer gains. The controller can be constructed as (rn +τ )/rn

v = − ξn with

βn

ξ1 = zˆ1 − z∗1 , z∗1 = 0, rk /rk−1 z∗k = −ξk−1 βk−1 , ξk = zˆk − z∗k ,

(10) zˆ1 = z1 k = 2, . . . , n

for appropriate controller constants β j > 0, j = 1, . . . , n. Denote Z = (z1 , z2 , . . . , zn , η2 , . . . , ηn )T F(Z) = (z2 , . . . , zn , v, fn+1 , . . . , f2n−1 )T .

(11) (12)

The closed-loop system (8)–(10) can be written in a compact form Z˙ = F(Z). Moreover, it can be verified that F(Z) is homogeneous of degree τ with dilation

Δ = (1, τ + 1, . . . , (n − 1)τ + 1, 1, τ + 1, . . . , (n − 2)τ + 1).

(13)

Lemma 4. [Qi05] The observer gains  j > 0, j = 1, . . . , n − 1 and controller gains β j > 0, j = 1, . . . , n can be recursively determined such that the closed-loop system (8)–(10) admits a Lyapunov function V (Z) with the following properties: 1) V is positive definitive and proper with respect to Z; 2) V is homogeneous of degree 2M − τ with dilation (13) for a constant M ≥ rn ; 3) the derivative of V (Z) along (8)–(10) satisfies

4 where ZΔ =

∂V F(Z) ≤ −CZ2M V˙ (Z(t)) = Δ , ∂Z 2

rj ∑2n−1 i=1 |z j | and C > 0 is a constant.

Nonlinear System Decentralized Output Feedback Control

265

Proof. A detailed proof of Lemma 4 can be found in [Qi05] and hence is omitted here. 2 Remark 2. If one sets τ = 0, it is easy to see r j = 1 for all 1 ≤ i ≤ n. In this case, (9)–(10) reduce to a linear dynamic output feedback controller; and Lemma 4 is simply linear Lyapunov stability theory where V (·) is a quadratic function. Remark 3. In the case when τ is any nonnegative real number, we are still able to design a homogeneous controller globally stabilizing the system (1) with the necessary modification to preserve the sign of function [·]r j /ri . Specifically, for any real number r j /ri > 0, we define [·]r j /ri = sign(·)| · |r j /ri . Obviously, the function [·]r j /ri defined above is C1 . Using this function, we are able to design the controller without requiring r j /ri to be odd. In this case, the controller can be constructed as u = −sign(ξn )|ξn |(rn +τ )/rn βn with rk

x∗k = −sign(ξk−1 )|ξk−1 | rn βk−1 , ξk = xk − x∗k , k = 1, . . . , n,

and

where x∗1 = 0. 3.3 A Generalized Homogeneous Domination Approach for Output Feedback Stabilization of Nonlinear Systems with Unstable/Uncontrollable Linearization In this section, we succinctly review a generalization of the method discussed in the previous section. This gives a generalized framework [Po06, PQ07b] which not only extends the class of nonlinear systems systematically stabilizable by output feedback, but also contains the previously mentioned results [Qi05, QL04, YL04] as its special cases. See [Po06, PQ07b] for the details. In the following subsection, we show that under Assumption 4, the problem of global output feedback stabilization for system (1) is solvable. We will first construct a homogeneous output feedback controller for the nominal nonlinear chain of power integrators: z˙1 = z2p1 , z˙2 = z3p2 , . . . , z˙n = v, y = z1 .

(14)

Following [Po06, PQ07b] one can design a homogeneous observer for the inherently nonlinear system (14) as

η˙ 2 = fn+1 = −1 zˆ2p1 , η˙ k = fn+k−1 =

p −k−1 zˆk k−1 ,

zˆ2p1

= [η2 + 1z1 ]

p zˆk k−1

r2 p1 r1

,

= [ηk + k−1 zˆk−1 ]

rk pk−1 rk−1

(15) ,

for k = 3, . . . , n, where r1 = 1,

ri pi−1 = τ + ri−1 ,

(16)

266

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

i = 2, . . . , n and li > 0, i = 1, . . . , n − 1 are the observer gains. τ is the degree of homogeneity and is assumed even for this formulation. The controller is then constructed as (r +τ )/ μ v∗ (z) = −βnξn n , (17) where z∗1

= 0,

∗pk−1

zk

rk pk−1

= −ξk−1μ

μ r

μ r

μ r

μ r

ξ1 = z11 − z∗1 1 , βk−1 , ξk = zk k − z∗k k ,

(18)

for k = 2, . . . , n with constants β1 , . . . , βn > 0, and μ = max{ri pi−1 }2≤i≤n+1. Let Z = (z1 , . . . , zn , η2 , . . . , ηn )T Z˙ = F(Z ) = (z2 , . . . , zn , v(z1 , η2 , . . . , ηn ), fn+1 , . . . , f2n−1 )T .

(19) (20)

By choosing the dilation weight

Δ= =



(r1 , r2 , . . . , r2n−1 )

τ + 1 (p1 + 1)τ + 1 (p1 · · · pn−2 + · · · + 1)τ + 1 1, , ,..., , p1 p1 p2 p1 · · · pn−1 67 8 5 for z1 , . . . , zn  τ + 1 (p1 + 1)τ + 1 (p1 · · · pn−3 + · · · + 1)τ + 1 1, , ,..., p1 p1 p2 p1 · · · pn−2 5 67 8 for η2 , . . . , ηn

(21)

it can be shown that (20) is homogeneous of degree τ . Lemma 5. [Po06,PQ07b] The observer gains  j > 0, j = 1, . . . , n − 1 and controller gains β j > 0, j = 1, . . . , n can be recursively determined such that the closed-loop system (14), (15), (17), with pi ∈ R≥1 odd := {q ∈ R : q ≥ 1, q is a ratio of odd integers}, admits a Lyapunov function V (Z) with the following properties: 1) V is positive definitive and proper with respect to Z; 2) V is homogeneous of degree 2 μ − τ with dilation (21); 3) the derivative of V (Z) along (14), (15), (17) satisfies

∂V 2μ F(Z) ≤ −CZΔ , V˙ (Z(t)) = ∂Z

4 2 rj where ZΔ = ∑2n−1 i=1 |z j | and C > 0 is a constant.

Proof. The detailed proof of Lemma 5 can be found in [Po06, PQ07b] and hence is omitted here. 2 Remark 4. In the case when τ is any nonnegative real number, we are still able to design a homogeneous controller globally stabilizing the system (1) with necessary

Nonlinear System Decentralized Output Feedback Control

267

modification to preserve the sign of function [·]ri pi−1 /μ , where μ is defined as before. Specifically, for any real number ri pi−1 /μ > 0, we define [·]ri pi−1 /μ = sign(·)| · |ri pi−1 /μ .

(22)

Using this function, we are able to design the controller without requiring ri pi−1 /μ to be odd. In this case, the controller can be constructed as u = −sign(ξn )|ξn |(rn +τ )/μ βn with

∗pk−1

xk

= −sign(ξk−1 )|ξk−1 |

ξk

= xk − xk ,

μ rk

μ ∗ rk

rk pk−1 μ

βk−1 , and

k = 1, . . . , n,

where x∗1 = 0. It can then be seen that this new controller u is also C1 when rn + τ ≥ μ or when rμ ≥ − rn +rτ −μ , for all k = 1, . . . , n. k

k

3.3.1 Global Output Feedback Stabilization for System (1) Utilization of the homogeneous controller and observer established in the preceding sections will enable us, in the following theorem [Po06,PQ07b], to globally stabilize the nonlinear system (1) with only its output fed back. Assumption 4 There is a constant τ ≥ 0 such that for i = 1, . . . , n,   ri +τ ri +τ ri +τ r1 r2 ri |φi (t, x, u)| ≤ c |x1 | , + |x2 | + · · · + |xi |

(23)

for a constant c > 0 with ri defined in (16). Theorem 1. [Po06, PQ07b] Under Assumption 4, the inherently nonlinear system (1) can be globally stabilized by output feedback. Proof. The detailed proof of Theorem 1 can be found in [Po06, PQ07b] and hence is omitted here. 2 Note that in establishing the previous theorem and proof [Po06,PQ07b], a change of coordinates was carried out with the introduction of a scaling gain L in addition to those established in Lemma 5. System (7) can now be stabilized by output feedback with q = 2, as detailed in the following example. Example 1. By Theorem 1, the inherently nonlinear system x˙1 = x32 x˙2 = u + d(t)x22 y = x1 ,

|d(t)| ≤ 1,

(24)

268

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

is globally stabilizable. Under Assumption 4, φ1 is trivial and φ2 = x22 , therefore p1 = 3, τ = 1/2, r1 = 1, r2 = 1/2, and μ = 3/2. By the form of τ the controller structure of Remark 4 is used and the output feedback controller is

η˙ 2 = −L1 sign(η2 + 1y)|η2 + 1y|3/2 , u = −L4/3 β2 ((η2 + 1y)3 + β1sign(y)|y|3/2 )4/3 , where β1 , β2 , 1 , and L are positive constants. In selecting all the gain assignments, we begin with the state feedback controller gains, β1 , β2 , and step through the proof of the state feedback controller for the nom3/2 3 ∗3 inal system, x˙1 = x32 , x˙2 = u. Denoting ξ1 = x1 , x∗3 2 = − β 1 ξ1 , ξ2 = x 2 − x 2 , u = 4/3 β2 ξ2 as in (17), we begin with the Lyapunov function as in [Po06, PQ07b],  x1   3 s 2 − 0 ds, V1 = 0

with time derivative 3 ∗3 2 V˙1 = ξ1 x˙1 = ξ1 x32 = ξ1 x∗3 2 + ξ1 (x2 − x2 ) = −β1 ξ1 + ξ1 ξ2 ,

and by choosing β1 = 2, the dimension of the system, we have V˙1 = −2ξ12 + ξ1 ξ2 ≤ − 74 ξ12 + ξ22 . By letting  x2 ! s3 − x∗3 W2 (x1 , x2 ) = 2 ds, x∗2

and V2 (x1 , x2 ) = V1 + W2 it can be seen that 4 ∂ W2 7 x˙1 + ξ23 u V˙2 ≤ − ξ12 + ξ22 + 4 ∂ x1 5 1 2 4 4 8 16 7 2 ≤ − ξ1 + ξ22 + 22/3ξ23 ξ13 + 22/3 ξ23 ξ13 + ξ23 u 4 3 3 4 11 2 ≤ − ξ1 + 207ξ22 + ξ23 u, 12

where the last inequality is given by Young’s inequality. And by choosing β2 = 208 we now have 11 V˙2 ≤ − ξ12 − ξ22 . 12 In selecting the observer gain, 1 , we define the Lyapunov function  x3 U2 =

2

(η2 +1 x1 )3/2



 s2/3 − (η2 + 1x1 ) ds

Nonlinear System Decentralized Output Feedback Control

269

and denote zˆ2 = (η2 + 1 x1 )1/2 . Next, by letting e2 = x32 − zˆ32 , it can be seen that the time derivative of this Lyapunov function is U˙ 2 = 3ux22 (x22 − zˆ22 ) − 1e22     2 2 2 2 2 2 4 2 2 − 21 3 3 3 3 3 3 3 2 3 2 e2 − 1e22 . ≤ 3 β2 e2 + 2β2ξ2 + (2 β2 + β2 β1 )ξ1 ξ2 + 2 ξ1 Examining just the first term resulting from distributing the above multiplicative relation, and by invoking Young’s inequality (p = 32 , q = 3),

3(2

− 21

2 )β2 e2 ξ2 ≤ 3 4 3

2 3

3(2

− 21

  1 32 8 3 1 )β2 e22 + ξ22 . 3 8

This yields an observer gain, 1 , in excess of 10, 000 to guarantee that the time derivative of T = V2 + U2 is negative definite, which will only increase upon evaluating the other terms. Obviously, for most applications such a value is undesirable and/or impossible to implement. However, if the technique in this section is used as a guide for choosing the gain values, then it can be seen that the values should satisfy β1 < β2 < 1 , with their exact assignments left as design parameters dependent upon the system constraints. More specifically, due to the global nature of this methodology, extreme caution is employed in proving the technique for the most general of settings, whereas specific implementations allow freedoms in assigning the gain values. 3.3.2 Discussion In this section we point out the universality of this methodology [Po06, PQ07b] by noting that the existing global output feedback stabilization results are special cases of this general framework. We also discuss a similar design scheme for the global output feedback stabilization of an unprecedented class of inherently nonlinear systems, namely systems that do not satisfy a triangularity condition, which entails a class of inherently nonlinear systems not previously globally stabilizable by output feedback. The following corollaries demonstrate the generalized framework of the methodology described in the preceding sections. When τ = 0 Assumption 4 reduces to the bound described in Assumption 2 [QL04], where r1 = 1, r2 = 1/p1, . . . , rn = 1/(p1 p2 · · · pn−1 ); and when pi = p, p ≥ 1 an odd integer, by selecting τ = p − 1, it is apparent that the Lipschitz-like growth condition covered by Assumption 3 [YL04] is contained in Assumption 4. Also, in the case when pi = 1, Assumption 4 reduces to Assumption 1 [Qi05], and (1) becomes a linear chain of integrators perturbed by a nonlinear vector field, which is the system covered in the main result of [Qi05]. Corollary 1. [QL04] Under Assumption 4, with τ = 0, there is an output feedback controller of the form (15)–(17) which will achieve global asymptotic stabilization of the system (1).

270

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

Corollary 2. [YL04] When pi = p, p ≥ 1 an odd integer, under Assumption 4, with τ = p − 1, there is an output feedback controller of the form (15)–(17) which globally asymptotically stabilizes (1). Corollary 3. [Qi05] When pi = 1, under Assumption 4, there is an output feedback controller of the form (15)–(17) which globally asymptotically stabilizes (1). The previous statements clearly show that the existing global output feedback stabilization results for inherently nonlinear systems are contained within this generalized framework [Po06,PQ07b]. In the following, the usefulness of this generalized methodology is illustrated by solving the global output feedback stabilization of a higher-order uncertain nonlinear system which cannot be stabilized using the existing schemes of [Qi05], [QL04], and [YL04], particularly not by a smooth feedback controller. However, by the technique developed here a C1 controller can be guaranteed for global output feedback stabilization of this system. Example 2. Consider the uncertain inherently nonlinear system 5/3

x˙1 = x2

x˙2 = x33 + d(t)ax22 x˙3 = u y = x1 ,

|d(t)| ≤ 1,

(25)

where a is a constant. It can easily be seen that the global output feedback stabilization of this system was previously impossible to guarantee due to the uncertain higher-order term added to a nonlinear chain of integrators with different powers. However, it is straightforward to see that Assumption 4 can be satisfied by choosing τ = 3/2. Therefore, by Theorem 1 there is an output feedback controller globally stabilizing system (25). To illustrate the novelty of our design scheme, we will first consider the case when a = 0 to design our output feedback controller and then show that by simply modifying the scaling gain, L, we can easily compensate for the case when a = 1. In constructing this output feedback controller we must first change the coordinates as z1 = x1 , z2 = x2 /L3/5 , z3 = x3 /L8/15 , v = u/L23/15, giving 5/3

z˙1 = Lz2

z˙2 = Lz33 + L3/5 d(t)az22 , z˙3 = Lv.

(26)

And by following the form of the homogeneous controller and observer as in our design scheme, 3/5 η˙ 2 = −L1 sign (η2 + 1 y) |η2 + 1 y|5/2 = −L1 zˆ2 2 ! η˙ 3 = −L2 sign η3 + 2(η2 + 1y)3 η3 + 2(η2 + 1y)3  = −L2 zˆ33   v = −β3 sign zˆ53 + β2 (sign(ˆz2 )|ˆz2 |10/3 + β1 z51 )

(27) (28)

Nonlinear System Decentralized Output Feedback Control

 1/2   ×zˆ53 + β2 (sign(ˆz2 )|ˆz2 |10/3 + β1z51 ) ,

271

(29)

it can be shown that (26) is globally asymptotically stable for any L > 0. Therefore, by this generalized framework, there is a large enough gain L such that output feedback controller (27)–(29) renders the system (26), and hence (25), globally asymptotically stable. For the case when a is nonzero, in this case equal to one, we can compensate for this perturbing nonlinearity with the exact same output feedback controller structure as before, as well as the same gain values, with the exception of the scaling gain L. In this case we choose L = 2. Remark 5. To state what the previous example showed more formally, note that by the design methodology, once the specific gains (βi and i ) are selected for the nominal nonlinear system (14), only the scaling gain L needs to be adjusted to accommodate various nonlinear terms, φi (·) under Assumption 4, exemplifying the universality of this approach in stabilizing uncertain nonlinear systems.

4 Decentralized Output Feedback Control of Interconnected Systems with High-Order Nonlinearities A challenging problem in the nonlinear control literature is the global stabilization of a nonlinear dynamic system by output feedback. This issue is compounded when considering large-scale systems, which increase the number of states to be estimated plus complicate the problem with all the interconnections between unmeasurable states. Nevertheless, partial sensing is a necessity when dealing with such a large amount of state variables. Unfortunately, the existing decentralized output feedback stabilization schemes have been very limited in the types of nonlinearities that can be handled. Without a true “separation principle” for nonlinear systems, this is indeed an interesting problem. Investigations into nonlinear large-scale systems originated decades ago, with a notable one [Dav74] focusing on time-varying stabilization. High-gain state feedback was used in [KS82] to stabilize large-scale system nonlinearities, while [Io86] used an adaptive scheme for a similar goal. Output feedback was utilized for linear systems in this large-scale setting in the works [BK84, Li84, SK85], among others. Adaptive and output feedback schemes for nonlinear large-scale systems were utilized in [JK97] and [Ji00]. In the nonlinear output feedback setting, however, most of the results for largescale systems consider only those interconnected by their outputs, with limited results for those that consider the interconnection of unmeasurable states. The work [Da89] did consider nonlinear functions that could depend on unbounded unmeasurable states, however, the result is a local one due to the use of a quadratic method. A recent result, [FQC05], did yield a global output feedback controller for a largescale nonlinear system interconnected in the unmeasurable states. To do so, though,

272

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

the authors required the nonlinear functions to be bounded by a linear growth rate, which implies a required symmetry in the interconnecting nonlinearities. 4.1 Linear Integrator Systems Perturbed by Higher-Order Nonlinearities Herein, we focus our attention on large-scale interconnected systems such as ⎧ x˙i1 = xi2 + φi1 (x, d(t)), ⎪ ⎪ ⎪ ⎪ x ⎪ ⎨ ˙i2 = xi3 + φi2 (x, d(t)), Subsystem i : ... 1≤i≤m ⎪ ⎪ ⎪ ⎪ x˙in = ui + φin (x, d(t)), ⎪ ⎩ yi = xi1 ,

(30)

where xi = (xi1 , . . . , xin )T , i = 1, . . . , m, x = (x1 , · · · , xm )T , ui and yi are the system and subsystem state, input and output, respectively. For i = 1, . . . , m, j = 1, . . . , n, φi j (x, d(t)) is an uncertain nonlinear function of all the states with bounded disturbance, d(t). In this section, we expand upon the current literature for global output feedback stabilization of large-scale nonlinear systems by allowing the nonlinear functions to depend on unmeasurable states while relaxing the linear growth restriction to a polynomial one. To handle this issue, we employ the concept of homogeneous domination to cover a larger class of large-scale nonlinear systems. In doing so, we bound φi j (·) by a high-order growth rate. This formalism will allow for more complex nonlinearities than those seen previously by utilizing the homogeneous domination approach introduced in [Qi05, PQ07b] and discussed in Section 3.2. This design scheme will allow for the stabilization of interconnected systems such as ⎧ ⎧ x˙21 = x22 x˙11 = x12 ⎪ ⎪ ⎪ ⎪ ⎨ ⎨ x˙22 = x23 x˙12 = x13 . (31) x ˙ = u + x + sin x x˙ = u2 + x521x212 ⎪ 1 12 23 ⎪ ⎪ ⎪ ⎩ 13 ⎩ 23 y1 = x11 y2 = x21 Notably, the unmeasurable state, x12 , is of higher order in φ23 (·). Moreover, the nonlinearities in φ13 (·) and φ23 (·) have different orders. Hence, system (31) was not previously stabilizable by output feedback. In this section, we will use the homogeneous domination approach [Qi05, PQ07b] to solve the global decentralized control via output feedback for system (31). The use of homogeneity will also allow the nonlinear functions of each subsystem to be vastly different than the others, thus not requiring any unnecessary symmetry in the interconnections.

Nonlinear System Decentralized Output Feedback Control

273

4.1.1 Global Output Feedback Stabilization for Large-Scale System (30) Utilization of the homogeneous controller and observer established in the preceding sections for each subsystem in (30) enable us to construct a decentralized stabilizer via output feedback for (30) under the following hypothetical assumption. Assumption 5 For i = 1, . . . , m, j = 1, . . . , n, there are constants τi ≥ 0 and c > 0 such that  ri j +τi ri j +τi ri j +τi |φi j (x, d(t))| ≤ c |x11 | r11 + · · · + |x1 j | r1 j + · · · + |xm1 | rm1 ri j +τi rm j

+ · · · +|xm j |

 (32)

,

where the constants ri j are defined as ri j + τi = ri, j+1 ,

ri1 = 1, and

τi − τl <

1 , n(n − 1)

for 1 ≤ l, i ≤ m.

(33)

(34)

For simplicity, we assume the degree of homogeneity, τi = qdii , with qi an even integer and di an odd integer for each subsystem. Under this assumption, the homogeneous weights, ri j , will always be a ratio of odd numbers. Note that an equivalent result will be achieved for the case when the ri j are not odd. Theorem 2. Under Assumption 5, the large-scale interconnected nonlinear system (30) can be globally stabilized by output feedback. Proof. For each subsystem, from i = 1, . . . , m, introducing new coordinates, as zi j = xi j /L j−1 , 1 ≤ j ≤ n, v = u/(Ln−1 )

(35)

with L > 1 being the scaling gain determined later, the ith subsystem in (30) can be rewritten as z˙i, j = Lzi, j+1 + φi, j (·)/L j−1 , z˙in = Lv + φin (·)/L

n−1

.

j = 1, · · · , n − 1 (36)

Next, we construct an observer/controller with the scaling gain L, following the design scheme introduced in [Qi05, PQ07b]. The observer for each subsystem is constructed as

274

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader ri2

η˙ i2 = −Li1 zˆi2 ,

zˆi2 = (ηi2 + i1 z1 ) ri1 ,

η˙ i3 = −Li2 zˆi3 , .. .

zˆi3 = (ηi3 + i2 zˆi2 ) ri2 , .. .

ri3

(37) rin

η˙ in = −Li,n−1 zˆin , zˆin = (ηin + i,n−1zˆi,n−1 ) ri,n−1 , and denoting fi,n+l = −il zˆi(l+1) , l = 1, . . . , n − 1, with each subsystem’s controller as (r +τi )/rin

vi (ˆzi ) = −βin ξin in

rin rin

(ˆzi )



rin ri,n−1

rin ri2

rin ri1

rinr +τi

= −βin zˆin + βi,n−1 zˆi,n−1 + · · · + βi2(ˆzi2 + βi1zi1 ) · · ·

in

, (38)

where zˆi1 = zi1 and the gain constants βi j and i j will be selected by applying Lemma 4. For each subsystem, using the following notation similar to (11) and (12): Zi = (zi1 , zi2 , . . . , zi,n , ηi2 , . . . , ηin )T Fi (Zi ) = (zi2 , . . . , zin , vi , fi(n+1) , . . . , fi(2n−1) )T , the closed-loop subsystem (36)–(38) can be written as

φi2 (·) φin (·) Z˙i = LFi (Zi ) + (φi1 (·), , . . . , n−1 , 0, · · · , 0)T . L L

(39)

By Lemma 4, we can design the gains i j and βi j such that Z˙i = Fi (Zi ) is globally asymptotically stable. Moreover, there is a Lyapunov function for each subsystem Vi (Zi ) with M = max{rin | i = 1, . . . , m}, such that

∂ Vi (Zi ) Fi (Zi ) ≤ −ci Zi 2M Δi , ∂ Zi 4 2 rk and ci > 0 is a constant. It can then be seen that for where Zi Δi = ∑2n−1 k=1 |zik | each ith subsystem

∂ Vi (Zi ) ∂ Vi (Zi )  φi2 (·) φin (·) T , . . . , n−1 , 0¯ Fi (Zi ) + φi1 (·), V˙i = L ∂ Zi ∂ Zi L L  T ∂ V (Z ) φ (·) φ i i in (·) i2 , . . . , n−1 , 0¯ , ≤ −Lci Zi 2M φi1 (·), Δi + ∂ Zi L L

(40)

where 0¯ = 0, · · · , 0. Under the change of coordinates (35), we deduce from Assumption 5 that

Nonlinear System Decentralized Output Feedback Control

  m  φi j (x, d(t))   =c∑  L j−1 

ri j +τi rlk

j

|Lk−1 zlk | ∑ L j−1 l=1 k=1

275

.

The power of the scaling gain L can be estimated as (k − 1)

Denote νi j,kl = k≥2

ri j + τi jτi + 1 −j − ( j − 1) = 1 + (k − 1) rlk (k − 1)τl + 1 j − j(k − 1)(τi − τl ) − (k − 1) = 1− . (k − 1)τl + 1

j− j(k−1)(τi −τl )−(k−1) . (k−1)τl +1

νi j,kl >

By the condition (34) (i.e. τi − τl <

1 − (k − 1) j − j(k − 1) n(n−1)

(k − 1)τl + 1

≥

1 1 − j( j − 1) n(n−1)

(k − 1)τl + 1

1 n(n−1) ),

for

≥ 0.

On the other hand, when k = 1, νi j,kl = j > 0. In summary, it can be concluded that νi j,kl > 0. Hence, ν = min{νi j,kl } > 0. By the nomenclature of L > 1, we obtain    φi j (x, d(t))  1−ν    L j−1  ≤ cL

m

j

∑ ∑ |zlk |

ri j +τi rlk

.

(41)

l=1 k=1

Recall that for j = 1, . . . , n − 1, ∂ Vi /∂ Zi j is homogeneous of degree 2M − τi − ri j . Then, by Lemma 1   ri j +τi  ∂ Vi  m j   ∑ ∑ |zlk | rlk (42)  ∂ Zi j  l=1 k=1

is homogeneous of degree 2M. With (41) and (42) in mind, by Lemma 2, we can find a constant ρi j such that

∂ Vi φi j (·) ≤ ρi j L1−ν Z 2M Δ , ∂ Zi j L j−1

(43)

where Z = (Z1 , Z2 , . . . , Zm )T and Δ = (Δ1 , Δ2 , . . . , Δm ) . Substituting (43) into (40) yields n

1−ν V˙i ≤ −Lci Zi 2M Z 2M Δ i + ∑ ρi j L Δ . j=1

Let the Lyapunov function for the large-scale system be V = V1 + V2 + · · · + Vm . Then it can be seen that the time derivative for the entire system is

(44)

276

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader m

1−ν Z 2M V˙ = ∑ V˙i ≤ −LCZ 2M Δ + ρL Δ

(45)

i=1

for positive constants C and ρ that do not depend on L. Apparently, when L is large enough the right-hand side of (45) is negative definite. Consequently, the closed-loop system is globally asymptotically stable. 2 Example 3. Consider an interconnected system 5/7

x˙11 = x12 , x˙12 = u1 + d(t)x12 + x22 , y1 = x11 2/5 x˙21 = x22 , x˙22 = u2 + d(t)x21 x22 , y2 = x21

(46)

where |d(t)| ≤ 1. Clearly, φ11 and φ21 are trivial. Moreover, it can be shown that 5/7

φ12 = d(t)x12 + x22 ≤ |x12 | + |x22|5/7 . Using Lemma 3, one has 2/5

φ22 = d(t)x12 x22 ≤ |x12 |9/5 + |x22 |9/7 . So we can choose τ1 = 0, r11 = 1, r12 = 1 and τ2 = 2/5, r21 = 1, r22 = 7/5 to satisfy Assumption 5. By Theorem 2 the decentralized output feedback controller is of the form

η˙ 12 = −L11 (η12 + 11y1 ) u1 = −Lβ12 (η12 + 11y1 + β11 y1 ) for the first subsystem and

η˙ 22 = −L21 (η22 + 21y2 )7/5   7/5 9/7 u2 = −Lβ22 (η22 + 21y2 )7/5 + β21y2 for the second, where βi j , i j , and L are appropriate positive constants, i = 1, 2, j = 1, 2. One advantage of using the homogeneous domination approach is that only the information of the nominal linear system is needed for the controller design, where the nonlinear terms can be very complicated and even unknown. In what follows, we show that this advantage can be further extended to solve the global output feedback stabilization problem of a more general class of interconnected nonlinear systems. Specifically, Theorem 2 can be extended under the following general assumption. Assumption 6 For i = 1, . . . , m and j = 1, . . . , n, there are constants τi ≥ 0 such that the following relation holds ∀L ∈ [1, ∞) and ∀(t, Z ) ∈ R × Rn·m :

Nonlinear System Decentralized Output Feedback Control

   φi j (Γ Z1 , . . . , Γ Zm , d(t))    ≤ cL1−ν   L j−1

m

r + τi

∑ Zk Δikj

,

277

(47)

k=1

where constant ν > 0, the dilation ri, j are defined  as in (33), and Γ is a diagonal matrix with the diagonal elements 1, L, . . . , Ln−1 . Theorem 3. Under Assumption 6, the problem of global output feedback stabilization of system (30) can be solved by a homogeneous output feedback controller of the form (37)–(38). It is obvious that Assumption 6 includes Assumption 5 as a specific case. Moreover, the condition (34) is no longer necessary for the homogeneous degree τi . Hence, Theorem 3 can be used to stabilize a more general class of nonlinear systems such as the motivating example (31). Example 4. For system (31), by choosing τ1 = 0 (r11 = r1,2 = r1,3 = 1) and τ2 = 2 (r21 = 1, r22 = 3, r2,3 = 5), it can be verified that Assumption 6 holds for system (31). As a matter of fact, we know that    φ13 (Lz12 , L2 z23 )  |Lz12 | + | sin(L2 z23 )|  = ≤ L−1 |z12 | + L−8/5|z23 |1/5   L2 L2 = L1−2 |z12 |(r13 +τ1 )/r12 + L1−13/5|z23 |(r13 +τ1 )/r23 .

(48)

On the other hand, by Lemma 3,    φ23 (Lz12 , z21 )    ≤ |z12 |7 + |z21|7   L2 = L1−1 |z12 |(r23 +τ2 )/r12 + L1−1 |z21 |(r23 +τ2 )/r21 .

(49)

With the help of (48) and (49), we know that Assumption 6 holds with ν = 1. Therefore, by Theorem 3, there is an output feedback controller globally stabilizing the system (31). 4.2 Power Integrator Systems The works of [Po06, PQ07b] extended and generalized the work [Qi05], which considered global output feedback stabilization of linear systems perturbed by a nonlinear vector field to the inherently nonlinear system setting. In turn, the work presented here could also be extended to the problem of decentralized output feedback control of inherently nonlinear large-scale systems of the form ⎧ pi1 + φi1 (x, d(t)), ⎪ ⎪ x˙i1 = xi2 ⎪ pi2 ⎪ x ˙ = x + φi2 (x, d(t)), ⎪ i2 i3 ⎨ .. i = 1, . . . , m . ⎪ ⎪ ⎪ x˙ = ui + φin (x, d(t)), ⎪ ⎪ ⎩ in yi = xi1 ,

278

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

where pi j is an odd positive integer. Herein we make the following assumption. Assumption 7 such that

For i = 1, . . . , m, j = 1, . . . , n, there are constants τi ≥ 0 and c > 0

 ri j +τi ri j +τi ri j +τi |φi j (x, d(t))| ≤ c |x11 | r11 + · · · + |x1 j | r1 j + · · · + |xm1 | rm1 + · · · +|xm j |

ri j +τi rm j

 ,

(50)

where the constants ri j are defined as ri1 = 1,

ri j + τi = ri, j+1 pi j .

(51)

Theorem 4. Under Assumption 7, the inherently nonlinear system (1) can be globally stabilized by output feedback. Proof. The proof is similar to that of Theorem 2 with the use of Lemma 5 and is omitted here. 2 4.3 Conclusion This section detailed methods for stabilizing large-scale nonlinear systems by output feedback, whose unmeasurable interconnected states contribute in a high-order fashion to the dynamics of each subsystem. Such a problem formulation has not been previously considered in a global setting as was done here. Nevertheless, precise knowledge of these nonlinearities is not necessary for stabilization, as this method only requires the knowledge of a nonlinear bounding function. Additionally, it is apparent that this stabilization scheme allows each subsystem’s nonlinearities to be quite distinct from each other, as exemplified in (31).

References [BR01]

[BK84] [Be98] [BZ83] [CH04]

A. Bacciotti and L. Roiser. Liapunov Functions and Stability in Control Theory, volume 267 of Lecture Notes in Control and Information Sciences. Springer, New York, 2001. W. Bachmann and D. Konik. On stabilization of decentralized dynamic output feedback systems. Systems Control Lett., 5(2):89–95, 1984. G. Besancon. State affine systems and obsever based control. NOLCOS, 2:399– 404, 1998. D. Bestle and M. Zeitz. Canonical form observer design for non-linear timevariable systems. Internat. J. Control, 38(2):419–431, August 1983. Z. Chen and J. Huang. Global output feedback stabilization for uncertain nonlinear systems with output dependent incremental rate. In Proceedings of 2004 American Control Conference, 2004.

Nonlinear System Decentralized Output Feedback Control [Da89] [Dav74] [DMK90]

[Day92] [FQC05]

[GHO92]

[Ha67]

[He91]

[Io86] [Is95] [JK97] [Ji00] [Ka89] [Ka90] [Kh92] [KS82]

[KS87]

[KI83] [KR85] [KX02] [KKK95]

279

Z. Da-Zhong. Decentralized output feedback stabilization for a class of nonlinear interconnected systems. IEEE Trans. Automat. Control, 34(12):1297–1300, 1989. E. Davison. The decentralized stabilization and control of a class of unknown non-linear time-varying systems. Automatica—J. IFAC, 10:309–316, 1974. W. P. Dayawansa, C. F. Martin, and G. Knowles. Asymptotic stabilization of a class of smooth two-dimensional systems. SIAM J. Control Optim., 28(6):1321–1349, 1990. W. P. Dayawansa. Recent advances in the stabilization problem for low dimensional systems. In Proceedings of 1992 IFAC NOLCOS, pages 1–8, 1992. M. Frye, C. Qian, and R. Colgren. Decentralized control of large-scale uncertain nonlinear systems by linear output feedback. Communications in Information and Systems, 4(3):191–210, 2005. J.-P. Gauthier, H. Hammouri, and S. Othman. A simple observer for nonlinear systems applications to bioreactors. IEEE Trans. Automat. Control, 37(6):875– 880, 1992. W. Hahn. Stability of Motion. Translated from the German manuscript by Arne P. Baartz. Die Grundlehren der mathematischen Wissenschaften, Band 138. SpringerVerlag, New York, 1967. H. Hermes. Homogeneous coordinates and continuous asymptotically stabilizing feedback controls. In Differential Equations (Colorado Springs, CO, 1989), volume 127 of Lecture Notes in Pure and Appl. Math., pages 249–260. Dekker, New York, 1991. P. A. Ioannou. Decentralized adaptive control of interconnected systems. IEEE Trans. Automat. Control, 31(4):291–298, 1986. A. Isidori. Nonlinear Control Systems. Communications and Control Engineering, Series third edition. Springer-Verlag, Berlin, 1995. S. Jain and F. Khorrami. Decentralized adaptive output feedback design for largescale nonlinear systems. IEEE Trans. Automat. Control, 42(5):729–735, 1997. Z.-P. Jiang. Decentralized and adaptive nonlinear tracking of large-scale systems via output feedback. IEEE Trans. Automat. Control, 45(11):2122–2128, 2000. M. Kawski. Stabilization of nonlinear systems in the plane. Systems Control Lett., 12(2):169–175, 1989. M. Kawski. Homogeneous stabilizing feedback laws. Control Theory Adv. Tech., 6(4):497–516, 1990. H. K. Khalil. Nonlinear Systems. Macmillan Publishing Company, New York, 1992. H. K. Khalil and A. Saberi. Decentralized stabilization of nonlinear interconnected systems using high-gain feedback. IEEE Trans. Automat. Control, 27(1):265–268, 1982. H. K. Khalil and A. Saberi. Adaptive stabilization of a class of nonlinear systems using high-gain feedback. IEEE Trans. Automat. Control, 32(11):1031–1035, 1987. A. J. Krener and A. Isidori. Linearization by output injection and nonlinear observers. Systems Control Lett., 3(1):47–52, 1983. A. J. Krener and W. Respondek. Nonlinear observers with linearizable error dynamics. SIAM J. Control Optim., 23(2):197–216, 1985. A. J. Krener and M. Xiao. Nonlinear observer design in the Siegel domain. SIAM J. Control Optim., 41(3):932–953, 2002. M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic. Nonlinear and Adaptive Control Design. John Wiley, New York, 1995.

280 [Li84]

Jason Polendo, Chunjiang Qian, and Cheryl B. Schrader

A. Linnemann. Decentralized control of dynamically interconnected systems. IEEE Trans. Automat. Control, 29(11):1052–1054, 1984. [MT91] R. Marino and P. Tomei. Dynamic output feedback linearization and global stabilization. Systems Control Lett., 17(2):115–121, 1991. [MPD94] F. Mazenc, L. Praly, and W. P. Dayawansa. Global stabilization by output feedback: examples and counterexamples. Systems Control Lett., 23(2):119–125, 1994. [Po06] J. Polendo. Global Synthesis of Highly Nonlinear Dynamic Systems with Limited and Uncertain Information. The University of Texas at San Antonio, Ph.D. Dissertation, 2006. [PQ05] J. Polendo and C. Qian. A generalized framework for global output feedback stabilization of genuinely nonlinear systems. In IEEE Conference on Decision and Control, pages 2646–2651, 2005. [PQ07a] J. Polendo and C. Qian. Decentralized output feedback control of interconnected systems with high-order nonlinearities. In Proceedings of 2007 American Control Conference, pages 1479–1484, 2007. [PQ07b] J. Polendo and C. Qian. A generalized framework for global output feedback stabilization of inherently nonlinear systems with uncertainties. In S. Tang and J. Yong, editors, Control Theory and Related Topics. World Scientific, Singapore, 2007. [PQ07c] J. Polendo and C. Qian. A generalized homogeneous domination approach for global stabilization of inherently nonlinear systems via output feedback. International Journal of Robust and Nonlinear Control, 17(7):605–629, May 2007. [PJ03] L. Praly and Z. Jiang. On global output feedback stabilization of uncertain nonlinear systems. In Proceedings of the 42nd IEEE Conference on Decision and Control, 2003. [Qi05] C. Qian. A homogeneous domination approach for global output feedback stabilization of a class of nonlinear systems. In Proceedings of 2005 American Control Conference, June 2005. [QL01] C. Qian and W. Lin. A continuous feedback approach to global strong stabilization of nonlinear systems. IEEE Trans. Automat. Control, 46(7):1061–1079, 2001. [QL02a] C. Qian and W. Lin. Output feedback control of a class of nonlinear systems: a nonseparation principle paradigm. IEEE Trans. Automat. Control, 47(10):1710– 1715, 2002. [QL02b] C. Qian and W. Lin. Smooth output feedback stabilization of planar systems without controllable/observable linearization. IEEE Trans. Automat. Control, 47(12):2068–2073, 2002. [QL04] C. Qian and W. Lin. Recursive observer design and nonsmooth output feedback stabilization of inherently nonlinear systems. In Proc. of 43rd IEEE Conference on Decision and Control, pages 4927–4932, Atlantis, Bahamas, 2004. [SK85] A. Saberi and H. K. Khalil. Decentralized stabilization of interconnected systems using output feedback. Internat. J. Control, 41(6):1461–1475, 1985. [Ts91] J. Tsinias. A theorem on global stabilization of nonlinear systems by linear feedback. Systems Control Lett., 17(5):357–362, 1991. [YL04] B. Yang and W. Lin. Homogeneous observers, iterative design, and global stabilization of high-order nonlinear systems by smooth output feedback. IEEE Trans. Automat. Control, 49(7):1069–1080, July 2004.

Volterra Control Synthesis Patrick M. Sain Raytheon Company, P.O. Box 902, El Segundo, CA 90245, USA [email protected]

Summary. This work presents a systematic method for deriving and realizing nonlinear controllers and nonlinear closed-loop systems. The nonlinear behavior is modeled using a finite number of the Volterra kernels of the plant and the desired input-output map of the closed-loop system. The control design takes place in the frequency domain and is realized as an interconnected set of linear systems. These linear systems enjoy a recursive structure that greatly simplifies realization, and a parallel structure that lends itself to distributed processing.

1 Introduction This work extends the nonlinear Volterra control design results of Al-Baiyat [ABS85, AB86, ABS86, ABS89] for linear analytic plants. The design takes place in the total synthesis problem (TSP) framework studied by Peczkowski, Sain et al. [Sai78, PS78,Pec79,PSL79,LPS79,Pec80,Gej80,PS80a,SM80,PS80b,SSD80,PS81,SP81, SWG+ 81, SAW+81, AS81, SY82, SP82, SS82, AS83, DSW83, AS84a, Dud84, PS84, SS84,AS84b,PS85,SP85]. In [Sai97,Sai05], the author presented a general approach for design and realization of nonlinear controllers to construct partially linear closedloop systems, wherein the first n Volterra kernels represent a purely linear system. Both the plant and the input-output map of the closed-loop system are required to be linear analytic, and the design uses the TSP paradigm. In the sequel, the partially linear restriction is removed for the closed-loop input-output map, and the generalized controller design and realization results are presented. In this manner, nonlinear closed-loop systems can be designed, a striking feature of the Volterra synthesis approach described herein. The sequel begins with a brief overview of the TSP framework, followed by a Volterra representation of linear analytic plants, and then controller synthesis. Two equivalent controller realizations are presented: one following directly from the synthesis equations, and a second, greatly simplified, form that takes advantage of repeated terms. Block diagrams of the realizations are shown to emphasize their structure as interconnections of linear subsystems, and to help visualize the synthesis equations. C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 14, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

282

Patrick M. Sain

2 The Total Synthesis Problem (TSP) System Model Based upon Rugh’s results [Rug83b] using Volterra series to represent nonlinear systems, Al-Baiyat and Sain [ABS86, ABS89] applied the use of Volterra operators to nonlinear regulator design in the context of the frequency-domain TSP. Since then, Sain et al. have used the TSP framework to apply Volterra operators in nonlinear servomechanism designs. A model-predictive control design was presented by Doyle et al. [IOP95] in 1995. Al-Baiyat showed that the Volterra operators comprising the controller can be realized as interconnections of linear systems, and Sain [Sai97] derived equivalent reduced-order realizations, and these have been applied to regulator and servomechanism designs [PMSBFS97, Sai97]. The following development will focus on regulator design. The extension to servomechanism design is readily accomplished [Sai97]. Let R, U and Y denote real vector spaces of dimensions p, m and p, respectively, where R is the space of requests, U is the space of plant inputs and Y is the space of plant outputs. Let P :U → Y denote an input-output description of a (possibly) nonlinear plant. Define the desired closed-loop response to a command by T : R → Y , and the desired plant input for a command by M : R → U. The operators P, T and M are assumed to have Volterra series representations at the point or in the region of operation. An illustration of these operators for a regulator design is given in Figure 1. In general, a pair (M, T ) is desired such that T = P ◦ M and the diagram in Figure 2 commutes. A common design approach is to specify T , and then find and realize a controller G :Y → U such that M : G → E, where E : R → Y , with Y being the space of output errors. E is assumed to have a Volterra series representation at the point of or in the region of operation.

3 Volterra Representation of Linear Analytic Plants The design approach herein uses the class of linear analytic systems, defined as follows. Let t ∈ [0, T ] ⊂ R be an interval, let u(t) ∈ U, x(t) ∈ X and y(t) ∈ Y , and let f :X → X, g:X → L(U, X ) and h:X → Y be analytic functions of x(t). Then a system is linear analytic if u(t) enters linearly and the state space description is

TL ML E r

9

−

e

G

u

Plant

y

Fig. 1. Feedback controller configuration, with ML = M and TL = T .

Volterra Control Synthesis

283

R M

T P

U

Y

Fig. 2. Commutative diagram for design equation T = P ◦ M.

x(t) ˙ = f [x(t)] + g[x(t)]u(t),

x(0) = x0 ,

y(t) = h[x(t)].

(1) (2)

In the sequel, the explicit dependence upon time will be suppressed for convenience. Linear analytic plants can be represented by what is called a bilinear approximation, and the specific approach herein uses the Carleman bilinearization. Begin by denoting Eqs. (1) and (2) as f1 (x, u) and f2 (x), respectively, and then taking a multivariable Taylor series expansion about the steady-state operating point (x0 , u0 ), truncating all terms of order higher than n: f1 (x0 , u0 ) = f2 (x0 ) =

n

n−1

i=1 n

i=1

˜ + D10u˜ ∑ A1i(x0 , u0 )x˜[i] + ∑ Di1 (x0 , u0 )(x˜[i] ⊗ u)

(3)

∑ C1i (x0 , u0)x˜[i] .

(4)

i=1

Here, x˜ = x − x0 , u˜ = u − u0 and x[i] = x ⊗ · · · ⊗ x, the i-fold Kronecker tensor product of x with itself. In this section, the notation assumes that tensor products have precedence over matrix multiplication in order to reduce notational complexity. Suppressing the explicit dependence on (x0 , u0 ), n n−1 ∂ j f1 [i] = A x ˜ + ji ∑ ∑ D j(i−1)x˜[i] ⊗ u.˜ ∂xj i= j i= j

Equivalently, in matrix form, ⎤ ⎡ x˜ A11 [2] ⎥ ⎢ ⎢ x ˜ d ⎢ ⎥ ⎢ 0 ⎢ .. ⎥ = ⎢ .. dt ⎣ . ⎦ ⎣ . ⎡

x˜[n]

0

A12 A22 .. . 0

··· ··· .. .

⎤⎡ ⎤ x˜ A1n ⎢ [2] ⎥ A2n ⎥ ⎥ ⎢ x˜ ⎥ .. ⎥ ⎢ .. ⎥ . ⎦⎣ . ⎦

0 Ann

x˜[n]

(5)

284

Patrick M. Sain

⎡

D11 ⎢ 0 ⎢ +⎢ . ⎣ .. 0

D12 D22 .. . 0

⎡ ⎤⎡ ⎤ ⎤ x˜ · · · D1(n−1) D10 ⎢ [2] ⎥ ⎢ 0 ⎥ · · · D2(n−1) ⎥ ⎢ ⎥ ⎢ x˜ ⎥ ⎥ ⊗ u ˜ + ˜ ⎢ ⎢ .. ⎥ u, ⎥ ⎥ . . .. .. ⎣ . ⎦ ⎦ ⎣ .. ⎦ . 0 0 Dn(n−1) x˜[n]

(6)

for 2 ≤ j ≤ n, and in a more compact form, the bilinear approximation becomes x˙˜ = Ax˜ + Dx˜ ⊗ u, ˜ ˜ y = Cx.

˜ = x˜0 x(0)

(7) (8)

In the sequel, Ak denotes the matrix partition of A composed of the leftmost k partitions of the topmost k partitions of A. Likewise, Dk denotes the matrix partition of D composed of the leftmost k − 1 partitions of the topmost k partitions of D (note D1 = D10 ), and Ck denotes the leftmost partitions of C. Having obtained the bilinear approximation, the next step is to consider Volterra representation. Given a multiple-input, multiple-output, causal, finite-dimensional, time-invariant plant, the homogeneous stationary Volterra kernel [CI84] is defined as pi (t1 , . . . ,ti ) =         CeAti D eA(ti−1 −ti ) D · · · D eA(t1 −t2 ) B ⊗ Im · · · ⊗ Im ⊗ Im ,

(9)

where t1 ≥ · · · ≥ t j ≥ 0 and Im is the m-dimensional matrix identity. The corresponding multilinear Volterra operator is defined as  t  τi−1 Pi [u(t)] = · · · pi (τ1 , . . . , τi )u(t − τ1 ) ⊗ · · · ⊗ u(t − τi ) d τi . . . d τ1 , (10) 0

0

where t ≥ τ1 ≥ τ2 ≥ · · · ≥ τi ≥ 0. In the TSP framework, the Volterra representation of the plant is then defined [ABS86, ABS89] as ∞

y(t) = ∑ Pi [u(t)].

(11)

i=1

Because Volterra operators are composed of multiple convolutions, the Laplace transform domain is a convenient workspace. Let ∞ ∞ Pi (s1 , . . . , si ) = · · · pi (τ1 , . . . , τi ) e−(s1 τ1 +···+si τi ) d τ1 . . . d τi (12) 0

0

define the ith Volterra kernel multidimensional Laplace transform [BEG74, BR71]. Given a stationary time-invariant bilinear system, the transforms of the first three kernels are P1 (s) = C(sI − A)−1B,   −1   P2 (s1 , s2 ) = C (s1 + s2 )I − A D (s1 I − A)−1 B ⊗ Im ,

(13) (14)

Volterra Control Synthesis

285

 −1 P3 (s1 , s2 , s3 ) = C (s1 + s2 + s3 )I − A D    −1   (s1 + s2 )I − A D (s1 I − A)−1 B ⊗ Im ⊗ Im , (15) where the general term is given by Pj (s1 , . . . , s j ) = −1  −1  (s1 + · · · + s j−1)I − A D C (s1 + · · · + s j )I − A D        · · · D (s1 I − A)−1B ⊗ Im · · · ⊗ Im ⊗ Im .

(16)

These expressions can be written in terms of partitions of the matrices A, D and C, leading to an equivalent reduced order realization [Sai97] P1 (s) = C1 (sI − A1 )−1 D1 , (17)  −1    (18) D2 (s1 I − A1)−1 D1 ⊗ Im , P2 (s1 , s2 ) = C2 (s1 + s2 )I − A2  −1 P3 (s1 , s2 , s3 ) = C3 (s1 + s2 + s3 )I − A3 D3  −1     (s1 + s2 )I − A2 D2 (s1 I − A1 )−1 D1 ⊗ Im ⊗ Im (19) with the general term  −1 Pj (s1 , . . . , s j ) = C j (s1 + · · · + s j )I − A j  −1 (s1 + · · · + s j−1)I − A j−1 Dj        D j−1 · · · D2 (s1 I − A1 )−1 D1 ⊗ Im · · · ⊗ Im ⊗ Im .

(20)

To make the equations more tractable in the sequel, let Pi = ((s1 + · · · + s j )I − Ai )−1 Di , j

(21)

and then the plant kernels are expressed as 1

P1 (s) = C1 P1 , P2 (s1 , s2 ) = P3 (s1 , s2 , s3 ) =

2 1 C2 P2 {P1 ⊗ Im }, 3 2 1 C3 P3 {[P2 {P1 ⊗ Im }] ⊗ Im},

(22) (23) (24)

286

Patrick M. Sain

with the general term j

j−1

2

1

Pj (s1 , . . . , s j ) = C j P j {[P j−1 {[· · · P2 {P1 ⊗ Im } · · · ] ⊗ Im}] ⊗ Im}.

(25)

j

Where the context is clear, the superscript on Pi will be suppressed in the sequel. Provided care is taken with grouping and ordering, the superscript is superfluous for the development herein [Sai97]. 3.1 Notes on Frequency Domain and Realization of Volterra Kernels A short discussion on realizing Volterra kernel frequency domain transforms is useful at this point. In addition to providing insight on the structure and interpretation of the equations presented so far, the realization properties and algorithms are part of the foundation for the sequel. The ideas and techniques are taken from Schetzen, AlBaiyat and Sain [Sch65, ABS85, AB86, ABS86, ABS89]. Consider linear, causal, time-invariant, homogeneous Volterra kernels h1 (t), h2 (t) and h3 (t), connected in the second order system configuration shown in Figure 3, such that z3 (t) = z1 (t) ⊗ z2 (t). (26) Then, ∞

  h3 (σ ) z1 (t − σ ) ⊗ z2(t − σ ) d σ 0∞   ∞ = h3 (σ ) h1 (σ1 )u(t − σ − σ1 ) d σ1 0 0  ∞  ⊗ h2 (σ2 )u(t − σ − σ2 ) d σ2 d σ .

y(t) =

(27)

(28)

0

Letting τ1 = σ + σi , i = 1, 2, and rearranging yields ∞ ∞ ∞   y(t) = h3 (σ ) h1 (τ1 − σ ) ⊗ h2(τ2 − σ ) 0 0 0   u(t − τ1 ) ⊗ u(t − τ2 ) d σ2 d σ1 d σ  ∞  ∞  ∞    = h3 (σ ) h1 (τ1 − σ ) ⊗ h2(τ2 − σ ) d σ 0 0 0  u(t − τ1 ) ⊗ u(t − τ2 ) d σ2 d σ1 ,

(29)

where the quantity in large parentheses in Eq. (29) shall be defined as the second order Volterra kernel h4 (τ1 , τ2 ) for the composite system in Figure 3. The transform of the Volterra kernel h4 (τ1 , τ2 ) is obtained per Eq. (12), yielding ∞ ∞ H4 (s1 , s2 ) = h4 (τ1 , τ2 )e−(s1 τ1 +s2 τ2 ) d τ1 d τ2 (30) 0

0

Volterra Control Synthesis

h1 (τ1 ) u

h2 (τ2 )

287

z1

z2

⊗

z3

h3 (τ3 )

y

Fig. 3. A second-order system.

 ∞  ∞  ∞ = 0

0

   h3 (σ ) h1 (τ1 − σ ) ⊗ h2(τ2 − σ ) d σ e−(s1 τ1 +s2 τ2 ) d τ1 d τ2 .

0

(31) The change of variables ti = τi − σ , i = 1, 2 yields   ∞ −(s1 +s2 )σ H4 (s1 , s2 ) = h3 (σ )e dσ 0  ∞   ∞  −s1t1 −s2 t2 h1 (t1 )e dt1 ⊗ h2 (t2 )e dt2 0 0  = H3 (s1 + s2 ) H1 (s1 ) ⊗ H2(s2 ) .

(32)

A multidimensional result given by Rugh [Rug83b] is useful. Let gi (τ1 , . . . , τi ) and hi (τ1 , . . . , τi ) be ith order causal, homogeneous Volterra kernels. Suppose hi is composed of a linear system g1 in series following the ith order system gi . Then, Hi (s1 , . . . , si ) = G1 (s1 + · · · + si )Gi (s1 , . . . , si ).

(33)

Thus, if a system can be decomposed into the form of Eq. (32), then it can be realized using the configuration shown in Figure 4. In a similar fashion, but without as much detail, the following result from AlBaiyat [AB86] is presented for an ith order Volterra kernel. Let h j (τ1 , . . . , τ j ), j = 1, . . . , i, be causal, homogeneous, time-invariant, multi-input, multi-output Volterra kernels, and let H j (s1 , . . . , s j ) be their respective Laplace transforms. Then Figure 5 shows a frequency-domain realization of the ith order system

H1 (s)

H2 (s)

⊗

H3 (s)

Fig. 4. Figure 3 in the transform domain.

288

Patrick M. Sain

...

H2i−2 (s)

H4 (s) H2 (s) H1 (s)

⊗

H3 (s)

⊗

H5 (s)

...

⊗

H2i−1 (s)

Fig. 5. Realization of the ith order kernel Hi (s1 , . . . , si ) in Eq. (34).



Hi (s1 , . . . , si ) = H2i−1 (s1 + · · · + si ) · · · H5 (s1 + s2 + s3 )      H3 (s1 + s2 ) H1 (s1 ) ⊗ H2(s2 ) ⊗ H4 (s3 ) ⊗ · · · ⊗ H2i−2(si ) . (34)

4 Controller Synthesis Now Volterra representations will be obtained for the general controller G in the TSP framework in terms of the kernels of the plant P and the desired closed-loop input-output map T . The development is based upon the work by Al-Baiyat and Sain [ABS86] as modified by Sain [Sai97]. To aid presentation, matrix multiplication is assumed to have precedence over the tensor product in this section. Thus, ab ⊗ cd is equivalent to (ab)⊗(cd). In addition, the definitions of the Volterra operators make precedence clear as well. Similar to the Volterra operator Pi [u(t)], define Volterra operators for the maps T and M: ∞

y(t) = T [r(t)] =

∑ Tj [r(t)]

(35)

∑ Mk [r(t)].

(36)

j=1 ∞

u(t) = M[r(t)] =

k=1

The fact that Volterra operators are multilinear [Rug83a, Por76, Sch80] supports derivation of a relation between operators Pi , T j and Mk : replace the command signal r(t) with cr(t), where c is an arbitrary real constant, and then equate terms having like powers of c:

Volterra Control Synthesis



∞

∞

j=1

i=1 ∞

j=1 ∞

∞

j1 =1

ji =1

∑ c j Tj [r(t)] = ∑ Pi ∑ c j M j [r(t)] ∑ · · · ∑ c j1 +···+ ji Pi

=

289

(37)   M j1 [r(t)], . . . , M ji [r(t)] ,

(38)

where for ri = r(t − τi ),   Pi M j1 [r(t)], . . . , M ji [r(t)] =  t  τ1  τi−1 ··· pi (τ1 , . . . , τi )M j1 [r(t − τ1 )] ⊗ · · · ⊗ M ji [r(t − τi )] d τi . . . d τ1 . 0 0

0

(39) Suppressing the argument r(t), ∞

∞

∞

∞

i=1

j1 =1

ji =1

∑ c Tj = ∑ ∑ · · · ∑ c j

j=1

j1 +···+ ji

Pi (M j1 , . . . , M ji ) ,

(40)

and equating terms having like powers of c yields T1 = P1 M1 ,

(41)

T2 = P1 M2 + P2(M1 , M1 ), T3 = P1 M3 + P2(M1 , M2 ) + P2(M2 , M1 ) + P3(M1 , M1 , M1 ),

(42) (43)

where the general term has the form ⎛ i

Ti = P1 Mi + ∑ ⎝ j=2

i−( j−1) i−( j−1)−(k1 −1)

∑

k1 =1

∑

i−( j−1)−(k1 −1)−···−(k j−2 −1)

···

k2 =1

∑

k j−1 =1

Pj (Mk1 , Mk2 , . . . , Mk j−1 , Mi−k1 −k2 −···−k j−1 ) .

(44)

Applying the multidimensional Laplace transform provides T1 (s) = P1 (s)M1 (s), T2 (s1 , s2 ) = P1 (s1 + s2 )M2 (s1 , s2 )   + P2 (s1 , s2 ) M1 (s1 ) ⊗ M(s2 ) , T3 (s1 , s2 , s3 ) = P1 (s1 + s2 + s3 )M3 (s1 , s2 , s3 )   + P2 (s1 , s2 + s3 ) M1 (s1 ) ⊗ M2 (s2 , s3 )   + P2 (s1 + s2 , s3 ) M2 (s1 , s2 ) ⊗ M1 (s3 )   + P3 (s1 , s2 , s3 ) M1 (s1 ) ⊗ M1(s2 ) ⊗ M1(s3 ) ,

(45) (46)

(47)

290

Patrick M. Sain

with the general term Ti (s1 , . . . , si ) = P1 (s1 + · · · + si )Mi (s1 , . . . , si ) ⎧ i−( j−1)−(k1 −1)−···−(k j−2 −1) i ⎨i−( j−1) i−( j−1)−(k1 −1) +∑ · · · ∑ ∑ ⎩ k∑ j=2 =1 k =1 k =1 1

2

j−1

Pj (s1 + · · · + sk1 , sk1 +1 + · · · + sk1 +k2 , . . . , sk1 +k2 +···+k j−1 +1 + · · · + si )  Mk1 (s1 , . . . , sk1 ) ⊗ Mk2 (sk1 +1 , . . . , sk1 +k2 )  ⊗ · · · ⊗ Mi−k1 −···−k j−1 (sk1 +k2 +···+k j−1 +1 , . . . , si ) .

(48)

Note that Eq. (45) is the TSP design equation. If the pair (M1 (s), T1 (s)) is chosen for P1 (s), then one can proceed to Eq. (46), where one chooses the pair (M2 (s1 , s2 ), T2 (s1 , s2 )), and so on. Next, given M = G ◦ E, by inspection from Eqs. (45)–(47) one obtains M1 (s) = G1 (s)E1 (s),

(49)

M2 (s1 , s2 ) = G1 (s1 + s2 )E2 (s1 , s2 )   + G2 (s1 , s2 ) E1 (s1 ) ⊗ E(s2 ) , M3 (s1 , s2 , s3 ) = G1 (s1 + s2 + s3 )E3 (s1 , s2 , s3 )   + G2 (s1 , s2 + s3 ) E1 (s1 ) ⊗ E2 (s2 , s3 )   + G2 (s1 + s2 , s3 ) E2 (s1 , s2 ) ⊗ E1(s3 )   + G3 (s1 , s2 , s3 ) E1 (s1 ) ⊗ E1 (s2 ) ⊗ E1(s3 ) ,

(50)

(51)

with the general term Mi (s1 , . . . , si ) = G1 (s1 + · · · + si )Ei (s1 , . . . , si ) ⎧ i−( j−1)−(k1 −1)−···−(k j−2 −1) i ⎨i−( j−1) i−( j−1)−(k1 −1) +∑ · · · ∑ ∑ ⎩ k∑ j=2 =1 k =1 k =1 1

2

j−1

G j (s1 + · · · + sk1 , sk1 +1 + · · · + sk1 +k2 , . . . , sk1 +k2 +···+k j−1 +1 + · · · + si )  Ek1 (s1 , . . . , sk1 ) ⊗ Ek2 (sk1 +1 , . . . , sk1 +k2 )  ⊗ · · · ⊗ Ei−k1 −···−k j−1 (sk1 +k2 +···+k j−1 +1 , . . . , si ) . (52) As an aside, from Figure 1, note e = r − y, so

Volterra Control Synthesis ∞

∞

i=1

i=1

∑ Ei [r(t)] = I[r(t)] − ∑ Ti [r(t)],

291

(53)

where I is the identity operator. A procedure similar to the one followed above for obtaining a relation between the operators P, T and M yields E1 (s) = I − T1 (s) Ei (s1 , . . . , si ) = −Ti (s1 , . . . , si ),

(54) i > 1.

(55)

Clearly, from Eqs. (45)–(47), Eqs. (49)–(51) and Eqs. (54)–(55), a direct relation between the Volterra kernels of the operators T , M and G can be found if the design is well posed on the first order, or linear, level, implying that E1 (s) has an inverse. As T is usually open to choice, the latter requirement is reasonable. In order to improve the tractability of the development, the frequency variables si , i = 1, 2, . . ., are suppressed without loss of information [Sai97] in the sequel. Proceeding with the controller synthesis, replace the quantities Mi , i = 1, 2, . . ., in Eqs. (45)–(48) with the expressions in Eqs. (49)–(52) to obtain T1 = P1 {G1 E1 }

(56)

  T2 = P1 G1 E2 + G2 (E1 ⊗ E1 )   + P2 G1 E1 ⊗ G1E1 ,

(57)

  T3 = P1 G1 E3 + G2 (E1 ⊗ E2 ) + G2(E2 ⊗ E1) + G3 (E1 ⊗ E1 ⊗ E1 )   + P2 G1 E1 ⊗ [G1E2 + G2 (E1 ⊗ E1)]   + P2 [G1 E2 + G2 (E1 ⊗ E1)] ⊗ G1 E1   + P3 G1 E1 ⊗ G1E1 ⊗ G1 E1 .

(58)

The general expressions for these intermediate results do not add value to this presentation, and are therefore omitted; only terms up to third order are explicitly derived in this presentation (complete details are given by Sain [Sai97]). In the discussion on realization, a compact recursive notation is developed that permits a tractable general expression. Solving each of the quantities Ti for the corresponding Gi yields G1 = −P1−1 T1 E1−1

(59)

292

Patrick M. Sain

  G2 = −P1−1 E2 E1−1 ⊗ E1−1   − G1E2 E1−1 ⊗ E1−1    − P1−1P2 G1 E1 ⊗ G1 E1 E1−1 ⊗ E1−1 ,

(60)

  G3 = −P1−1E3 E1−1 ⊗ E1−1 ⊗ E1−1   − G1 E3 E1−1 ⊗ E1−1 ⊗ E1−1    − G2 E1 ⊗ E2 E1−1 ⊗ E1−1 ⊗ E1−1    − G2 E2 ⊗ E1 E1−1 ⊗ E1−1 ⊗ E1−1    − P1−1P2 G1 E1 ⊗ G1 E2 E1−1 ⊗ E1−1 ⊗ E1−1    − P1−1P2 G1 E1 ⊗ G2 (E1 ⊗ E1 ) E1−1 ⊗ E1−1 ⊗ E1−1    − P1−1P2 G1 E2 ⊗ G1 E1 E1−1 ⊗ E1−1 ⊗ E1−1    − P1−1P2 G2 (E1 ⊗ E1 ) ⊗ G1E1 E1−1 ⊗ E1−1 ⊗ E1−1    − P1−1P3 G1 E1 ⊗ G1 E1 ⊗ G1 E1 E1−1 ⊗ E1−1 ⊗ E1−1 ,

(61)

−(i)

where E1 is the i-fold tensor product of E1−1 with itself. Incorporating the E1−1 terms using repeated applications of the tensor identity (a ⊗ b)(c ⊗ d) = (ac) ⊗ (bd)

(62)

and using the notation T i (s1 , . . . , si ) = [(s1 + · · · + si )I − Ai ]−1 Di ,

i = 1, 2, . . . ,

where Ai and Di are defined similarly to their counterparts Ai and Di , yields   G2 = P1−1C2 T 2 T 1 E1−1 ⊗ E1−1   + G1C2 T 2 T 1 E1−1 ⊗ E1−1   − P1−1P2 G1 ⊗ G1 , and

  G3 = P1−1C3 T 3 [T 2 (T 1 E1−1 ⊗ E1−1)] ⊗ E1−1   + G1C3 T 3 [T 2 (T 1 E1−1 ⊗ E1−1)] ⊗ E1−1

(63)

(64)

Volterra Control Synthesis

293

  + G2 I p ⊗ C2 T 2 (T 1 E1−1 ⊗ E1−1)   + G2 C2 T 2 (T 1 E1−1 ⊗ E1−1 ) ⊗ I p   + P1−1P2 G1 ⊗ G1C2 T 2 (T 1 E1−1 ⊗ E1−1)   − P1−1P2 G1 ⊗ G2(I p ⊗ I p)   + P1−1P2 G1C2 T 2 (T 1 E1−1 ⊗ E1−1 ) ⊗ G1   − P1−1P2 G2 (I p ⊗ I p) ⊗ G1   − P1−1P3 G1 ⊗ G1 ⊗ G1 .

(65)

  G2 = P1−1C2 T 2 T 1 E1−1 ⊗ E1−1   + G1C2 T 2 T 1 E1−1 ⊗ E1−1   − P1−1C2 P2 P1 G1 ⊗ G1 ,

(66)

Collecting terms yields

  G3 = P1−1C3 T 3 [T 2 (T 1 E1−1 ⊗ E1−1)] ⊗ E1−1   + G1C3 T 3 [T 2 (T 1 E1−1 ⊗ E1−1)] ⊗ E1−1  + G2 I p ⊗ C2 T 2 (T 1 E1−1 ⊗ E1−1)  +C2 T 2 (T 1 E1−1 ⊗ E1−1) ⊗ I p  + P1−1C2 P2 P1 G1 ⊗ G1C2 T 2 (T 1 E1−1 ⊗ E1−1 ) −P1 G1 ⊗ G2 +P1 G1C2 T 2 (T 1 E1−1 ⊗ E1−1 ) ⊗ G1  −P1 G2 ⊗ G1   − P1−1C3 P3 [P2 (P1 G1 ⊗ G1 )] ⊗ G1 .

(67)

The realizations corresponding to the equations in Eqs. (66) and (67) are shown in Figures 6 and 7. The following notational definitions are used in the diagrams. First, the “C n prime” linear systems, denoted Cn , consist of the output state matrix of the Carleman bilinear system of order n for the desired closed-loop mapping T . Second, the boxes labeled Gi , 1 ≤ i ≤ n − 1 represent instances of the ith order general con-

294

Patrick M. Sain

troller. As can be seen from the diagrams, the realization of the nth order controller (n ≥ 2) kernel contains instances of all controllers of order 1 through n − 1. Third, in order to reduce the size of the block diagrams somewhat, part of the notation used in the simplification process presented in the sequel is introduced here. Specifically, if e ∈ Y represents the output error of the closed loop, and δ11 is its Laplace transform, [i] then let δ11 denote the i-fold tensor product of δ11 with itself, and define

γii =

i

∑ Gk (s1 , . . . , sk )(δ11 ), (k)

(68)

k=1

where Gi is the Laplace transform of the ith order Volterra controller, so that γii is the Laplace transform of the output of the ith order controller. Also, define

σ1 = E1−1 δ11

(69)

= (I − T1 )−1 δ11 .

(70)

Also, an unlabeled line represents a direct feed of the transformed error signal δ11 . Because tensor multiplication is not generally commutative, junctions containing the tensor product symbol ⊗ in Figures 6–12 are shown uniformly to have two inputs: one from the left and one from the top. Denoting these two inputs as a and b, respectively, then the output of the junction is a ⊗ b. Also, variables positioned below a line in a figure represent assignment, meaning that the variable is being defined to be equal to the quantity represented by the line itself. Variables positioned above the line indicate reference, meaning that the quantity represented by the line (usually an input to a system) is being defined to be equal to the value of the variable. In this case, it is assumed that the value of the variable is assigned elsewhere. As is readily apparent, the general expression for the controller Gn , in addition to being unwieldy, also contains large numbers of recurring terms, corresponding to repeated linear subsystems in various combinations. Sain [Sai97] capitalized on these

σ1 σ1

T1

⊗

P1−1 C2 T 2

T1

⊗

T2

P1

⊗

−P1−1 C2 P 2

σ1 σ1

−

C2

G1

+

γ11 γ11

Fig. 6. Realization of G2 given in Eq. (66).

+

Volterra Control Synthesis

295

σ1 σ1 σ1

T1

⊗

T2

⊗

P1−1 C3 T 3

T1

⊗

T2

⊗

T3

T1

⊗

T2

−

σ1 σ1 σ1

−

C3

+

G1

σ1 σ1

C2 ⊗

σ1 σ1

T1

⊗

T2

−

C2

⊗

T1

⊗

T2

−

C2

G1

+

+

G2

σ1 σ1 γ11

⊗

P1

γ22 γ11 γ11

P1

⊗

T1

⊗

P1

⊗

P1

⊗

+

σ1 σ1

T2

−

C2

G1

P1

⊗

+

γ11 γ22

+

−P1−1 C2 P 2

+

γ11 γ11 γ11

P2

⊗

−P1−1 C3 P 3

+

Fig. 7. Realization of G3 given in Eq. (67).

recurring features to produce simplified controller design equations and realizations. Two approaches were used to arrive at the simplified controller: one analytical, and one graphical, both of which are presented in the sequel.

296

Patrick M. Sain

δ11

P1−1 C1 T 1

E1−1 σ 1

γ11 = β11

Fig. 8. Simplified realization of G1 .

σ2

P1−1 C2 T 2

σ1 σ1 β11

⊗σ

T2

2

β11

P1

α22

⊗

σ2 β22

 −C 2

G1

δ21

γ21

−P1−1 C2 P 2

+ +

Fig. 9. Simplified realization of G2 .

σ3

P1−1 C3 T 3

σ1 σ2 δ21 δ11 δ11 δ21 β21 α22 β11

⊗σ

T3

3

σ3

 −C 3

G1

δ31

γ31

+

⊗ ⊗

+

δ32

G2

+

γ32

+

γ21 γ22 β11 β22

+

β21

P2

α33

P1 ⊗

α32 β33

⊗

+

β32

−P1−1 C2 P 2

−P1−1 C3 P 3

Fig. 10. Simplified realization of G3 .

+ +

Volterra Control Synthesis

σn

P1−1 Cn T n

σ1 ⊗σ

δ(n−2)1 δ21 δ(n−3)1 ...

δ31

...

 −C n

δn1

G1

γn1

⊗

+

⊗

+

⊗

+

δn2

G2

γn2

δ(n−j+1)1 δ(n−j)1 δj(j−1) δ(n−j−1)1

...

δ(j+1)(j−1)

⊗ ⊗

+

⊗

+

δ(n−j−i)1 ⊗

δ11

...

δ(n−1)(j−1)

+ ...

...

δ(j+1)(j−1)

⊗

+

δnj

Gj

γnj

δ21 δ(n−2)(n−2) δ11 δ(n−1)(n−2)

+

...

δ(j−1)(j−1)

+

⊗

δ11 δ(n−1)1

σn

...

δ11

Tn

n

+ ...

δ(n−1)1

...

 σn−1

⊗ ⊗

+

δn(n−1)

Gn−1

γn(n−1)

+

Fig. 11. Simplified realization of the Gi j and Tn blocks of Gn (see also Figure 12).

297

Patrick M. Sain

β(n−1)1 α22 β(n−2)1 α32

⊗ + ...

...

⊗

β(n−i)1 α(i+1)2

+ ...

...

⊗

β21 α(n−1)2 β11 γ(n−1)1 γ(n−1)2

⊗

+

...

... ...

β(n−j−i)1 α(j+i+1)j

...

β21 α(n−1)j β11 β(n−1)(j−1) β21 α(n−1)(n−1) β11 β(n−1)(n−2) β11 β(n−1)(n−1)

β(n−1)1

P1

αn2

⊗

+

βn2

−P1−1 C2 P 2 ...

β(n−j+1)1 αjj β(n−j)1 α(j+1)j

+

⊗ ⊗

+

⊗

+

⊗ P j−1

+ αnj

⊗

+

⊗

+

βnj

−P1−1 Cj P j

+ ...

...

γ(n−1)(n−1)

...

...

+

...

298

⊗ P n−2 P n−1

αn(n−1) αnn

⊗

βn(n−1) βnn

−P1−1 Cn−1P n−1

+

−P1−1 Cn P n

+

Fig. 12. Simplified realization of the Pn block of Gn (see also Figure 11).

Volterra Control Synthesis

299

4.1 Simplified General Controller Design and Realization: An Analytical Approach This section introduces a simplified general controller design and realization. The simplification rests upon a recursive notation that permits the expanded summations used for the general expressions to be expressed in a very compact manner. The design for the controller is achieved using the general equations in just a few lines of mathematics. The approach is based upon the key result given in Lemma 1 that reduces the general expressions describing the ith order Volterra kernels to an extremely compact form using a recursive subscripted notation. Lemma 1. Given ak ∈ Cnk ×mk , 1 ≤ k ≤ i − 1 and κi = ∑ij=1 k j , κ0 = 0, i

bi =

⎧ ⎨i− j+1

i− j−κ1 +2

∑⎩ ∑

j=2

∑

k1 =1

i−κ j−2 −1

∑

···

k2 =1

ai−κ j−1 ⊗ · · · ⊗ ak2 ⊗ ak1

k j−1 =1

⎫ ⎬ ⎭

,

i ≥ 2,

(71)

let i− j+1

bi j =

∑

i− j−κ1 +2

∑

k1 =1

i−κ j−2 −1

···

k2 =1

∑

ai−κ j−1 ⊗ · · · ⊗ ak2 ⊗ ak1 ,

i ≥ 2,

(72)

k j−1 =1

so that

i

bi =

∑ bi j ,

(73)

j=2

and define bi1 = bi . Then i− j+1

bi j =

∑

b(i−k)( j−1) ⊗ ak ,

i ≥ 2,

(74)

k=1

and

i i− j+1

bi =

∑ ∑

b(i−k)( j−1) ⊗ ak ,

i ≥ 2.

(75)

j=2 k=1

Proof. The expression in Eq. (72) represents the sum of all possible j-fold tensor products of elements in the set {ak : 1 ≤ k ≤ i − 1} such that the subscripts on the elements ak sum to i. To simplify the expression, first consider   bi j 

i− j−k+2 k1 =k

=

∑

k2 =1

i− j−k−k2 +3

∑

k3 =1

i−k−k2 −···−k j−2 −1

···

∑

k j−1 =1

ai−k−k2 −···−k j−1 ⊗ · · · ⊗ ak2 ⊗ ak .

(76)

300

Patrick M. Sain

Substituting q = j − 1, r1 = k2 , r2 = k3 , . . ., rq−1 = r j−2 = k j−1 , and

κm =

m

∑ rl

(77)

l=1

yields   bi j 

i−q−k+1 k1 =k

=

∑

i−q−k−r1 +2

∑

r1 =1

i−k−r1 −···−rq−2 −1

∑

···

r2 =1

rq−1 =1

ai−k−r1 −···−rq−1 ⊗ · · · ⊗ ar1 ⊗ ak =

(i−k)−q+1

(i−k)−q−κ1 +2

r1 =1

r2 =1

∑

∑

a(i−k)−κ 

q−1

 −1 (i−k)−κq−2

···

∑

rq−1 =1

⊗ · · · ⊗ a r1 ⊗ a k

(78)

= b(i−k)q ⊗ ak = b(i−k)( j−1) ⊗ ak .

(79) (80)

Eqs. (74) and Eq. (75) follow immediately.

2

Corollary 1. bii =

[i] a1 .

Proof. We obtain the proof by a straightforward application of Lemma 1.

2

Proposition 1. Consider an expression of the form used for Ti shown in Eq. (44). Let π be any permutation over the set of integers {1, 2, . . . , j}. For i ≥ 2, ⎛ i

Ti = P1 Mi + ∑ ⎝ j=2

i−( j−1) i−( j−1)−(k1 −1)

∑

∑

k1 =1

i−( j−1)−(k1 −1)−···−(k j−2 −1)

···

k2 =1

∑

k j−1 =1

Pj (Mkπ (1) , Mkπ (2) , . . . , Mkπ ( j−1) , Mkπ ( j) ) .

(81)

Proof. In the expression for Ti given in Eq. (44), the operator Pj , 1 ≤ j ≤ i, operates on all possible ordered sets of j arguments (Mm1 , Mm2 , . . . , Mm j ) such that j

i=

∑ mq ,

1 ≤ mq ≤ i.

(82)

q=1

Thus, for example, if i = 5 and j = 3, P3 will operate on six ordered sets of arguments (Mm1 , Mm2 , Mm3 ), where (m1 , m2 , m3 ) ∈ {(1, 1, 3), (1, 3, 1), (3, 1, 1), (1, 2, 2), (2, 1, 2), (2, 2, 1)}.

(83)

Therefore, if Pj (Mm1 , Mm2 , . . . , Mm j ) is a term in the expression for Ti , then so are all of the terms Pj (Mmπ (1) , Mmπ (2) , . . . , Mmπ ( j) ) generated by the permutation π . The

Volterra Control Synthesis

301

effect of the permutation π is simply to reorder the addends in the outermost sum in the expression for Ti , and the commutative property of addition means the reordering does not change the result. Corollary 2. Reordering the arguments for Ei does not affect its value. Proof. Straightforward adaptation and application of Proposition 1.

2

Using Proposition 1 to reorder the subscripts on Ek , 1 ≤ k ≤ Ei in in the general expression for Mi and then applying Lemma 1 yields the recursive expression  i

i− j+1

j=2

k=1

Mi = G1 Ei + ∑ G j

∑

E(i−k)( j−1) ⊗ Ek ,

(84)

i

= G1 E i + ∑ G j E i j ,

(85)

j=2

where Ei1 = Ei . Similarly, given Mi1 = Mi , one obtains  i

i− j+1

j=2

k=1

Ti = P1 Mi + ∑ Pj

∑



M(i−k)( j−1) ⊗ Mk ,

(86)

i

= P1 Mi + ∑ Pj Mi j .

(87)

j=2

With the goal of obtaining a compact expression for the Laplace transform of the ith controller kernel Gi , consider i

Ti = P1 Mi + ∑ Pj Mi j 

j=2

i

(88) 

i

= P1 G1 Ei + ∑ G j Ei j + ∑ Pj Mi j , j=2

from which

(89)

j=2

i

i

j=2

j=2

P1−1 Ti = G1 Ei + ∑ G j Ei j + ∑ P1−1 Pj Mi j .

(90)

The second term on the right-hand side contains the only instance of Gi in the equation, and can be rewritten as i

i−1

j=2

j=2

∑ G j Ei j = ∑ G j Ei j + Gi Eii i−1

=

∑ G j Ei j + Gi E11.

j=2

[i]

(91) (92)

302

Patrick M. Sain

E1−1 is presumed to exist by design. Let −[i]

−1 −1 E11 = E11 ⊗ · · · ⊗ E11 .

(93)

Then, solving for Gi produces −[i]

Gi = P1−1 Ti E1

−[i]

− G1 E i j E 1

i−1

− ∑ G j Ei j E1

−[i]

j=2

i

− ∑ P1−1 Pj Mi j E1

−[i]

j=2

(94) −[i]

= P1−1 Ti E1

i−1

− ∑ G j Ei j E1 j=1

−[i]

i

− ∑ P1−1 Pj Mi j E1 . −[i]

(95)

j=2

The goal is now to use Eq. (95) to generate an equivalent, but simpler and more efficient means of obtaining the controller output for a given error signal than is offered by the design realization presented in the previous section. The procedure identifies a common set Γ0 of recurring terms. For implementation, these terms are calculated and stored when first encountered, and then simply recalled from memory when encountered again. The first class of common set members is generically denoted δi j and represents inputs to the Gi blocks. In particular, δ11 is the Laplace transform of the closedloop output error e ∈ Y . A second class is denoted σ j and σ j . These quantities are associated with the transform of the request signal and the output of the mapping T . A third class is denoted γi j , and is associated with the outputs of the Gi blocks. Fourth and fifth classes αi j and βi j are associated with the outputs of the Mi blocks. Their definitions will be made precise shortly. [i] −[i] [i] Given the output of the first term in the equation for Gi δ11 , P1−1 Ti E1 δ11 , define the common set elements −1 σ1 = E11 δ11  σi = T i σi σi+1 = σi ⊗ σ1 = (T i σi ) ⊗ σ1 .

(96) (97) (98) (99)

Then, −[i] [i]

P1−1 Ti E1 δ11 = P1−1Ci T i          [i] T i−1 · · · T 2 T 1 ⊗ I p ⊗ I p · · · ⊗ I p ⊗ I p σ1 = P1−1Ci T i          T i−1 · · · T 2 T 1 σ1 ⊗ σ1 ⊗ σ1 · · · ⊗ σ1 ⊗ σ1        = P1−1Ci T i T i−1 · · · T 2 σ2 ⊗ σ1 · · · ⊗ σ1 ⊗ σ1

Volterra Control Synthesis

= P1−1Ci T i σi .

303

(100)

The simplification does not continue to σi because P1−1 is, in general, not realizable by itself; instead, the product P1−1Ci T i must be realized as a single linear system. The identification ti = P1−1Ci T i σi (101) is used in the sequel to refer to the block output. Next, consider the output of the second term on the right-hand side of Eq. (95), i−1

i−1

− ∑ G j Ei j E1 δ11 = − ∑ G j Ei j σ1 . −[i] [i]

j=2

[i]

(102)

j=2

Using Lemma 1 in reverse yields [i]

Ei j σ1 =

i− j+1

∑

k1 =1

i− j−κ1 +2

∑

···

i−κ j−2 −1 

k2 =1

 [i] Ei−κ j−1 ⊗ · · · ⊗ Ek2 ⊗ Ek1 σ1 .

∑

(103)

k j−1 =1

Defining the common set element

δi j = −Ci σi ,

i ≥ 2,

(104)

the argument can be rewritten as       i−κ −Ti−κ j−1 σ1 j−1 ⊗ · · · ⊗ −Tk2 σ1k2 ⊗ −Tk1 σ1k1   ! !    = −Ci− κ j−1 T i−κ j−1 σi−κ j−1 ⊗ · · · ⊗ −Ck2 T k2 σk2 ⊗ −Ck1 T k1 σk1 = δi−κ j−1 ⊗ · · · ⊗ δk2 ⊗ δk1 .

(105)

Replacing this quantity back into the sum yields [i] Ei j σ1

=

i− j+1

i− j−κ1 +2

k1 =1

k2 =1

∑

i− j+1

=

∑

∑

i−κ j−2 −1

···

∑

δi−κ j−1 ⊗ · · · ⊗ δk2 ⊗ δk1

(106)

k j−1 =1

δ(i−k)( j−1) ⊗ δk1

(107)

k=1

= δi j .

(108)

Defining the common set element

γi j = G j δi j and returning to Eq. (102) produces

(109)

304

Patrick M. Sain i−1

i−1

− ∑ G j Ei j σ1 = − ∑ G j δi j [i]

j=2

(110)

j=2

i−1

= − ∑ γi j .

(111)

j=2

The identifications gi j = γi j

(112)

i−1

gi =

∑ gi j

(113)

j=1

are used in the sequel when the emphasis is on the block outputs and not the common set. Finally, consider the output of the third and last term in Eq. (95), i

i

− ∑ P1−1 Pj Mi j E1 δ11 = − ∑ P1−1 Pj Mi j σ1 , −[i] [i]

j=2

[i]

(114)

j=2

where using the Lemma 1 in reverse yields [i]

Mi j σ1 =

i− j+1

∑

k1 =1

i− j−κ1 +2

∑

···

i−κ j−2 −1 

k2 =1

∑

(115)

k j−1 =1

and

[i] Mi σ1

 [i] Mi−κ j−1 ⊗ · · · ⊗ Mk2 ⊗ Mk1 σ1 ,

i

G1 Ei + ∑ G j Ei j σ1

=

[i]

(116)

j=2

i

= G1 Ei σ1 + ∑ G j Ei j σ1 [i]

[i]

(117)

j=2

i

= −G1Ci T i σi + ∑ γi j

(118)

j=2

i

=

∑ γi j

(119)

j=1

 βi1 ,

(120)

with the last equality being part of the definition of the common set element βi1 . Substituting this back into Eq. (114) produces

Volterra Control Synthesis

− P1−1Pj Mi j σ1 = −P1−1 Pj [i]

i− j+1

∑

i− j−κ1 +2

∑

k1 =1

k2 =1

305

i−κ j−2 −1

···

∑

k j−1 =1

β(i−κ j−1 )1 ⊗ · · · ⊗ β(k2)1 ⊗ β(k1)1 .

(121)

Defining

αi j = P j−1 β(i−1)( j−1),

(122)

then i

i

i− j+1

j=2

k=1

− ∑ P1−1 Pj Mi j σ1 = − ∑ P1−1 Pj [i]

j=2

∑

α(i−k+1) j ⊗ βk1

(123)

i

= − ∑ P1−1 Pj βi j .

(124)

j=2

The identifications pi j = P1−1 Pj βi j

(125)

i

pi =

∑ P1−1Pj βi j

(126)

j=2

are used in the sequel to refer to the block outputs. Now, having considered each term of the expression in Eq. (95) individually, they can be pieced together to form the transform of the output of the ith order controller kernel, i−1

i

j=1

j=2

Gi δ11 = P1−1Ci T i σi − ∑ γi j − ∑ P1−1C j P j βi j [i]

= ti − gi − pi .

(127) (128)

Of note in this expression is that it is composed of three parts, each of which can be evaluated in a semi-independent manner. The recursive notation yields a tractable size for the expression, even though it is for the (transformed) output of an arbi[i] trary ith order controller kernel. Finally, note that Gi δ11 is the difference between the output of the kernel associated with the desired closed-loop behavior and the two outputs formed by the G j , j < i, and −PCPj , j ≤ i. For convenience, the expressions for all of the common set members are listed below.

306

Patrick M. Sain

αi j = Pi β(i−1)( j−1), i, j ≥ 2, ⎧ i−1 ⎪ ⎪ ⎪ j = 1, ⎪ ∑ γik , ⎨ k=1 βi j = i− j+1 ⎪ ⎪ ⎪ ⎪ ⎩ ∑ α(i−k+1) j ⊗ βk1 , j ≥ 2,

(129)

(130)

k=1

⎧ L {e}, i = j = 1, ⎪ ⎪ ⎪ ⎨ −C σ  , j = 1, i i δi j = i− j+1 ⎪ ⎪ ⎪ ⎩ ∑ δ(i−k)( j−1) ⊗ δk1 , j ≥ 2, 

γi j = 

σi =

=

k=1

Gi δii , j = i, Gi δi j , j < i, E1−1 δ11 ,

i = 1,

 ⊗σ , σi−1 1



σi

(131)

i ≥ 2,

T 1 δ11 , i = 1, T i σi ,

i ≥ 2.

(132)

(133)

(134)

The simplification procedure described above originated when frequencydomain block diagrams of the controller were examined up to fifth order [Sai97], revealing that many quantities appearing in G j reappeared in Gi , j < i. Based upon these recurrences, members of the common set Γ0 were selected, motivating the analytical definitions presented earlier. Space restrictions limit this presentation to third order and the general nth order case, shown in Figures 8–12.

5 Conclusion In summary, the generalized controller design and realization for Volterra feedback synthesis (VFS) has been presented using the TSP framework. The design provides the capability to specify desired nonlinear closed-loop input-output behavior, or to partially linearize a nonlinear multi-input, multi-output linear analytic plant at an operating point or in an operating region. To facilitate further theoretical work and implementation, an equivalent simplified form of reduced order is derived and presented. The analytical expressions are of particular note because they utilize the expressions for a controller of arbitrary order, made possible by a recursive notation that produces very compact expressions. Extensive notes on implementation and simulation of these controllers are given by Sain [Sai97], including the use of symmetric tensor products instead of Kronecker tensor products so as to further reduce the order of the realization.

Volterra Control Synthesis

307

Future work includes examination of the stability properties of VFS systems, and analytically examining performance properties of the higher order VFS realizations, such as disturbance rejection and sensitivity to unmodeled dynamics. Extension to classes of systems other than linear analytic is also possible.

Acknowledgments This work received financial support from the University of Notre Dame Arthur J. Schmitt Fellowship, the Jesse H. Jones Research Development Fund, the Structural Dynamics and Control/ Earthquake Engineering Laboratory under the National Science Foundation Grant CMS95-28083, the Center for Applied Mathematics, the Clark-Hurth Equipment Company, the Department of Electrical Engineering and the Frank M. Freimann Chair in Electrical Engineering. Sincere thanks and appreciation to Dr. Michael K. Sain for his encouragement, perseverance, guidance and constructive criticisms during the development of this work. Congratulations on your 70th birthday and a long, successful and distinguished career. God’s blessings and best wishes in the years to come.

References [AB86]

[ABS85]

[ABS86]

[ABS89]

[AS81]

[AS83]

[AS84a]

Samir A. Al-Baiyat. Nonlinear Feedback Synthesis: A Volterra Approach. Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 1986. Samir A. Al-Baiyat and Michael K. Sain. An application of Volterra series to the design of nonlinear feedback controls. In Proceedings of the 23rd Allerton Conference on Communication, Control and Computing, pages 103–112, Urbana, Illinois, Oct. 1985. University of Illinois at Urbana-Champaign. Samir A. Al-Baiyat and Michael K. Sain. Control design with transfer functions associated to higher-order Volterra kernels. In Proceedings of the 25th IEEE Conference on Decision and Control, pages 1306–1311, 1986. Samir A. Al-Baiyat and Michael K. Sain. A Volterra method for nonlinear control design. Preprints of the IFAC Symposium on Nonlinear Control System Design, pages 76–81, June 1989. Panos J. Antsaklis and Michael K. Sain. Unity feedback compensation of unstable plants. In Proceedings of the 20th IEEE Conference on Decision and Control, pages 305–308, Dec. 1981. Panos J. Antsaklis and Michael K. Sain. Feedback controller parameterizations: Causality and hidden modes. In Proceedings of the Sixth International Symposium on Measurement and Control, pages 437–440. International Association of Science and Technology for Development, Aug. 1983. Panos J. Antsaklis and Michael K. Sain. Feedback controller parameterizations: Finite hidden modes and causality. In S. G. Tzafestas, editor, Multivariable Control: New Concepts and Tools, pages 85–104. D. Reidel, Dordrecht, Holland, 1984.

308 [AS84b]

Patrick M. Sain

Panos J. Antsaklis and Michael K. Sain. Feedback synthesis with two degrees of freedom: {G,H,P} controller. In Proceedings of the Ninth Triennial World Congress of the International Federation of Automatic Control, volume IX, pages 5–10, July 1984. [BEG74] J. J. Bussgang, L. Ehrman, and J. Graham. Analysis of nonlinear systems with multiple inputs. Proceedings of the IEEE, 62(8):1088–1119, Aug. 1974. [BR71] E. Bedrosian and S. O. Rice. The output properties of Volterra systems (nonlinear systems with memory) driven by harmonic and Gaussian inputs. Proceedings of the IEEE, 59(12):1688–1707, Dec. 1971. [CI84] P. E. Crouch and M. Irving. On finite Volterra series which admit Hamiltonian realizations. Mathematical Systems Theory, 17:825–830, 1984. [DSW83] Kenneth P. Dudek, Michael K. Sain, and Bostwick F. Wyman. Module considerations for feedback synthesis of sensitivity comparisons. In Proceedings of the 21st Allerton Conference on Communication, Control and Computing, pages 115–124. Department of Electrical Engineering and the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Oct. 1983. [Dud84] Kenneth P. Dudek. The Total Synthesis Problem for Linear Multivariable Systems with Disturbances. Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 1984. [Gej80] R. R. Gejji. On the Total Synthesis Problem of Linear Multivariable Control. Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 1980. [IOP95] Francis J. Doyle III, Babatunde A. Ogunnaike, and Ronald K. Pearson. Nonlinear model-based control using second-order Volterra models. Automatica, 31(5):697–714, May 1995. [LPS79] R. J. Leake, Joseph L. Peczkowski, and Michael K. Sain. Step trackable linear multivariable plants. International Journal of Control, 30(6):1013–1022, Dec. 1979. [Pec79] Joseph L. Peczkowski. Multivariable synthesis with transfer functions. In Proceeedings of the NASA Propulsion Controls Symposium, pages 111–128, May 1979. [Pec80] Joseph L. Peczkowski. Total multivariable synthesis with transfer functions. In Proceedings of the Bendix Controls and Control Theory Symposium, pages 107–126, South Bend, Indiana, Apr. 1980. [PMSBFS97] Patrick M. Sain, Michael K. Sain, and Billie F. Spencer, Jr. Volterra feedback synthesis: A systematic algorithm for simplified Volterra controller design and realization. In Thirty-Fifth Annual Allerton Conference on Communication, Control and Computing, pages 1053–1062, Monticello, Illinois, September 1997. University of Illinois. [Por76] W. A. Porter. An overview of polynomic system theory. Proceedings of the IEEE, 64(1):18–23, Jan. 1976. [PS78] Joseph L. Peczkowski and Michael K. Sain. Linear multivariable synthesis with transfer functions. In Michael K. Sain, Joseph L. Peczkowski, and James L. Melsa, editors, Alternatives for Linear Multivariable Control, pages 71–87. National Engineering Consortium, 1978. [PS80a] Joseph L. Peczkowski and Michael K. Sain. Control design with transfer functions: An application illustration. In Proceedings of the Twenty-Third Midwest Symposium on Circuits and Systems, pages 47–52, 1980.

Volterra Control Synthesis [PS80b]

[PS81]

[PS84]

[PS85]

[PSL79]

[Rug83a]

[Rug83b] [Sai78]

[Sai97]

[Sai05] [SAW+ 81]

[Sch65] [Sch80] [SM80]

[SP81]

[SP82]

[SP85]

309

Joseph L. Peczkowski and S. A. Stopher. Nonlinear multivariable synthesis with transfer functions. In Proceedings of the 1980 Joint Automatic Control Conference, Aug. 1980. Paper WA-8D. Joseph L. Peczkowski and Michael K. Sain. Scheduled nonlinear control design for a turbojet engine. In Proceedings of the IEEE International Symposium on Circuits and Systems, pages 248–251, Apr. 1981. Joseph L. Peczkowski and Michael K. Sain. Design of nonlinear multivariable feedback controls by total synthesis. In Proceedings of the 1984 American Control Conference, pages 688–697, 1984. Joseph L. Peczkowski and Michael K. Sain. Synthesis of system responses: A nonlinear multivariable control design approach. In Proceedings of the 1985 American Control Conference, pages 1322–1329, June 1985. Joseph L. Peczkowski, Michael K. Sain, and R. J. Leake. Multivariable synthesis with inverses. In Proceedings of the Eighteenth Joint Automatic Control Conference, pages 375–380, June 1979. Wilson J. Rugh. A method for constructing minimal linear-analytic realizations for polynomial systems. IEEE Transactions on Automatic Control, AC28(11):1036–1043, Nov. 1983. Wilson J. Rugh. Nonlinear System Theory: The Volterra/Wiener Approach. The Johns Hopkins University Press, Baltimore, Maryland, Nov. 1983. Michael K. Sain. The theme problem. In Michael K. Sain, Joseph L. Peczkowski, and James L. Melsa, editors, Alternatives for Linear Multivariable Control, pages 20–30. National Engineering Consortium, 1978. Patrick Sain. Volterra Control Synthesis, Hysteresis Models and Magnetorheological Structure Protection. Ph.D. Dissertation, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 1997. Patrick M. Sain. Nonlinear input/output control: Volterra synthesis. In WaiKai Chen, editor, The Electrical Engineering Handbook, chapter Controls and Systems, pages 1131–1138. Elsevier Academic Press, Amsterdam, 2005. Michael K. Sain, Panos J. Antsaklis, Bostwick F. Wyman, R. R. Gejji, and Joseph L. Peczkowski. The total synthesis problem of linear multivariable control, part II: Unity feedback and the design morphism. In Proceedings of the 20th IEEE Conference on Decision and Control, pages 875–884, Dec. 1981. Martin Schetzen. Synthesis of a class of nonlinear systems. International Journal of Control, pages 401–414, 1965. Martin Schetzen. The Volterra and Wiener Theories of Nonlinear Systems. Wiley Interscience, New York, 1980. Michael K. Sain and Abraham Ma. Multivariable synthesis with reduced comparison sensitivity. In Proceedings of the 1980 Joint Automatic Control Conference, Aug. 1980. Paper WP-8B. Michael K. Sain and Joseph L. Peczkowski. An approach to robust nonlinear control design. In Proceedings of the Twentieth Joint Automatic Control Conference, June 1981. Paper FA-3D. Michael K. Sain and Joseph L. Peczkowski. Nonlinear multivariable design by total synthesis. In Proceedings of the 1982 American Control Conference, pages 252–260, June 1982. Michael K. Sain and Joseph L. Peczkowski. Nonlinear control by coordinated feedback synthesis with gas turbine applications. In Proceedings of the 1985 American Control Conference, pages 1121–1128, June 1985.

310

Patrick M. Sain

[SS82]

[SS84]

[SSD80]

[SWG+ 81]

[SY82]

Michael K. Sain and R. Michael Schafer. A computer-assisted approach to total feedback synthesis. In Proceedings of the 1982 American Control Conference, pages 195–196, June 1982. R. Michael Schafer and Michael K. Sain. Computer aided design package for the total synthesis problem. In Proceedings of the Ninth Triennial World Congress of the International Federation of Automatic Control, volume VIII, pages 179–184, July 1984. Michael K. Sain, R. M. Schafer, and K. P. Dudek. An application of total synthesis to robust coupled design. In Proceedings of the 18th Allerton Conference on Communication, Control and Computing, pages 386–395. Department of Electrical Engineering and the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Oct. 1980. Michael K. Sain, Bostwick F. Wyman, R. R. Gejji, Panos J. Antsaklis, and Joseph L. Peczkowski. The total synthesis problem of linear multivariable control, part I: Nominal design. In Proceedings of the Twentieth Joint Automatic Control Conference, June 1981. Paper WP-4A. Michael K. Sain and Stephen Yurkovich. Controller scheduling: A possible algebraic viewpoint. In Proceedings of the 1982 American Control Conference, pages 261–269, June 1982.

Part IV

Engineering Education

The First Professional Degree: Master of Engineering? Peter Dorato Department of Electrical and Computer Engineering, MSC01 1100, University of New Mexico, Albuquerque, NM 87131-0001, USA. [email protected]

Summary. This chapter deals with the issue of the first professional degree in engineering in the United States of America (USA). At the present time, the bachelor’s degree is commonly recognized as the first professional degree. However, there is increasing recognition that a 4year bachelor’s degree is no longer sufficient to properly prepare an engineering professional. The US National Academy of Engineering recommends that the 4-year bachelor’s degree be a pre-engineering program, and that a master’s degree be the first professional degree. This chapter explores a practice-oriented Master of Engineering degree, as a candidate for the first professional (ABET accredited) engineering degree.

1 Introduction The issue of the first professional degree in engineering has been a topic of discussion for some time. Books have been written on the subject, including Pletta’s monograph, The Engineering Profession [Pl84], and the National Society of Professional Engineers’ report, Challenge for the Future...Professional Schools of Engineering [NSPE76], published in in 1984 and 1976, respectively. A generally accepted definition of the first professional degree involves the following components: – Some period of pre-professional preparation, – A period of broad professional education focused on the preparation for professional practice, leading to the first professional degree, – The first professional degree being the only accredited degree (for example, for engineering the only ABET-accredited degree), – The first professional degree being the only degree recognized for professional licensing. The US Department of Education officially recognizes a degree as a first professional degree only if the degree program requires at least a cycle of 2 years of pre-professional preparation, and a total (including the pre-professional period) of 6

C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 15, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

314

Peter Dorato

years. First professional degrees in Law, Medicine, and Pharmacy require cycles of 4/3, 4/4, and 2/4, respectively, where the first number represents a pre-professional period in years, and the second number represents years required for the professional degree (J.D., M.D., and Pharm. D., respectively). Law and Medicine easily meet the requirements of professional recognition, while Pharmacy just meets the minimal requirements. Bachelor degrees in engineering are normally accredited and required for licensing, however the 4-year degree, with no university-level pre-engineering requirements, falls far short of the minimum 6 years required by the US Department of Education for recognition as professional education. In Reference [FrDo87] a case is made for the Doctor of Engineering (Eng. D.) as, the first professional degree, with a 4-year bachelor’s degree as pre-engineering, and a 3-year doctor’s degree as professional degree, i.e. a 4/3 cycle. While there is a growing consensus that the 4-year bachelor’s degree is no longer sufficient as a first professional degree in engineering, extending the education period to 7 years appears too large a step to take at this time. For this reason we explore in this article the possibility of a practice-oriented master’s degree as the first professional degree. The Master of Engineering (M. Eng.) appears to be a suitable candidate. It is discussed in more detail in Section 5.

2 A Short History of Engineering Education in the USA Engineering education in the USA was initiated at the United States Military Academy at West Point in 1802. The curriculum was modelled after the French ´ Ecole Polytechnique, and included a strong foundation in mathematics and science. The first engineering degree from West Point was granted in 1833. In 1824 the Rensselaer School was founded in Troy, New York. This school, eventually called the Rensselaer Polytechnic Institute (RPI), offered the first civilian engineering program in the USA in 1849. In time, engineering education in the USA moved away from the French model of engineering education, with its rigorous foundations in mathematics and science, to the English model with its emphasis on engineering practice. This emphasis on engineering practice lasted until the end of World War II, with the addition in the 1930s of humanities and social sciences to the engineering bachelor’s degree curriculum. In the 1950s a typical bachelor’s degree in engineering required 140 semester hours of course work. With the strong competition in space and missile design from the USSR that developed after World War II, recommendations (see the Grinter report [Gr55]) were made to add more mathematics and basic science to the engineering curriculum. Also during World War II, the major technological developments such as atomic energy, radar, missiles, etc., were accomplished mostly by mathematicians and scientists, rather than engineers. This is generally attributed to the limited amount of mathematics and basic science included in engineering programs at the time. Thus, in the late 1950s the amount of mathematics and science in bachelor’s degree programs was significantly increased. Near the same time, the number of semester hours for the bachelor’s degree was reduced (commonly to about

The First Professional Degree: Master of Engineering?

315

130 semester hours). To accommodate the additional math and science within the 130 hour bachelor’s degree, the amount of time devoted to engineering practice was largely reduced. This resulted in 4-year institutions of the USA with Engineering Technology programs, to put emphasis on practice rather than theory. In the article of Cranch and Nordby [CrNo86], published in 1986, arguments were made for the importance of adding courses in communications (both oral and written) and management to the engineering curriculum. A major dilemma faced by engineering education at the present time is how to accommodate all the additions that have been been recommended over the years within a 4-year bachelor’s engineering program. The national tendency appears to be to make the bachelor’s degree highly focused. In the USA, we have been structuring our engineering programs along the broad areas of Civil Engineering, Electrical Engineering, Mechanical Engineering, etc. However, outside the USA, engineering degrees can be very focused, for example engineering degrees are offered in Europe in such specialized areas as Systems and Control, Communications, Electrical Power, etc. Given the current trend towards specialization, it may become necessary to rename our degree programs, otherwise we may be offering degrees in Electrical Engineering to graduates who have never taken a single course in electromagnetics. In summary, at the present time the 4-year bachelor’s degree is the first professional degree in engineering in the USA, and there is a strong trend to make the bachelor’s degree more and more focused on a given specialty.

3 Limitations of the Bachelor’s Degree as First Professional Degree In 1918, when the Mann report [Ma18] on 4-year engineering education was published, the report stated, “There is almost unanimous agreement among schools, parents, and practicing engineers that at present the engineering curriculum, whatever its organization, is congested beyond endurance.” Now, with the expansion in knowledge required in mathematics, science, computers, social science/humanities, engineering science, and engineering design, the 4-year bachelor’s degree has become a serious problem for the education of a practicing engineer. In spite of all the new material that the modern engineers need to learn, developments have occurred at the bachelor’s level in which the time available for technical education is reduced. Some of the changes are as follows. – Many universities are mandating that all bachelor’s degrees be limited to 120 semester hours. – Many universities are mandating a “University Core” for all bachelor’s degrees. This core commonly goes beyond the social science/humanities that have been required in engineering programs by the Accreditation Board for Engineering Technology (ABET) in the past. – Because of poor preparation in mathematics and science at the high school level, many engineering programs have to include more mathematics and science courses than in the past.

316

Peter Dorato

There is a national trend in the USA to make the bachelor’s degree in engineering a very narrowly focused program. Many engineering science courses not related to a very specialized area have been dropped. For example, at some universities one can now get a Bachelor of Science (BS) degree in Electrical Engineering without taking any course in electromagnetics. This focused specialization does solve some of the limitations listed above, but it does not provide the broad education generally expected from a first professional degree. Also, the narrowly focused bachelor’s degree often does not include a broad enough preparation for the Fundamentals of Engineering exam required for professional licensing. As noted in the Summary, the National Academy of Engineering has recommended, see [NAE05], that the first professional degree in engineering be expanded beyond the bachelors.

4 Trends in Engineering Education Outside the USA: The Bologna Process In 1999, twenty-nine European ministers of higher education met in Bologna, Italy, to start plans for a unified program of higher education, which included a change to a two-cycle (undergraduate/graduate) education process. The first cycle lasts 3 years, leading to something equivalent to our undergraduate bachelor’s degree, and the second cycle lasts 2 years, leading to something equivalent to our graduate master’s degree. This has become known as the Bologna Process. Prior to the Bolgona Process most engineering degrees in Europe required 5 years of university studies. The first professional engineering degree in Europe was thus a 5-year program. Law and Medicine excluded themselves from the Bologna Process, but Engineering did not, so that at the present time, the first engineering degree in Europe has been reduced from 5 years to 3 years. Recently Russia [Mac07] has started to move to a 3/2 cycle for higher education. However Engineering, along with Medicine, has elected to be excluded from the process, and in Russia the 5-year engineering program will continue to be the first professional degree in Engineering. The Bologna Process has met some opposition, but a majority of European countries are now are on track with the process. This raises the question as to what now constitutes the first professional engineering degree in Europe. Some countries, especially in northern Europe, consider the first 3-year cycle as “pre-engineering” and the second 2-year cycle as the first professional engineering degree. But the picture is confusing, and many feel that the Bologna Process has hurt the status of engineering in Europe. In the United Kingdom the first degree in engineering has been the 3-year bachelor’s degree for some time. In many countries that have followed the old English system, e.g. China, India, Canada, etc., the first professional engineering degree has been the 4-year bachelor’s degree, as currently in the USA. In Mexico and most countries of South America, engineering education requires 5 years of university studies, like the old European system. There has been some discussion of the Bologna Process in South America, but no action in this direction has been taken so far.

The First Professional Degree: Master of Engineering?

317

5 The Master of Engineering Degree The M.Eng. degree is commonly recognized as a practice-oriented graduate degree, in contrast to the M.S. (Master of Science) degree, which focuses on theory and research preparation. Currently, about 30% of schools in the USA offering M.S. degrees also offer M.Eng. degrees. Typically, M.Eng. degrees require a design project instead of a thesis, and a course-work focus on design courses rather than theory courses. In 1992, the Massachusetts Institute of Technology (MIT) initiated an M.Eng. degree, which was viewed as a first professional degree for students in Electrical Engineering and Computer Science. However, the bachelor’s degree remained the accredited degree, so that the M.Eng. degree could not be viewed as the first professional degree, as normally defined. One school in the country which has used the M.Eng. as a first professional degree, with ABET accreditation, is Louisville University. The M.Eng. program at Louisville University is based on 2 years of preengineering, followed by 3 years of engineering studies, i.e. a 2/3 cycle. Many cycles are possible for the M.Eng. as a first professional degree, e.g. 3/2, 2/3, 4/1. However, the first pre-engineering cycle is where most of the mathematics, science, and non-technical courses would best fit. The 4/1 cycle would need to include engineering science and some engineering design courses in the 4-year preengineering program.

6 Conclusions It will probably take some time for the M.Eng. to be recognized as a first professional degree. It will require the consensus and support of a number of national agencies, such as the Engineering Deans Council, National Society of Professional Engineers, Accreditation Board for Engineering and Technology, professional engineering societies, etc. In the meantime, given the need to expand engineering education beyond the 4-year bachelor’s degree, it would be useful if the M.Eng. were instituted in more engineering schools offering M.S. degree programs. With a critical mass of M.Eng. degree programs in the country, one can then move to its establishment as the first professional degree. At some more distant future, as education for the engineer becomes more and more demanding, the Eng.D. could become the first professional degree for engineering, and engineering education can join Law, Medicine, and Pharmacy as officially recognized professional educations. Extending engineering education from 4 to 5 years (M.Eng.), or from 4 to 7 years (Eng.D.), will, of course, add to the cost of engineering education. However, the additional education should improve the professional preparation of graduates, and hopefully yield higher salaries and longer professional lifetimes. Higher salaries may also be acceptable to industry, if graduates are truly better prepared.

318

Peter Dorato

References [ASCE98] American Society of Civil Engineers, ASCE Policy Statement No. 465, American Society of Civil Engineers, New York, NY, 1998. [CrNo86] E. T. Cranch and G. M. Nordby, Engineering Education at the Cross-roads Without a Compass, Engineering Education, Vol. 76, pp. 742–747, 1986. [FrDo87] B. Friedland and P. Dorato, A Case for the Doctor of Engineering as a First Professional Degree, Engineering Education, Vol. 77, pp. 707–713, 1987. [Gr93] Lawrence P. Grayson, The Making of An Engineer, John Wiley & Sons, New York, 1993. [Gr55] L. E. Grinter, Report on the Evaluation of Engineering Education, Engineering Education, Vol. 46, pp. 25–63, 1955. [NAE05] National Academy of Engineering, Educating the Engineer of 2020, The National Academies Press, Washington, DC, 2005. [Mac07] Byron Macwilliams, Russia Moves Toward 2-Tiered Degree System, Phasing Out 5-Year Cycle, The Chronicle of Higher Education, p. A38, March 23, 2007. [Ma18] Charles Riborg Mann, A Study of Engineering Education, Carnegie Foundation for the Advancement of Teaching, New York, NY, 1918. [NSPE76] National Society of Professional Engineers, Challenge for the Future...Professional Schools of Engineering, National Society of Professional Engineers, Washington, DC, 1976. [Pl84] Dan H. Pletta, The Engineering Profession, University Press of America, Lanham, MD, 1984.

Theology and Engineering: A Conversation in Two Languages Barbara K. Sain University of St. Thomas, 2115 Summit Ave., Saint Paul, MN 55105, USA [email protected]

Summary. Engineering and theology are not commonly perceived as conversation partners. At most American universities that offer these programs, engineering is grouped with technical or scientific disciplines and theology is classified with the liberal arts. Questions about how they relate to each other are subsumed into larger discussions of the integration of technical education with the liberal arts. Differences in method and content among the disciplines can make the challenge of integration seem difficult. Drawing on recent research by the author and Michael Sain, this essay shows that engineering and theology are not as different as might be surmised. In fact, some methods of analysis used in engineering can be applied to theological issues with great nuance. Similarities in method between the two disciplines allow for sophisticated mutual conversation. For example, both engineers and theologians study decision making in the presence of accepted principles (or constraints) and variables. There is also a similarity between the use of analogy in theology and the role of modeling in engineering. To demonstrate the application of engineering theory to theology, this chapter examines Saint Augustine’s struggle to convert to Christianity, made famous by his autobiography Confessions. Augustine’s will seemed to be divided against itself, influenced by a habit based on sensual experience, and unable to implement the decision of his intellect until he received assistance from God. The complex interplay of the intellect, will, senses, and “exogenous signals” from God can be analyzed in terms of classical feedback control theory. By opening a conversation between two disciplines not usually associated with each other, this research facilitates the integration of technical education with liberal arts education in the university. From the perspective of theology, it can also be seen as part of a longstanding conversation about the integration of faith and reason in human life.

1 Introduction “What indeed has Athens to do with Jerusalem? What concord is there between the Academy and the Church?” [Ter1885] These questions, posed by the North African theologian Tertullian early in the third century, have become one of the most famous quotations from early Christian theology, often cited in discussions of how theology relates to philosophy, science, or other forms of human knowledge. The important issue in Tertullian’s mind was the relationship between the philosophical wisdom of the ancient world (represented by Athens) and the faith of Christians (represented C.-H. Won et al. (eds.), Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics, DOI: 10.1007/978-0-8176-4795-7 16, © Birkh¨auser Boston, a part of Springer Science+Business Media, LLC 2008

320

Barbara K. Sain

by Jerusalem). Although Tertullian did not object in principle to philosophy, he was concerned that Christian beliefs could be distorted by worldviews that were in some ways incompatible with Christianity [Hel94]. At the time he was writing, Christianity was coming into its own as a religion in the midst of the very diverse Roman empire. Some Christians saw their faith as testimony to a radical reversal of the existing world order: a new way of living and thinking that showed how dramatically human history had been changed by Christ. The education and expectations of traditional society held little interest for them. Other Christians, while no less serious in their beliefs about Christ, saw the classical wisdom of the world as seeds of the truth that was revealed in Christ, that is, as first steps toward the fullness of wisdom that could be found in Christianity. At that time, Tertullian’s voice was one among many discussing how Christian faith could relate to other sources of human wisdom. In the centuries since Tertullian, the image of Athens and Jerusalem has risen above its original context to become a metaphor for the relation of human reason and faith. Can insights from non-Christian sources be used to enrich Christian understanding of God and the world? Does the knowledge of revealed truth offered by Christian theology shed a different light on the world than the light of natural reason? Does faith itself make possible a deeper understanding of the world and human beings, or are faith and reason better off tending to their own areas of expertise? Such questions have been present in various forms throughout the history of Christian theology. In the early centuries of Christianity, these issues arose as Christians interacted with the various religions and cultures of the Roman empire. Later, in the high Middle Ages, European Christian theologians debated whether incorporating ideas from newly rediscovered writings of Aristotle into Christian theology would enhance or distort the comprehensive Christian vision they had developed. In recent centuries the rise of modern science with its distinct empirical method has resulted in debates, both friendly and polemical, about the relation of religion and science. In 1998 Pope John Paul II added his voice to the ongoing discussion with the publication of his encyclical Fides et Ratio (Faith and Reason). In that work he encouraged Christians to make full use of all modes through which human beings discover truth about themselves and the world. He focused particularly on the value of philosophy and its relation to Christian theology. The voices cited above come from such different circumstances that it may seem an exaggeration to say that they are participating in the same conversation. Indeed, scholars of both antiquity and modernity would be quick to say that the issues of faith and reason faced by Tertullian in the third century were very different from the issues that John Paul II addressed in Fides et Ratio. While that is true, it can also be said that some common threads run through this long conversation and are present in today’s discussions of the role faith plays in our lives, our society, and even our universities. In the context of a university the question of the relation of reason and faith often emerges in discussions of how the religious identity of a university affects its specific goals and activities or how the core curriculum is integrated with professional programs. If reason and faith can hope for nothing more than an uneasy alliance, then assuming or attempting too close of an integration between theology and other academic disciplines might result in the limitation of those disciplines

Theology and Engineering: A Conversation in Two Languages

321

or a distortion of the truths of faith. However, if reason and faith are meant to be integrated in the human person then creative cooperation among disciplines in the university might result in new insights and the mutual enrichment of disciplines. The challenges faced in such collaborations vary with the disciplines involved. Some disciplines have similar methods and vocabulary and may even be accustomed to addressing similar topics. Other disciplines have such different approaches that collaboration requires explicit consideration of each discipline’s presuppositions and method. The next two sections of this essay will present one example of interdisciplinary research: the relation of theology and engineering as it has been developed in the recent work of Michael Sain. Michael Sain and I have worked together preparing a course that uses the language of feedback system theory to explore traditional theological topics such as moral decision making and the relation of human beings to God. The project grew out of discussions of Saint Augustine’s autobiography, Confessions, and a scene from that work will be used to demonstrate the combined application of ideas from theology and engineering. The final sections of the essay will discuss the importance of analogy for such a project and return to the issues of faith and reason.

2 Theology and Engineering What does the discipline of engineering have to do with the life of faith or, more specifically, with the tradition of Christian theology that articulates the truths of that faith? The first response from some engineers and theologians would likely be that the two areas of study have very little to do with each other. Engineers deal with the natural world, creating designs that solve human problems or improve the quality of human life. Theologians deal with God, who transcends the natural world, and with faith, which by definition involves relation with the unknown. To the extent that engineers build on the empirical method of the natural sciences and rely on known characteristics of the natural world, it may seem that their method has little in common with theology, a discipline whose main objects cannot be clearly measured or tested. To the extent that theologians study God, faith, salvation, eternity, and other topics that are difficult to quantify, and do so in conversation with other theologians who have been dead for centuries (if not millennia), it may seem that theology has little in common with engineering, a discipline that expects clearly applicable results, produced reasonably on time and reasonably within budget. What indeed could engineering and theology have to do with each other? While there is some truth in these characterizations of engineering and theology and in the impression that they might not fit easily together, the descriptions do not do full justice to either discipline. There are similarities between the disciplines that reveal common ground and allow for sophisticated mutual discussions. For example, both engineering and theology are accustomed to making decisions in the presence of accepted principles (or constraints) and variables. For engineers the laws of nature and the characteristics of the materials always serve as constraints on the design. Other constraints can result from the purpose of the design or the limits of time and

322

Barbara K. Sain

money. Despite such constraints, there are typically many factors left undetermined that allow for creativity in the design process. There are also usually some features of the design, or of the situation in which the design will function, that the engineer cannot know in advance. Such uncertainty is a familiar characteristic of engineering design and can be addressed by allowing margins of error or perhaps including a control system within the design itself to make necessary adjustments. Theologians also study situations which combine accepted principles and uncertainty. This can be said of theology generally, but it is particularly true for moral theology, the subdiscipline that studies the virtues and choices involved in living out the Gospel. The collective wisdom of the Christian tradition has much to say about how to live a Christian life. Some principles have been widely accepted as norms, or laws, for Christian behavior and belief. The Ten Commandments are a famous example of such norms. However, in addition to accepted principles, there are many variables that affect human decisions. Some of these can be predicted in advance well enough that reasonable responses can be postulated and discussed. Other variables cannot be known ahead of time. Living a Christian life therefore requires the application of theological ideas to complex situations: multiple variables must usually be considered and more than one course of action may be available that is consistent with accepted moral principles. In some situations one choice will clearly be better than others, but it is often the case that several options promise desirable results. There is room for creativity and freedom in living the Christian life. The same uncertainty that makes human life interesting and challenging gives richness and complexity to the discipline of moral theology. There is a significant similarity between the methods of engineering design and Christian moral theology. Each discipline works with a combination of fixed principles and variables to achieve or work toward goals. For engineers the short-term goal is usually a design that meets certain specifications. Long-range goals may include the development of a body of theory or an innovative approach to a type of design problem. For Christians, the ultimate goal is a virtuous life followed by eternal life with God in heaven. Theologians provide explanations and guidelines based on the cumulative wisdom of the Church’s tradition. Sometimes that requires very practical analysis, not unlike the analysis needed for a specific design problem in engineering. At other times it means contributing to a body of theory. To continue exploring the relation of theology and engineering, let’s examine a scene from one of the classics of Christian theology, the Confessions of Saint Augustine.

3 The Divided Will of Saint Augustine Augustine lived in the fourth and fifth centuries. By the end of his life he had gained renown as a brilliant Christian teacher, writer, and bishop. However, early in life he trod a rather different path. Disdaining the Christian faith that his mother Monica tried to impart to him, he sought pleasure and truth in other sources. A classical education and outstanding natural abilities allowed Augustine to establish himself as a teacher of rhetoric. However, he wanted to do more than use language eloquently

Theology and Engineering: A Conversation in Two Languages

323

and persuasively. Augustine was a relentless seeker with nuanced questions about the nature of God, the cause of evil, and the role of human beings in the cosmos. He would not commit himself to any religion or philosophy that could not provide satisfactory answers to his questions. The very sophistication of his questions, which later led Augustine to be one of the most influential thinkers in Western Christianity, also caused him to spend years searching for adequate answers. He found none, until he moved to Italy and met Ambrose, the Christian bishop of Milan. Through Ambrose Augustine realized that Christianity was richer and more sophisticated than he had perceived in his youth. He discovered an intellectual conversation within Christian theology that gave him convincing answers to his most perplexing questions. Augustine also learned about the monastic life lived by Anthony of Egypt and other Christian monks. Inspired by the radical dedication of that lifestyle, Augustine realized that he wanted to become a Christian monk. He had at last found an intellectual and spiritual home. Augustine was surprised to find that choosing the life he now desired was difficult for him. In Christianity he had discovered convincing answers to his most profound questions, and in the life of the Christian monks he saw a simplicity and dedication for which he longed. Yet choosing the life of a monk would mean walking away from the pleasures of female companionship. After a youth of sexual adventures, Augustine had settled into a fifteen-year monogamous relationship with a woman for whom he cared deeply. When he was in his early thirties, however, he was persuaded by his mother that he should marry. He separated from his concubine and entered into an engagement with a young woman of equal social standing. Augustine had been accustomed to the physical companionship of a woman and was planning for the continuation of such intimacy in married life. It was at this point that he began to desire the celibate life of a monk. Augustine was in an unexpected dilemma. He had finally found intellectual satisfaction and he was strongly attracted to monastic life, but he found that he did not have the strength to be baptized and embrace that new life. There was more needed for conversion than intellectual conviction. In chapter eight of his autobiography, Augustine describes the struggle that ensued in him. The climax came one day in Milan. Overcome with anguish and frustration at his own indecision, Augustine retreated to the garden of the home where he was staying. There he observed that, although he was able to move his hands, pull his hair, and do other physical actions, he was not able to will himself to choose the Christian life that he wanted. On that point it seemed as if he had two wills inside him or as if his will was divided against itself. “The mind orders itself to make an act of will, and it would not give this order unless it willed to do so; yet it does not carry out its own command. But it does not fully will to do this thing and therefore its orders are not fully given. It gives the order only in so far as it wills, and in so far as it does not will the order is not carried out. The reason, then, why the command is not obeyed is that it is not given with the full will. So there are two wills in us, because neither by itself is the whole will, and each possesses what the other lacks.” [Aug61]

324

Barbara K. Sain

What does it mean for a will to be divided against itself? Can a person truly want and not want the same thing? Such division, or at least the feeling of being so divided, is not unfamiliar for human beings. We can decide upon a new course of action and then find that we prefer to follow an old habit. We sometimes struggle with ourselves at crucial moments of decision, when the resolve for a renewed life finds itself sorely tested against the attraction of familiar patterns of past choices. When we struggle with ourselves in this way, what exactly is happening? Is the will refusing the direction of the intellect? Is the will actually struggling with itself? Are the senses or the emotions affecting the process? In Augustine’s case, we can answer some of these questions with the assistance of his own analysis. He explains to readers of the Confessions that years of following an undisciplined will had led to a habit so strong that it seemed to be a necessity in his life. Although he had grown weary of the demands of his career and genuinely longed for a celibate life, Augustine could not overcome his old habit. He felt as if he were bound by the chains of his own will, which was divided against itself. Augustine’s struggle with himself in the garden in Milan is one of the most famous stories in Christian literature and has been analyzed extensively by theologians. I suggest that we examine it here using the language of feedback system theory. Consider the following model of the will depicted in Figure 1. The model shows the input of a decision, which would be coming from the intellect. The output is the action that results when the will carries out the decision. Within the will itself, there is a feedback loop that affects the value of the output of the will. In the feedback paradigm being applied here (unity negative feedback), the value of k k affects the output according to the following equation: awill = 1+k dintellect . When k has a large positive value, the gain of the loop within the model is high and the performance of the model is quite stable and predictable. The relationship between the input and the output cannot be altered in a significant way unless there is a dramatic change somewhere in the system. The will can be said to be strong. Now consider a second model of the will given in Figure 2.

Will

∑

k –

decision

Fig. 1. Model of the will.

action

Theology and Engineering: A Conversation in Two Languages

325

Fig. 2. Second model of the will.

In this model there are two loops within the will. One loop gain has the value k1 k. The other has the value k2 k. The value of the output now depends upon the relative values of k1 k and k2 k. In such a situation, with two loops present, it is possible for one loop to dominate the other. If the loops represent different objects of the will, then the model shows a will that is “battling” against itself. When one loop represents a habit and the other loop represents a new desire for change, the final output will be determined by the relative values of the loop gains. If the old habit is stronger than the new resolution, the change in behavior will be small or non-existent. If the new desire for change is stronger, it will overcome the old habit. Augustine found that his heartfelt desire to embrace a monastic vocation was simply not strong enough to overcome his longstanding habit. The loop gain of desire for conversion was strong enough to alter the output in a noticeable way: it reduced Augustine to an agony of indecision and frustration. However, it was not strong enough to produce the desired output of conversion. One might ask about the role of Augustine’s intellect in this process. Perhaps the problem lay more in the strength of the decision from the intellect than in the will itself. If Augustine was not truly convinced that monastic life would bring him deeper fulfillment than the life of a married Christian, then perhaps the input of the intellect to the will was too weak to produce change in the will. Such situations are familiar in human life. To examine this possibility using feedback theory it is necessary to have a model that shows the relation of the intellect and the will, such as Figure 3. A careful discussion would also consider the role of the senses and physical desires. Augustine was struggling with a habit based on physical experience. To model such a habit accurately, some consideration should be given to the role of the body with its senses and desires. Although it will not be shown here, these models can be expanded to show relations of the intellect and will with physical senses and appetites. Fortunately for Augustine, his story did not end with the agonizing impasse of a divided will. In Confessions Augustine explains that the impetus to overcome the

326

Barbara K. Sain

Will ∑

dintellect

k

awill

– k3d

gGod

∑

– Intellect Fig. 3. Third model of the will.

problem was provided by the grace of God. He describes how, in the middle of his anguish in the garden, he was prompted to read a copy of St. Paul’s letter to the Romans that was lying nearby. Reading the inspired text, which seemed to speak directly to his situation, brought lasting peace to Augustine’s heart. His attachments dissipated, and he found himself joyfully, confidently ready to choose the life of a Christian monk. Augustine had no doubt that he owed his final discovery of peace and truth to the grace of God. In fact, his autobiography is not fundamentally a confession of his own sins (although it is that). It is primarily a grateful confession of how God worked in his life. To return to our model of the process, this means that the relative balance of the loop gains within the will was changed by input from God. Modeling this requires considerable theological subtlety. How can God’s input be represented when God is fundamentally different than anything else in the model? God is not part of the world, yet Christianity does claim that God acts in the world to guide, assist, and strengthen human beings. God is at once more foreign and more intimate to us than anything in the world. To the extent that God’s assistance is truly external to the human being, it can be fairly represented as an exogenous signal, entering the system of the will in a way that affects the competing loop gains. However, to the extent that God works within human beings, assisting them in a way that enhances rather than reduces their freedom, God can be represented within the system. The solution that has been adopted in Figure 4 is to show input from God at multiple sources. God may give input to the intellect, the will, or both. God may also act in other ways, represented by aGod , that affect the final output without directly affecting the intellect or the will. The action of God could also be represented by loops at various places within the system.

Theology and Engineering: A Conversation in Two Languages

dGod

∑

327

aGod

k

awill

∑

a

– Will

dintellect k3d Intellect e gGod

∑

–

y

a aGod awill dGod dintellect e gGod y

total action God’saction will’s action God’s “decision” intellect’s decision error God’s goals observations

Fig. 4. Fourth model of the will.

At this point the model has become so complex that a much longer essay would be needed to explain the various ways that it can be used to analyze human experience. Readers familiar with feedback system theory may have already seen interesting applications. One simple example was presented above: how the output of the will model depends on the relation of the two loop gains within the model. Similar analyses could be performed on the relation between the intellect and the will and on the relation of the intellect and the will to the body, with the possibility of multiple loops at many places in the system. The mathematics required to explore these relationships becomes quite complex, of course, so it is often advantageous to focus on a single relation or small set of relations, minimizing the effects of other factors for the purpose of analysis. Although Augustine explained his experience in the elegant prose of a fourth century rhetorician rather than in the diagrams and equations of feedback theory, it is possible to make an accurate and nuanced translation of his ideas. The remarkable level of self-awareness in Confessions is one reason that it has become a classic of Christian spiritual writing. Feedback theory provides a sophisticated set of tools for exploring the relations between the will, intellect, and the body that Augustine describes. Theological readers of this essay will likely have many questions in their minds about the nuances of the various relationships depicted, particularly about how divine grace is represented. It is at this point, perhaps, that the engineering diagrams might seem too simple and concrete, too mathematically precise, to do

328

Barbara K. Sain

justice to a discussion about human beings and God. However, as will be explained in the next section, the engineering models are capable of great sophistication and nuance, precisely because they are understood as models. If this discussion could be extended to great length, it would be possible to examine the characteristics of the various relations depicted in the earlier models and to go beyond the story of Augustine’s conversion to join the broader conversation in Christian theology about habits, grace, wisdom, love, and conversion. As it stands, this section offers a brief glimpse of the type of conversation that is possible.

4 The Importance of Analogy The preceding exploration of Augustine’s struggle with his divided will gives some indication of how engineering ideas can be used to model theological problems. There are a number of important methodological issues that must be addressed when theology and engineering are integrated in this way. One of the most important issues is the role of analogy in theology and its similarity to the use of models in engineering. One of the fundamental claims of the Judeo-Christian tradition is that God transcends the world. God is not one piece within the universe, subject to natural laws and the whims of larger forces or fates. Rather, God is independent of the world as we know it: the creator of the heavens, the earth, and all that they contain. Because God transcends the world, human language and experience are inadequate for describing and understanding God. It may seem in light of this that wise Christians would simply stand silent before the divine mystery. However, Christianity also claims that truth about God is revealed to human beings through the natural world, prophets, the Scriptures, and personal experiences of revelation. The greatest revelation, of course, was Jesus Christ: God incarnate as a human being. From all of these sources, human beings can gain true knowledge of God, and it is possible to speak of God with some accuracy in spite of the limitations of human language and experience. Christian theologians employ analogy as a methodological principle to maintain correct nuance when speaking about God. In everyday speech, analogy is a linguistic device used to express similarity between two things that are otherwise different. In Christian theology analogy is used in a carefully refined way to maintain the correct understanding of God in relation to the world. The genuine similarity between human experience and the truth about God which enables us to have knowledge of God always stands within the greater dissimilarity of the difference between God and creation. When we claim that God is wise, there is enough similarity between God’s wisdom and our experience of human wisdom that we have a true understanding of what it means to say that God is wise. At the same time, God’s wisdom transcends human wisdom and remains, to a certain extent, mysterious to us. Theologians are aware that their language for God is always analogous in this way. The conceptual and linguistic tool of analogy explains how the two Christian claims, that God transcends human experience and that we can have real understanding of God, can be reconciled.

Theology and Engineering: A Conversation in Two Languages

329

There are some similarities between the use of analogy in theology and the use of models in engineering. Engineers use diagrams and equations to describe real relations in the natural world. A certain degree of accuracy is necessary if the designs based on these ideas are going to function correctly when they are constructed. However, there is quite a bit of flexibility and creativity in how engineers use diagrams and equations as models to develop their ideas. Sometimes a simple model of a very complex system is the most helpful tool for studying one aspect of the system. At other times it may be necessary to model the whole system with as much detail as possible or to focus on a detailed model of a small portion of the system. This flexibility in how models are used in engineering shows a similarity of method between engineering and theology: theologians know that their language is always an analogy for the reality of God, and engineers know their diagrams and equations are models of the natural world. Of course, a distinction must be introduced immediately because the natural world cannot be compared directly to God. The transcendence of God introduces a distinction into theological method that is not present in engineering. However, the comparison is still valuable because both disciplines are aware that they are working with concepts that, although they are accurate descriptions of their objects, are not identical with those objects. Perhaps it could be said that engineering models are analogies for the natural world. This methodological similarity between theology and engineering gives their mutual engagement tremendous potential. Analogies can be drawn between ideas in the two disciplines that allow for discussion at a highly sophisticated level.

5 Theology and Engineering: A Conversation in Two Languages The introduction to this chapter set the relation of engineering and theology in the context of the long conversation in Western tradition about the relation of human reason and Christian faith. In the contemporary university, it may not be immediately evident how two disciplines with such different histories can understand each other and work together on the same problems. It may seem to some observers that engineering is a product of human reason, with practical applications that have little to do with theology, or that the subject matter of theology does not lend itself to rigorous analysis. The analysis of Augustine’s conversion in this essay shows, at least in the form of a sketch, how the principles of feedback system theory can be used to engage in a sophisticated theological analysis of an event that seems at first glance to have little to do with engineering. Such analysis is an exercise in translation. The richness of Augustine’s experience has been studied for centuries, first by Augustine himself and then by generations of Christians who have read Confessions. Other theologians have contributed different insights to the discussion about the will, intellect, senses, desires, and other components of the human person. There has also been extensive discussion of the relation of divine grace to human nature and free will. The many nuances of these debates are expressed in carefully defined theological terms. Describing the same issues in the language of engineering allows for a surprisingly rich mutual conversa-

330

Barbara K. Sain

tion between theology and engineering. Although great care is needed to ensure the accuracy of the translation when theological principles are expressed in engineering terms (and the principle of analogy must always be kept in mind), in this project feedback system theory has proven capable of expressing theological ideas with nuance and rigor equal to the traditional theological terms. A project such as this can also assist the integration of liberal arts education with technical education. For some engineering students, the translation of theological ideas into the language of engineering theory brings the fundamental issues and complexity of the theological discussions into clearer focus. Accustomed to recognizing and manipulating relationships expressed in equations and diagrams, they begin to see the implications of theological claims with greater nuance. For example, after analyzing part of Augustine’s conversion, students often go back to the text of Confessions and discover levels of sophistication in Augustine’s analysis that they did not recognize in their first reading. A reverse translation takes place: students realize that many of the insights they gained by applying feedback models are already present, with subtle analysis, in Augustine’s prose. The explicit comparison of theological method with engineering method that must accompany such a project also assists the students in integrating different facets of their education. Understanding similarities between theological method and engineering method can increase student appreciation for the sophistication of both disciplines and decrease the perception that liberal arts and professional disciplines operate in separate spheres with presuppositions that place them at odds with each other. Students can realize that the discipline of theology involves a complex interplay of faith and reason and that the methods of engineering and other technical disciplines have some common ground with the methods of the liberal arts. This research project that Michael Sain has tackled is no casual exploration of the relation between a professional career in engineering and a personal life of faith. Rather, this is an engagement of theory with theory at an advanced level. The diagrams and equations used in feedback theory to model the performance of the engineering designs provide a sophisticated language into which to translate the complex questions about human beings and God that interest theologians. If engineering belongs to the category of human reason, then this research joins the long conversation about the relation of reason and faith as evidence that Athens and Jerusalem do indeed have something to do with each other.

References [Ter1885] Tertullian, The Prescription against Heretics, in The Anti-Nicene Fathers, vol. 1, ed. Alexander Roberts and James Donaldson, trans. Peter Holmes (Buffalo, N.Y.: Christian Literature, 1885), 246. [Hel94] Wendy E. Helleman, On the meaning of Athens and Jerusalem in Tertullian’s work and in later interpretation, see Wendy E. Helleman, Tertullian on Athens and Jerusalem, in Hellenization Revisited: Shaping a Christian Response within the Greco-Roman World (New York: University Press of America, 1994), 361–81. [Aug61] Saint Augustine, Confessions, trans. R. S. Pine-Coffin (New York: Penguin, 1961), 172.

Publications of Michael K. Sain

Theses, Dissertations Directed R.W. Diersing, Ph.D. Dissertation, H∞ , Cumulants, and Games, 2006 Y. Shang, Ph.D. Dissertation, Semimodule Morphic Systems: Realization, Model Matching, and Decoupling, 2006 K. D. Pham, Ph.D. Dissertation, Statistical Control Paradigms for Structural Vibration Suppression, 2004 G. Jin, Ph.D. Dissertation, System Identification for Controlled Structures in Civil Engineering Application: Algorithm Development and Experimental Verification, 2002 Y. Chi, M.S. Thesis, The Base Isolation Method and Its Application to Asymmetric Buildings, 2000 P. Quast, Ph.D. Dissertation, Output Feedback Minimum Cost Variance Control Schemes with Application to Aseismic Protection of Civil Engineering Structures, 1997 P. Quast, M.S. Thesis, Implementation of Active Control Strategies for Aseismic Protection of Civil Engineering Structures, 1997 P.M. Sain, Ph.D. Dissertation, Volterra Control Synthesis, Hysteresis Models and Magnetorheological Structure Protection, 1997 H. Dai, Ph.D. Dissertation, Investigations on the Control of a Hydraulic Shaker Table and of Magnetorheological Dampers for Seismic Response Reduction, 1997 T.T. Nguyen, M.S. Thesis, Scheduled Nonlinear Control for Ramp Tracking: The Role of Architecture and Performance, 1996 C.-H. Won, Ph.D. Dissertation, Cost Cumulants in Risk-Sensitive and Minimal Cost Variance Control, 1995 D.P. Newell, M.S. Thesis, Modelling and Control of a Nonlinear, Hydraulic Based Seismic Simulator, 1993 L.H. McWilliams, Ph.D. Dissertation, Qualitative Features of Linear Time-Invariant System Transient Responses, 1993 C.-H. Won, M.S. Thesis, Cost Variance Control and Risk Sensitivity, 1992 H. Jiang, M.S. Thesis, A Class of Hysteretic Servomechanisms, 1991 C.B. Schrader, Ph.D. Dissertation, Feedback Effects on Poles and Zeros: A Global Approach Using Generalized Systems, 1990 P.M. Sain, M.S. Thesis, Nonlinear Servomechanisms: A Volterra Approach, 1990 C.B. Schrader, M.S. Thesis, Subzeros of Linear Multivariable Systems, 1987 E.D. Alden, M.S. Thesis, Nonlinear Modeling of a Supersonic Inlet, 1987

332

Publications of Michael K. Sain

S.A. Al-Baiyat, Ph.D. Dissertation, Nonlinear Feedback Synthesis: A Volterra Approach, 1986 J.A. O’Sullivan, Ph.D. Dissertation, Nonlinear Optimal Regulation by Polynomic Approximation Methods, 1986 L.H. McWilliams, M.S. Thesis, Nonlinear Models by Layered Tensors, 1985 K.P. Dudek, Ph.D. Dissertation, The Total Synthesis Problem for Linear Multivariable Systems with Disturbances, 1984 S. Yurkovich, Ph.D. Dissertation, Multilinear Modeling and Linear Analysis of Nonlinear Systems, 1984 D. Bugajski, M.S. Thesis, Monomial Reduction of Nonlinear Tensor Models, 1984 J.A. O’Sullivan, M.S. Thesis, The Computer Implementation of the Tensor Solution to the Nonlinear Optimal Control Problem, 1984 T.A. Klingler, M.S. Thesis, Nonlinear Modeling of a Turbofan Jet Engine: A Tensor Method Application, 1982 J.P. Hill, M.S. Thesis, Solution of Nonlinear Optimal Control Problems Using the Algebraic Tensor: An Example, 1982 S. Yurkovich, M.S. Thesis, Application of Tensor Ideas to Nonlinear Modeling and Control, 1981 R.M. Schafer, Ph.D. Dissertation, On the Design of Dynamical Compensation for Diagonal Dominance, 1980 R.R. Gejji, Ph.D. Dissertation, On the Total Synthesis Problem of Linear Multivariable Control, 1980 V. Seshadri, Ph.D. Dissertation, Multivariable Loop Closures: An Approach by Exterior Algebra, 1979 J.G. Comiskey, M.S. Thesis, Time Optimal Control of a Jet Engine Using a Quasi-Hermite Interpolation Model, 1979 R.M. Schafer, M.S. Thesis, A Graphical Approach to System Dominance, 1977 P. Hoppner, M.S. Thesis, The Direct Approach to Compensation of Multivariable Jet Engine Models, 1977 A. Maloney, M.S. Thesis, Graphics Analysis of Dominance in Jet Engine Control Models, 1976 R.R. Gejji, M.S. Thesis, Polynomic Techniques Applied to Multivariable Control, 1976 V. Seshadri, M.S. Thesis, Compensation of Multivariable Control Systems, 1976 L.-N. Lee, M.S. Thesis, A Local Convergence Diagnostic for the LEADICS Simulation Algorithm, 1972 D. Sciacovelli, M.S. Thesis, Effects of Terminal Penalties and Noisy Measurements on LQG Performance Densities, 1972 M.V. Maguire, M.S. Thesis, An Indefinite Asymmetric Riccati Equation of Linear Quadratic Gaussian Control, 1971 S.R. Liberty, Ph.D. Dissertation, Characteristic Functions of LQG Control, 1971 F.H. Burrows, M.S. Thesis, A Viskovatov Technique for Rational Approximation of Probability Density Functions, 1971 D.L. Wiener, M.S. Thesis, A Control Theory Laboratory Manual for Students with Diverse Programming Backgrounds, 1970 S.R. Liberty, M.S. Thesis, Minimal Variance Feedback Controllers: Initial Studies of Solutions and Properties, 1969 L. Cosenza, Ph.D. Dissertation, On the Minimum Variance Control of Discrete-Time Systems, 1969 L. Cosenza, M.S. Thesis, Covariance Matrices and Optimal Modes in a Minimum Variance Control Problem, 1967

Publications of Michael K. Sain

333

Volumes Edited, Partially Edited, Books Founding Editor-in-Chief, Circuits and Systems, IEEE Circuits and Systems Society Magazine, 2001–2003. Editor, Chapter on Control and Systems, Electrical Engineering Handbook, Wai-Kai Chen, Editor, Academic Press, 2001. Editor-in-Chief, IEEE Circuits and Systems Society Newsletter, 1990–2000. Guest Co-Editor, with J.J. Uhran, Jr., IEEE Communications Magazine, Special Issue on “International Engineering Education,” November 1992. Guest Co-Editor, with J.J. Uhran, Jr., IEEE Communications Magazine Special Issue on “Engineering Education: Where Are We?,” December 1990. Honorary Editorial Advisory Board, Encyclopedia of Systems and Control, Pergamon Press, 1987. Associate Editor, Modeling and Simulation, Volume 16, William G. Vogt and Marlin H. Mickle, Editors. Research Triangle Park, N.C.: Instrument Society of America, 1985. Member, Guest Editorial Board, IEEE Transactions on Circuits and Systems, IEEE Transactions on Automatic Control, IEEE Transactions on Systems, Man and Cybernetics Joint Special Issue on “Large Scale Systems,” 1983. Editor, IEEE Transactions on Automatic Control, February 1979 – May 1983. Michael K. Sain, Introduction to Algebraic Systems Theory. New York: Academic Press, May 1981. Guest Editor, IEEE Transactions on Automatic Control Special Issue on “Linear Multivariable Control Systems,” February 1981. Member, Editorial Board, Journal for Interdisciplinary Modeling and Simulation, 1977–1980. Michael K. Sain, Joseph L. Peczkowski, and James L. Melsa, Editors, Alternatives for Linear Multivariable Control. Chicago: National Engineering Consortium, 1978. Associate Editor, IEEE Proceedings, Special Issue on “Recent Trends in System Theory,” January 1976. Associate Editor, Journal of the Franklin Institute, 150th Anniversary Issue on “Recent Trends in System Theory,” January–February 1976. Associate Editor, IEEE Control Systems Society Newsletter, July 1968–January 1973.

Publications 347.

346.

345.

344.

Ying Shang and Michael K. Sain, Fixed Zeros in the Model Matching Problem for Systems over a Semiring, Proceedings 46th IEEE Conference on Decision and Control, New Orleans, December 2007 Ronald Diersing, Michael K. Sain, and Chang-Hee Won, Discrete-Time, Bi-Cumulant Minimax and Nash Games, Proceedings 46th IEEE Conference on Decision and Control, New Orleans, December 2007 Ying Shang and Michael K. Sain, Fixed Poles in the Model Matching Problem for Systems over Semirings, Proceedings American Control Conference, New York, July 11–13, 2007 R.W. Diersing, M.K. Sain, K.D. Pham, and C. Won, Output Feedback Multiobjective Cumulant Control with Structural Applications, Proceedings American Control Conference, New York, July 11–13, 2007

334

Publications of Michael K. Sain

343.

R.W. Diersing and M.K. Sain, Nash and Minimax Bi-Cumulant Games, Proceedings 45th IEEE Conference on Decision and Control, San Diego, December 13–15, 2006 Barbara K. Sain and Michael K. Sain, A Course in Integration: Faith, Engineering, and Feedback, The Role of Engineering at Catholic Universities, University of Dayton, September 22–24, 2005 R.W. Diersing and M.K. Sain, Multi-Cumulant, Multi-Objective Structural Control: A Circuit Analog, Analog Integrated Circuits and Signal Processing, Special Issue for MWSCAS 2005 R.W. Diersing, M.K. Sain, K.D. Pham, and C. Won, Multi-Cumulant, Multi-Objective Structural Control: A Circuit Analog, Proceedings Midwest Symposium on Circuits and Systems, Cincinnati, Ohio, Pages 3090–3095, August 8–10, 2005 R.W. Diersing, M.K. Sain, K.D. Pham, C. Won, A Multiobjective Cost Cumulant Control Problem: A Nash Game Solution, Proceedings American Control Conference, Pages 309–314, Portland, Oregon, June 8–10, 2005 Khanh D. Pham, Michael. K Sain, and Stanley R. Liberty, Statistical Control for Smart Base-Isolated Buildings via Cost Cumulants and Output Feedback Paradigm, Proceedings American Control Conference, Portland, June 8–10, 2005 R.W. Diersing, M.K. Sain, K.D. Pham, and C. Won,, The Third Generation Wind Structural Benchmark: A Nash Cumulant Approach, Proceedings American Control Conference, Portland, Pages 3078–3083, June 8–10, 2005 Ying Shang and Michael. K. Sain, On Zero Semimodules of Systems over Semirings with Applications to Queueing Systems, Proceedings American Control Conference, Portland, June 8–10, 2005 Patrick M. Sain, Ying Shang, and Michael K. Sain, Reachability Analysis for NSquared State Charts over a Boolean Semiring Applied to a Hysteretic Discrete Event Structural Control Model, Proceedings American Control Conference, Portland, June 2005 G. Jin, M.K. Sain, and B.F. Spencer, Jr., Nonlinear Blackbox Modeling of MRDampers for Civil Structural Control, IEEE Transactions on Control Systems Technology, Volume 13, Number 3, Pages 345–355, May 2005 Luis Cosenza, Michael K. Sain, and Ronald W. Diersing, Cumulant Control Systems: The Cost-Variance, Discrete-Time Case, Proceedings Forty-Second Allerton Conference on Communication, Control, and Computing, Pages 230–239, October 2004 Chang-Hee Won, Michael Sain, and Stanley Liberty, Infinite-Time Minimal Cost Variance Control and Coupled Algebraic Riccati Equations, Proceedings American Control Conference, Denver, Colorado, Pages 5155–5160, June 2003. K.D. Pham, M.K. Sain, and S.R. Liberty, Cost Cumulant Control: State-Feedback, Finite-Horizon Paradigm with Application to Seismic Protection, Journal of Optimization Theory and Applications, Volume 115, Number 3, Pages 685–710, December 2002 G. Yang, B.F. Spencer, Jr., J.D. Carlson, and M.K. Sain, Large-Scale MR Fluid Dampers: Modeling and Dynamic Performance Considerations, in Engineering Structures, Volume 24, Pages 309–323, 2002 Khanh D. Pham, Michael K. Sain, and Stanley R. Liberty, Finite Horizon Full-State Feedback kCC Control in Civil Structures Protection, in Stochastic Theory and Control, Bozenna Pasik-Duncan, Editor. New York: Springer-Verlag Lecture Notes in Control and Information Sciences, Number 280, Pages 369–384, 2002 Khanh D. Pham, Michael K. Sain, and Stanley R. Liberty, Robust Cost-Cumulants Based Algorithms for Second and Third Generation Structural Control Benchmarks, Proceedings American Control Conference, Anchorage, Pages 3070–3075, May 2002

342.

341.

340.

339.

338.

337.

336.

335.

334.

333.

332.

331.

330.

329.

328.

Publications of Michael K. Sain 327.

326.

325.

324.

323.

322.

321.

320.

319.

318.

317.

316.

315.

335

Gang Jin, Michael K. Sain, and B.F. Spencer Jr., Modeling MR-Dampers: A Nonlinear Blackbox Approach, Proceedings American Control Conference, Anchorage, Pages 429–434, May 2002 K.D. Pham, G. Jin, M.K. Sain, B.F. Spencer, and S.R. Liberty, Generalized LQG Techniques for the Wind Benchmark Problem, Journal of Engineering Mechanics, Special Issue on Wind Benchmark, 2002 Chang-Hee Won, Michael K. Sain, and Stanley R. Liberty, Full State Feedback Minimal Cost Variance Control on an Infinite Time Horizon, Proceedings 40th IEEE Conference on Decision and Control, Pages 819–824, December 2001 Michael K. Sain and Khanh D. Pham, Cost Cumulant Control for Protection of Civil Structures, Abstracts Workshop on Stochastic Theory and Adaptive Control, University of Kansas, Lawrence, October 2001 B.F. Spencer, Jr. and Michael K. Sain, Controlling Civil Infrastructures, in Perspectives in Control Engineering: Technologies, Applications, and New Directions, Tariq Samad, Editor. New York: IEEE Press, 2001, Pages 417–441 Gang Jin, Michael K. Sain, Khanh D. Pham, Billie F. Spencer, Jr., and J.C. Ramallo, Modelling MR-Dampers: A Nonlinear Blackbox Approach, Proceedings American Control Conference, Arlington, Pages 429–434, June 2001 Libin Mou, Stanley R. Liberty, Khanh D. Pham, and Michael K. Sain, Linear Cumulant Control and Its Relationship to Risk-Sensitive Control, Proceedings ThirtyEighth Allerton Conference on Communication, Control, and Computing, Pages 422–430, October 2000 Gang Jin, Khanh D. Pham, B.F. Spencer, Jr., Michael K. Sain, and Stanley R. Liberty, A Study of the ASCE Wind Benchmark Problem by Generalized LQG Techniques, CD-ROM Proceedings 2nd European Conference on Structural Control, 6 Pages, July 3–6, 2000 Yun Chi, Michael K. Sain, Billie F. Spencer, Jr., and Khanh D. Pham, Base Isolation and Control of an Asymmetric Building, CD-ROM Proceedings 8th ASCE Probabilistic Mechanics and Structural Reliability Conference, Notre Dame, Indiana, A. Kareem, A. Haldar, B.F. Spencer, and E.A. Johnson, Editors, 6 Pages, July 24–26, 2000 Khanh D. Pham, Michael K. Sain, Stanley R. Liberty, and B.F. Spencer, Jr., Protecting Tall Buildings under Stochastic Winds Using Multiple Cost Cumulants, CD-ROM Proceedings 8th ASCE Probabilistic Mechanics and Structural Reliability Conference, Notre Dame, Indiana, A. Kareem, A. Haldar, B.F. Spencer, and E.A. Johnson, Editors, 6 Pages, July 24–26, 2000 Gang Jin, Michael K. Sain, and Billie F. Spencer, Closed-Loop Identification and Control Re-Design: An Experimental Structural Control Example, CD-ROM Proceedings 8th ASCE Probabilistic Mechanics and Structural Reliability Conference, Notre Dame, Indiana, A. Kareem, A. Haldar, B.F. Spencer, and E.A. Johnson, Editors, 6 Pages, July 24–26, 2000 Khanh D. Pham, Michael K. Sain, Stanley R. Liberty, and B.F. Spencer, Jr., Optimum Multiple Cost Cumulants for Protection of Civil Structures, CD-ROM Proceedings 8th ASCE Probabilistic Mechanics and Structural Reliability Conference, Notre Dame, Indiana, A. Kareem, A. Haldar, B.F. Spencer, and E.A. Johnson, Editors, 6 Pages, July 24–26, 2000 Gang Jin, Michael K. Sain, and B.F. Spencer, Jr., Frequency Domain Identification with Fixed Zeros: First Generation Seismic-AMD Benchmark, Proceedings American Control Conference, Pages 981–985, June 28–30, 2000

336

Publications of Michael K. Sain

314.

Khanh D. Pham, Michael K. Sain, Stanley R. Liberty, and B.F. Spencer, Jr., First Generation Seismic-AMD Benchmark: Robust Structural Protection by the Cost Cumulant Control Paradigm, Proceedings American Control Conference, Pages 1–5, June 28–30, 2000 G. Yang, J.C. Ramallo, B.F. Spencer, Jr., J.D. Carlson, and M.K. Sain, Dynamic Performance of Large-Scale MR Fluid Dampers, CD-ROM Proceedings 14th ASCE Engineering Mechanics Division Conference, Austin, Texas, J.L. Tassoulas, Editor, 6 Pages, May 21–24, 2000 Yun Chi, Michael K. Sain, B.F. Spencer, Jr., and Khanh D. Pham, Base Isolation and Control of an Asymmetric Building, CD-ROM Proceedings 14th ASCE Engineering Mechanics Division Conference, Austin, Texas, J.L. Tassoulas, Editor, 6 Pages, May 21–24, 2000 Khanh D. Pham, Gang Jin, B.F. Spencer, Jr., Michael K. Sain, and Stanley R. Liberty, Third Generation Wind-AMD Benchmark: Cost Cumulant Control Methodology for Wind Excited Tall Buildings, CD-ROM Proceedings 14th ASCE Engineering Mechanics Division Conference, Austin, Texas, J.L. Tassoulas, Editor, 6 Pages, May 21–24, 2000 Khanh D. Pham, Michael K. Sain, Stanley R. Liberty, and B.F. Spencer, Jr., The Role and Use of Optimal Cost Cumulants for Protection of Civil Structures, CD-ROM Proceedings 14th ASCE Engineering Mechanics Division Conference, Austin, Texas, J.L. Tassoulas, Editor, 6 Pages, May 21–24, 2000 Gang Jin, Michael K. Sain, and Billie F. Spencer, A Bench-Scale Experiment for AMD-Building Control Systems, CD-ROM Proceedings 14th ASCE Engineering Mechanics Division Conference, Austin, Texas, J.L. Tassoulas, Editor, 6 Pages, May 21–24, 2000 J.C. Ramallo, E.A. Johnson, B.F. Spencer, Jr., and M.K. Sain, Semiactive Base Isolation Systems, ASCE 2000 Structures Congress: Advanced Technology in Structural Engineering, Philadelphia, Pennsylvania, May 8–10, 2000. CD-ROM Proceedings (M. Elgaaly, ed.), paper number 40492-005-002, 8 pages. Michael K. Sain and Cheryl B. Schrader, Bilinear Operators and Matrices, Mathematics for Circuits and Filters, Wai-Kai Chen, Editor, CRC Press, Pages 19–36, 2000 Cheryl B. Schrader and Michael K. Sain, Linear Operators and Matrices, Mathematics for Circuits and Filters, Wai-Kai Chen, Editor, CRC Press, Pages 1–18, 2000 Michael K. Sain, Chang-Hee Won, B.F. Spencer, Jr., and Stanley R. Liberty, Cumulants and Risk-Sensitive Control: A Cost Mean and Variance Theory with Application to Seismic Protection of Structures, Pages 427–459 in Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games, Volume 5, Jerzy A Filar, Vladimir Gaitsgory, and Koichi Mizukami, Editors. Boston: Birkh¨auser, 2000 Michael K. Sain, 35 Years of Cost Cumulant Surprises, Workshop on Advances in Systems and Control, in honor of Professor William R. Perkins on the occasion of his 65th birthyear, Computer & Systems Research Laboratory, University of Illinois at Urbana-Champaign, September 25, 1999 Khanh D. Pham, Michael K. Sain, Stanley R. Liberty, and B.F. Spencer, Jr., Evaluating Cumulant Controllers on a Benchmark Structure Protection Problem in the Presence of Classic Earthquakes, Proceedings Thirty-Seventh Allerton Conference on Communication, Control, and Computing, Pages 617–626, October 1999 J.C. Ramallo, Erik A. Johnson, B.F. Spencer, Jr., and M.K. Sain, Effects of Semiactive Damping on the Response of Base-Isolated Buildings, Proceedings 13th ASCE

313.

312.

311.

310.

309.

308.

307. 306. 305.

304.

303.

302.

Publications of Michael K. Sain

301.

300.

299.

298.

297.

296.

295.

294.

293.

292.

291.

290.

289.

288.

337

Engineering Mechanics Division Conference, Johns Hopkins University, Baltimore, 6 Pages, CD-ROM, June 13–16, 1999 Guangqiang Yang, Gang Jin, B.F. Spencer, Jr., and Michael K. Sain, Bench-Scale Structural Control Experiment, Proceedings 13th ASCE Engineering Mechanics Division Conference, Johns Hopkins University, Baltimore, 6 Pages, CD-ROM, June 13– 16, 1999 Khanh D. Pham, Stanley R. Liberty, Michael K. Sain, and B.F. Spencer, Jr., Generalized Risk Sensitive Building Control: Protecting Civil Structures with Multiple Cost Cumulants, Proceedings American Control Conference, Pages 500–504, June 1999 J.C. Ramallo, E.A. Johnson, B.F. Spencer, Jr., and Michael K. Sain, Semiactive Building Base Isolation, Proceedings American Control Conference, Pages 5l5–519, June 1999 Michael K. Sain, Patrick M. Sain, Hongliang Dai, Chang-Hee Won, and B.F. Spencer, Jr., Nonlinear Paradigms for Structural Control: Some Perspectives, Proceedings Second World Conference on Structural Control, Kyoto, Japan, June 1998, Volume 3, John Wiley & Sons, West Sussex, England, Pages 1911–1920, 1999 Gang Jin, Yun Chi, Michael K. Sain, and Billie F. Spencer, Jr., Frequency Domain Optimal Control of the Benchmark Wind-Excited Building, Proceedings Second World Conference on Structural Control, Kyoto, Japan, June 1998, Volume 2, John Wiley & Sons, West Sussex, England, Pages 1417–1425, 1999 Erik A. Johnson, Juan C. Ramallo, Billie F. Spencer, Jr., and Michael K. Sain, Intelligent Base Isolation Systems, Proceedings Second World Conference on Structural Control, Kyoto, Japan, June 1998, Volume 1, John Wiley & Sons, West Sussex, England, Pages 367–376, 1999 Billie F. Spencer, Jr., Guangqiang Yang, J. David Carlson, and Michael K. Sain, ‘Smart Dampers’ for Seismic Protection of Structures: A Full-Scale Study, Proceedings Second World Conference on Structural Control, Kyoto, Japan, June 1998, Volume 1, John Wiley & Sons, West Sussex, England, Pages 417–426, 1999 Khanh D. Pham, Stanley R. Liberty, and Michael K. Sain, Linear Optimal Cost Cumulant Control: A k-Cumulant Problem Class, Proceedings Thirty-Sixth Allerton Conference on Communication, Control, and Computing, Pages 460–469, October 1998 B.F. Spencer, Jr., E.A. Johnson, and Michael K. Sain Technological Frontiers of ‘Smart’ Protective Systems, Proceedings Asia-Pacific Workshop on Seismic Design and Retrofit of Structures, Taipei, Taiwan, Pages 346–360, August 1998 B.F. Spencer, Jr. and Michael K. Sain Controlling Buildings: A New Frontier in Feedback, The Shock and Vibration Digest, Volume 30, Number 4, Pages 267–281, July 1998 Patrick M. Sain, Michael K. Sain, B.F. Spencer, Jr. and John D. Sain, The Bouc Hysteresis Model: An Initial Study of Qualitative Characteristics, Proceedings American Control Conference, Pages 2559–2563, June 1998 S.J. Dyke, B.F. Spencer, Jr., M.K. Sain, and J.D. Carlson, An Experimental Study of MR Dampers for Seismic Protection, Smart Materials and Structures, Special Issue on Large Civil Structures, Volume 7, Pages 693–703, 1998 Patrick M. Sain, Michael K. Sain, and B.F. Spencer, Jr., Volterra Feedback Synthesis: A Systematic Algorithm for Simplified Volterra Controller Design and Realization, Proceedings Thirty-Fifth Allerton Conference on Communication, Control, and Computing, Pages 1053–1062, October 1997 Stanley R. Liberty, Peter Quast, and Michael K. Sain, Control Access to Performance Measure Statistics: A Second Moment Case, Proceedings Thirty-Fifth Allerton Conference on Communication, Control, and Computing, Pages 933–942, October 1997

338

Publications of Michael K. Sain

287.

B.F. Spencer, Jr. and Michael K. Sain, Controlling Buildings: A New Frontier in Feedback, Control Systems, Volume 17, Number 6, Special Issue on Emerging Technologies, Tariq Samad, Guest Editor, Pages 19–35, December 1997 S.J. Dyke, B.F. Spencer, Jr., M.K. Sain, and J.D. Carlson, On the Efficacy of Magnetorheological Dampers for Seismic Response Reduction, CD ROM Proceedings ASME Sixteenth Biennial Conference on Mechanical Vibration and Noise, Paper No. DETC 9697 96VIB3828, 10 Pages, September 1997 Chang-Hee Won, Michael K. Sain, and B.F. Spencer, Jr., Connections and Advances in Risk-Sensitive and MCV Stochastic Control, Proceedings Second Asia-Pacific Control Conference, Seoul, Korea, Volume III, Pages 59–62, July 1997 B.F. Spencer, Jr., J. David Carlson, M.K. Sain, and G. Yang, On the Current Status of Magnetorheological Dampers: Seismic Protection of Full-Scale Structures, Proceedings American Control Conference, Pages 458–462, June 1997 Patrick M. Sain, Michael K. Sain, and B.F. Spencer, Models for Hysteresis and Application to Structural Control, Proceedings American Control Conference, Pages 16–20, June 1997 Hongliang Dai, Michael K. Sain, and B.F. Spencer, Jr., Using Tensors to Track Earthquakes on Hydraulic Shaker Tables, Proceedings American Control Conference, Pages 1–5, June 1997 Michael K. Sain, Cumulants and Risk in Stochastic Control, Notre Dame Workshop on Response and Reliability of Stochastic Dynamical Systems: Numerics and Applications, Notre Dame, Indiana, May 1997 B.F. Spencer, Jr., M.K. Sain, and S.J. Dyke, Semi-Active Control of Civil Engineering Structures, Proceedings Eleventh Symposium on Structural Dynamics and Control, May 1997 S.J. Dyke, B.F. Spencer, Jr., M.K. Sain, and J.D. Carlson, An Experimental Study of Semi-Active Dampers for Seismic Hazard Mitigation, Proceedings of the ASCE Structures Congress XV, Pages 1358–1362, April 1997 B.F. Spencer, Jr., S.J. Dyke, M.K. Sain, and J.D. Carlson, Phenomenological Model for Magnetorheological Dampers, Journal of Engineering Mechanics, Volume 123, Number 3, Pages 230–238, March 1997 B.F. Spencer, Jr., S.J. Dyke, and M.K. Sain, Magnetorheological Dampers: A New Approach to Seismic Protection of Structures, Proceedings IEEE Conference on Decision and Control, Kobe, Japan, Pages 676–681, December 1996 Chang-Hee Won, Michael K. Sain, and B.F. Spencer, Jr., Interpretations of RiskSensitivity in Dynamic Optimization of Circuits and Systems, Proceedings IEEE Asia-Pacific Conference on Circuits and Systems, Seoul, Korea, Pages 191–194, November 1996 B.F. Spencer, Jr., T.L. Timlin, M.K. Sain, and S.J. Dyke, Series Solution of a Class of Nonlinear Optimal Regulators, Journal of Optimization Theory and Applications, Volume 91, Number 5, Pages 321–345, November 1996 S.J. Dyke, B.F. Spencer, Jr., M.K. Sain, and J.D. Carlson, Modeling and Control of Magnetorheological Dampers for Seismic Response Reduction, Smart Materials and Structures, Volume 5, Pages 565–575, 1996 S.J. Dyke, B.F. Spencer, Jr., P. Quast, D.C. Kaspari, Jr., and M.K. Sain, Implementation of an Active Mass Driver Using Acceleration Feedback Control, Microcomputers in Civil Engineering—Journal of Computer-Aided Civil and Infrastructure Engineering, Special Issue on Active and Hybrid Structural Control, Volume 11, Pages 305– 323, 1996

286.

285.

284.

283.

282.

281.

280.

279.

278.

277.

276.

275.

274.

273.

Publications of Michael K. Sain 272.

271.

270.

269.

268.

267.

266. 265.

264.

263.

262.

261. 260.

259.

258.

339

Chang-Hee Won, Michael K. Sain, and B.F. Spencer, Jr., Relationships in Modern Stochastic Control: Risk-Sensitive and Minimal Cost Variance Control, Proceedings Thirty-Fourth Allerton Conference on Communication, Control, and Computing, Pages 343–352, October 1996 S.J. Dyke, B.F. Spencer, Jr., P. Quast, M.K. Sain, D.C. Kaspari, Jr., and T.T. Soong, Acceleration Feedback Control of MDOF Structures, Journal of Engineering Mechanics, Volume 122, Number 9, Pages 907–918, September 1996 S.J. Dyke, B.F. Spencer, Jr., M.K. Sain, and J.D. Carlson, Experimental Verification of Semi-Active Structural Control Strategies Using Acceleration Feedback, Proceedings Third International Conference on Motion and Vibration Control, Chiba, Japan, Volume 3, Pages 291–296, September 1996 S.J. Dyke, B.F. Spencer, Jr., M.K. Sain, and J.D. Carlson, Seismic Response Reduction Using Magnetorheological Dampers, Proceedings Thirteenth Triennial World Congress, International Federation of Automatic Control, San Francisco, Volume L, Pages 145–150, June/July 1996 B.F. Spencer, Jr., S.J. Dyke, M.K. Sain, and J.D. Carlson, Nonlinear Identification of Semi-Active Control Devices, Proceedings Eleventh ASCE Engineering Mechanics Specialty Conference, Fort Lauderdale, Florida, Pages 164–167, May 1996 S.J. Dyke, B.F. Spencer, Jr., M.K. Sain, and J.D. Carlson, A New Semi-Active Control Device for Seismic Response Reduction,Proceedings Eleventh ASCE Engineering Mechanics Specialty Conference, Fort Lauderdale, Florida, Pages 886–889, May 1996 M.K. Sain, Cumulants and Risk in Stochastic Feedback Control, Notre Dame Symposium in Applied Mathematics, Notre Dame, Indiana, April 19, 1996 B.F. Spencer, Jr., S.J. Dyke, P. Quast, M.K. Sain, and J.D. Carlson, Dynamical Model of a Magnetorheological Damper, Proceedings Twelfth ASCE Conference on Analysis and Computation, Chicago, Illinois, Pages 1277–1287, April 1996 D.P. Tomasula, B.F. Spencer, Jr., and M.K. Sain, Nonlinear Control Strategies for Limiting Dynamic Response Extremes, Journal of Engineering Mechanics, Volume 122, Number 3, Pages 218–229, March 1996 M.K. Sain, Risk, Reliability, and Reluctance: Three Intriguing Paradigms for Control Practice, 1996 Advanced Control Applications Workshop, Hughes Electro-Optics Sensors Technology Network, Mechanics, Cryogenics, and Controls Core Competency Team, El Segundo, California, February 22, 1996 Michael K. Sain, Chang-Hee Won, and B.F. Spencer, Jr., Cumulants in Risk-Sensitive Control: The Full-State-Feedback Cost Variance Case, Proceedings IEEE Conference on Decision and Control, Pages 1036–1041, December 1995 Michael K. Sain, Coding and System Theory: The Early Years, Proceedings IEEE Conference on Decision and Control, Pages 3247–3252, December 1995 B.F. Spencer, Jr., S.J. Dyke, and M.K. Sain, Experimental Verification of Acceleration Feedback Control Strategies for Seismic Protection, Proceedings Third Colloquium on Vibration Control of Structures, Japan Society of Civil Engineers, Tokyo, Japan, Part A, Pages 259–265, August 1995 B.F. Spencer, Jr., D.C. Kaspari, Jr., and M.K. Sain, Stochastic Stability Robustness of Parametrically Uncertain Systems, Proceedings Symposium on Quantitative and Parametric Feedback Theory, Wright Laboratory and Purdue University, Pages 127–133, August 1995 Chang-Hee Won, Michael K. Sain, and B.F. Spencer, Jr., Performance and Stability Characteristics of Risk-Sensitive Controlled Structures under Seismic Disturbances, Proceedings American Control Conference, Pages 1926–1930, June 1995

340

Publications of Michael K. Sain

257.

Devin P. Newell, Michael K. Sain, Peter Quast, Hongliang Dai, and B.F. Spencer, Jr., Nonlinear Modeling and Control of a Hydraulic Seismic Simulator, Proceedings American Control Conference, Pages 801–805, June 1995 D.C. Kaspari, Jr., B.F. Spencer, Jr., and M.K. Sain, Optimal Structural Control: A Reliability-Based Approach, Proceedings ASME Fifteenth Biennial Conference on Mechanical Vibration and Noise; Symposium on Dynamics and Control of Stochastic Dynamical Systems, Boston, Pages 855–862, 1995 D.C. Kaspari, Jr., B.F. Spencer, Jr., and M.K. Sain, Reliability Based Optimal Control of MDOF Structures, Engineering Mechanics: Proceedings of the Tenth Conference, Boulder, Colorado, Volume 2, Pages 810–813, May 1995 S.J. Dyke, B.F. Spencer, Jr., P. Quast, and M.K. Sain, Experimental Study of an Active Mass Driver Using Acceleration Feedback Control Strategies, Engineering Mechanics: Proceedings of the Tenth Conference, Boulder, Colorado, Volume 2, Pages 1227– 1230, May 1995 Michael K. Sain and Cheryl B. Schrader, Bilinear Operators and Matrices, The Circuits and Filters Handbook, Wai-Kai Chen, Editor, CRC Press and IEEE Press, Pages 23–41, 1995 Cheryl B. Schrader and Michael K. Sain, Linear Operators and Matrices, The Circuits and Filters Handbook, Wai-Kai Chen, Editor, CRC Press and IEEE Press, Pages 3–22, 1995 S.J. Dyke, B.F. Spencer, Jr., P. Quast, and M.K. Sain, The Role of Control-Structure Interaction in Protective System Design, Journal of Engineering Mechanics, Volume 121, Number 2, Pages 322–338, February 1995, Remark: Nominated for Norman Medal. P. Quast, M.K. Sain, B.F. Spencer, Jr., and S.J. Dyke, Microcomputer Implementation of Digital Control Strategies for Structural Response Reduction, Microcomputers in Civil Engineering—Journal of Computer-Aided Civil and Infrastructure Engineering, Special Issue on New Directions in Computer Aided Structural System Analysis, Design, and Optimization, Volume 10, Number 1, Pages 13–25, January 1995 Leo H. McWilliams and Michael K. Sain, Qualitative Features of Discrete-Time System Response, Proceedings IEEE Conference on Decision and Control, Pages 18–23, December 1994 Michael K. Sain, Humility in Academic Life: Over the Rainbow?, Faculty Upper Room Dinner and Discussion Series on Faith and the Professional Life, September 27, 1994 Leo H. McWilliams and Michael K. Sain, Discrete-Time Systems: New Results on Undershoot and Overshoot, Proceedings Thirty-Second Allerton Conference on Communication, Control, and Computing, Pages 11–20, September 1994 S.J. Dyke, B.F. Spencer, Jr., P. Quast, M.K. Sain, D.C. Kaspari, Jr., and T.T. Soong, Experimental Verification of Acceleration Feedback Control Strategies for an Active Tendon System, National Center for Earthquake Engineering Research, Technical Report NCEER-94-0024, August 1994 P. Quast, S.J. Dyke, B.F. Spencer, Jr., A.E. Belknap, K.J. Ferrell, and M.K. Sain, Acceleration Feedback Control Strategies for the Active Mass Driver (AMD), Video Presentation of Experimental Results in the SDC/EEL Facility, University of Notre Dame, August 1994 S.J. Dyke, B.F. Spencer, Jr., A.E. Belknap, K.J. Ferrell, P. Quast, and M.K. Sain, Absolute Acceleration Feedback Control Strategies for the Active Mass Driver, Proceedings First World Conference on Structural Control, Volume 2, Pages TP1-51— TP1-60, August 1994

256.

255.

254.

253.

252.

251.

250.

249.

248.

247.

246.

245.

244.

Publications of Michael K. Sain 243.

242.

241.

240.

239.

238.

237.

236.

235.

234.

233.

232.

231.

230.

341

D.P. Tomasula, B.F. Spencer, Jr., and M.K. Sain, Limiting Extreme Structural Responses Using an Efficient Nonlinear Control Law, Proceedings First World Conference on Structural Control, Volume 3, Pages FP4c-22—FP4-31, August 1994 B.F. Spencer, Jr., D.C. Kaspari, Jr., and M.K. Sain, Reliability-Based Optimal Structural Control, Proceedings Fifth U. S. National Conference on Earthquake Engineering, Earthquake Engineering Research Institute, Chicago, Volume 1, Pages 703–712, July 1994 B.F. Spencer, Jr., D.C. Kaspari, and M.K. Sain, Structural Control Design: A Reliability-Based Approach, Proceedings American Control Conference, Baltimore, Maryland, Pages 1062–1066, June/July 1994 S.J. Dyke, B.F. Spencer, Jr., P. Quast, M.K. Sain, and D.C. Kaspari, Jr., Experimental Verification of Acceleration Feedback Control Strategies for MDOF Structures, in Computational Stochastic Mechanics, P. Spanos, Editor. Rotterdam: A.A. Balkema, 1995, Pages 137–148, Proceedings Second International Conference on Computational Mechanics, Athens, Greece, June 1994 Chang-Hee Won, Michael K. Sain, and B.F. Spencer, Jr., Risk-Sensitive Structural Control Strategies, in Computational Stochastic Mechanics, P. Spanos, Editor. Rotterdam: A.A. Balkema, 1995, Pages 203–211. Proceedings Second International Conference on Computational Mechanics, Athens, Greece, June 1994 B.F. Spencer, Jr., P. Quast, S.J. Dyke, and M.K. Sain, Digital Signal Processing Techniques for Active Structural Control, Proceedings Eleventh ASCE Conference on Analysis and Computation, Atlanta, Georgia, Pages 327–336, April 1994 Michael K. Sain, Inverses of Linear Sequential Circuits: On Beyond Poles and Zeros..., in Communications and Cryptology: Two Sides of One Tapestry*, Richard E. Blahut, Daniel J. Costello, Jr., Ueli Maurer, and Thomas Mittelholzer, Editors. Boston: Kluwer Academic Publishers, Pages 367–379, 1994, *Based upon James L. Massey 60th Birthday Symposium, Ascona, Switzerland, February 1994 Cheryl B. Schrader and Michael K. Sain, Zero Principles for Implicit Feedback Systems, Circuits, Systems, and Signal Processing, Special Issue on Implicit and Robust Systems, Volume 13, Numbers 2–3, Pages 273–293, 1994 Joachim Rosenthal, Michael Sain, and Xiaochang Wang, Topological Considerations for Autoregressive Systems with Fixed Kronecker Indices, Circuits, Systems, and Signal Processing, Special Issue on Implicit and Robust Systems, Volume 13, Numbers 2–3, Pages 295–308, 1994 B.F. Spencer, Jr., J. Suhardjo, and M.K. Sain, Frequency Domain Optimal Control Strategies for Aseismic Protection, Journal of Engineering Mechanics, Volume 120, Number 1, Pages 135–158, January 1994 B.F. Spencer, Jr., M.K. Sain, C.-H. Won, D.C. Kaspari, and P.M. Sain, ReliabilityBased Measures of Structural Control Robustness, Structural Safety, Volume 15, Pages 111–129, 1994 S.J. Dyke, B.F. Spencer, Jr., P. Quast, and M.K. Sain, Protective System Design: The Role of Control-Structure Interaction, Proceedings International Workshop on Structural Control, Honolulu, Hawaii, August 1993, G.W. Housner and S.F. Masri, Editors, University of Southern California Press Pages 100–114, 1994 Cheryl B. Schrader and Michael K. Sain, Generalized System Poles and Zeros: The Generic and Global Connection, Proceedings IEEE Conference on Decision and Control, San Antonio, Texas, Pages 2866–2871, December 1993 Michael K. Sain, A Fresh Look at Inverse Dynamical Systems, Proceedings ThirtyFirst Allerton Conference on Communication, Control, and Computing, Pages 1196–1205, October 1993

342

Publications of Michael K. Sain

229.

B.F. Spencer, Jr., M.K. Sain, and J.C. Kantor, Reliability-Based Measures of Stability for Actively Controlled Structures, in Structural Safety and Reliability, G.I. Schueller, M. Shinozuka, and J.T.P. Yao, Editors. Rotterdam, A.A. Balkema Publishers, Pages 1591–1598, 1994, Proceedings Sixth International Conference on Structural Safety and Reliability, Innsbruck, Austria, August 1993 Cheryl B. Schrader and Michael K. Sain, On the Relationship Between Extended Zeros and Wedderburn-Forney Spaces, in Systems and Networks: Mathematical Theory and Application, Volume II, Uwe Helmke, Reinhard Mennicken, and Josef Saurer, Editors, Mathematical Research, Volume 79, Berlin, Akademie Verlag, Pages 471– 476, 1994, Proceedings Eleventh International Symposium on the Mathematical Theory of Networks and Systems, Regensburg, Germany, August 1993 Patrick M. Sain and Michael K. Sain, Astrom Receives IEEE Medal of Honor, Control Systems, Volume 13, Number 4, Pages 7–15, August 1993 S.J. Dyke, P. Quast, B.F. Spencer, Jr., M.K. Sain, and J.C. Kantor, Acceleration Feedback Control Strategies for Aseismic Protection, Video Presentation of Experimental Results in the SDC/EEL Facility, University of Notre Dame, June 1993 B.F. Spencer, Jr., M.K. Sain, C.-H. Won, and L.M. Barroso, Analysis of Structural Control Robustness: Reliability Methods, in Advances in Structural Reliability Methods. New York: Springer-Verlag, 1993, Pages 504–517, Proceedings IUTAM Symposium on Probabilistic Structural Mechanics, San Antonio, Texas, June 1993 B.F. Spencer, Jr., S. Dyke, M.K. Sain, and P. Quast, Acceleration Feedback Control Strategies for Aseismic Protection, Proceedings American Control Conference, San Francisco, California, Pages 1317–1321, June 1993 Chang-Hee Won, Michael K. Sain, and B.F. Spencer, Jr., Active Structural Control: A Risk Sensitive Approach, in Dynamics and Control of Large Structures, Proceedings Ninth Virginia Polytechnic Institute and State University Symposium, Blacksburg, Virginia, Pages 155–166, May 1993 B.F. Spencer, Jr., M.K. Sain, D.C. Kaspari, and J.C. Kantor, Reliability-Based Design of Active Control Strategies, Proceedings Seminar on Seismic Isolation, Passive Energy Dissipation, and Active Control, Applied Technology Council, Redwood City, California, Pages 761–772, March 1993 B.F. Spencer, Jr., M.K. Sain, J.C. Kantor, and C. Montemagno, Probabilistic Stability Measures for Controlled Structures Subject to Real Parameter Uncertainties, Smart Materials and Structures, Volume 1, Pages 294–305, 1992 Michael K. Sain, System Singularity: A Module Theoretic Approach, Proceedings International Symposium on Implicit and Nonlinear Systems, Fort Worth, Texas, Pages 411–425, December, 1992 Michael K. Sain, Chang-Hee Won, and B.F. Spencer, Jr., Cumulant Minimization and Robust Control, in Stochastic Theory and Adaptive Control, Tyrone E. Duncan and Bozenna Pasik-Duncan, Editors. New York: Springer-Verlag Lecture Notes in Control and Information Sciences, Number 184, Pages 411–425, 1992 J. Suhardjo, B.F. Spencer, Jr., M.K. Sain, and D. Tomasula, Nonlinear Control of a Tension Leg Platform, in Innovative Large Span Structures - Vol. I, N.K. Srivastava, A.N. Sherbourne and J. Roorda, Editors. Montreal: Canadian Society for Civil Engineering, Pages 464–474, 1992 B.F. Spencer, Jr., C. Montemagno, M.K. Sain, and P.M. Sain, Reliability of Controlled Structures Subject to Real Parameter Uncertainty, Proceedings Sixth ASCE Specialty Conference on Probabilistic Mechanics and Structural and Geotechnical Safety, Pages 369–372, July 1992

228.

227. 226.

225.

224.

223.

222.

221.

220.

219.

218.

217.

Publications of Michael K. Sain 216.

215.

214.

213.

212.

211.

210.

209.

208.

207.

206.

205.

204.

203.

202.

201.

343

Hong Jiang, Michael K. Sain, Patrick M. Sain and B.F. Spencer, Jr., Feedback Control of a Family of Nonlinear Hysteretic Systems, Proceedings Second IFAC Symposium on Nonlinear Control Design, Bordeaux, France, Pages 63–68, June 1992 P.M. Sain, B.F. Spencer, M.K. Sain, and J. Suhardjo, Structural Control Design in the Presence of Time Delays, Proceedings Ninth ASCE Engineering Mechanics Conference, Pages 812–815, May 1992 Patrick M. Sain and Michael K. Sain, Individualized Problem Assignments: An Answer to a Pedagogical Challenge, in Proceedings ASEE Illinois/Indiana Spring Conference, Section 4C, Pages 15–19, March 1992 M.K. Sain and C.B. Schrader, Feedback, Zeros, and Blocking Dynamics, in Recent Advances in Mathematical Theory of Systems, Control, Networks and Signal Processing I, H. Kimura, S. Kodama, Editors. Tokyo: Mita Press, Pages 227–232, 1992 J. Suhardjo, B.F. Spencer, Jr., and M.K. Sain, Nonlinear Optimal Control of a Duffing System, International Journal of Nonlinear Mechanics, Volume 27, Number 2, Pages 157–172, 1992 Cheryl B. Schrader and Michael K. Sain, Module Theoretic Results for Feedback System Matrices, in New Trends in Systems Theory, G. Conte, A.M. Perdon and B. Wyman, Editors. Boston: Birkh¨auser, Pages 652–659, 1991 B.F. Spencer, J. Suhardjo, and M.K. Sain, Frequency Domain Control Algorithms for Civil Engineering Applications, Proceedings International Workshop on Technology for Hong Kong’s Infrastructural Development, Hong Kong, Pages 169–178, December 1991 Cheryl B. Schrader and Michael K. Sain, Pole Zero Conservation Results for Nonminimal Systems, Proceedings IEEE Conference on Decision and Control, Brighton, England, Pages 378–383, December 1991 Anthony N. Michel and Michael K. Sain, Qualitative Theory for Dynamical Systems with Saturation Nonlinearities, Proceedings IEEE Conference on Decision and Control, Brighton, England, Pages 392–393, December 1991 Bostwick F. Wyman, Michael K. Sain, Giuseppe Conte, and Anna-Marie Perdon, Poles and Zeros of Matrices of Rational Functions, Journal of Linear Algebra and Its Applications, Special Issue on Algebraic Linear Algebra, Volume 157, Pages 113– 139, November 1991 Cheryl B. Schrader and Michael K. Sain, Extended Zeros, Poles and Anticausal Systems, Proceedings Third International Conference on Advances in Communication and Control Systems, Victoria, Canada, Pages 363–374, October 1991 Michael K. Sain, Cumulant Minimization and Robust Control, Abstracts Workshop on Stochastic Theory and Adaptive Control, University of Kansas, Lawrence, Page 19, September 1991 Patrick M. Sain, Michael K. Sain, and Anthony N. Michel, Nonlinear ModelMatching Design of Servomechanisms, Preprints First IFAC Symposium on Design Methods of Control Systems, Z¨urich, Switzerland, Pages 594–599, September 1991 Joachim Rosenthal and Michael Sain, On Kronecker Indices of Transfer Functions and Autoregressive Systems, Proceedings Second International Symposium on Implicit and Robust Systems, Warsaw, Poland, Pages 173–176, July 1991 Cheryl B. Schrader and Michael K. Sain, Extended Notions of Zeros in Implicit Feedback Systems, Proceedings Second International Symposium on Implicit and Robust Systems, Warsaw, Poland, Pages 177–180, July 1991 Michael K. Sain and Cheryl B. Schrader, Feedback, Zeros, and Blocking Dynamics, Tenth International Symposium on the Mathematical Theory of Networks and Systems, Kobe, Japan, June 1991

344

Publications of Michael K. Sain

200.

Michael K. Sain, Bostwick F. Wyman, and Joseph L. Peczkowski, Extended Zeros and Model Matching, SIAM Journal on Control and Optimization, Volume 29, Number 3, Pages 562–593, May 1991 B.F. Wyman, M.K. Sain, G. Conte, and A.M. Perdon, Algebraic and System Theoretic Notions of Poles and Zeros for Matrices of Rational Functions, Rendiconti del Seminario Matematico, Special Issue on the Mathematical Theory of Dynamical Systems and Ordinary Differential Equations, Part II, Universita’ e Politecnico, Torino, Volume 48, Number 3, Pages 225–250, 1990 Michael K. Sain and Bostwick F. Wyman, The Pole Module of a Feedback Compensator System is Contained in the Zero Module of the Resulting Feedback System, in Realization and Modelling in System Theory, M.A. Kaashoek, J.H. van Schuppen and A.C.M. Ran, Editors. Boston: Birkh¨auser, Pages 207–214, 1990 Michael K. Sain and Cheryl B. Schrader, Making Space for More Zeros, Proceedings IEEE Conference on Decision and Control, Pages 1–12, December 1990 J. Suhardjo, B.F. Spencer, Jr., and M.K. Sain, Feedback-Feedforward Control of Structures under Seismic Excitation, Journal of Structural Safety, Volume 8, Pages 69–89, 1990 Bostwick F. Wyman and Michael K. Sain, Rings and Modules for Linear Systems - Attempts at a Synthesis, Second SIAM Conference on Linear Algebra in Signals, Systems, and Control, November 1990 Michael K. Sain and Cheryl B. Schrader, Blocking Zeros and Dynamic Feedback, Proceedings Twenty-Eighth Allerton Conference on Communication, Control, and Computing, Pages 517–526, October 1990 Michael K. Sain and Cheryl B. Schrader, The Role of Zeros in the Performance of Multi-Input, Multi-Output Feedback Systems, IEEE Transactions on Education, Special Issue on Teaching Automatic Control, Volume 33, Number 3, Pages 244–257, August 1990 Patrick M. Sain, Michael K. Sain, and Anthony N. Michel, Higher Order Methods for Nonlinear Feedback Design, Proceedings Thirty-Third Midwest Symposium on Circuits and Systems, Pages 68–71, August 1990 Michael K. Sain, Cheryl B. Schrader, and Bostwick F. Wyman, Poles, Zeros, and Feedback: A Module Point of View, Proceedings Thirty-Third Midwest Symposium on Circuits and Systems, Pages 60–63, August 1990 Cheryl B. Schrader and Michael K. Sain, Module Theoretic Results for Feedback System Matrices, Joint Conference on New Trends in Systems Theory, Universita di Genova - The Ohio State University, Columbus - 500th Anniversary of Christopher Columbus Voyage - Genova, Italy, July 1990 J. Suhardjo, B.F. Spencer, Jr., and M.K. Sain, Feedback-Feedforward Control of Seismic Structures, Proceedings Fourth U.S. National Conference on Earthquake Engineering, Pages 437–446, May 1990 Michael K. Sain, Patrick M. Sain, and Anthony N. Michel, On Coordinated Feedforward Excitation of Nonlinear Servomechanisms, Proceedings American Control Conference, Pages 1695–1700, May 1990 J. Suhardjo, B.F. Spencer, Jr., and Michael. K. Sain, Control of Buildings under Earthquake Excitation, Proceedings American Control Conference, Pages 2595–2600, May 1990 Michael K. Sain, Cheryl B. Schrader, and Bostwick F. Wyman, A Theorem on the Effects of Compensator Poles on Feedback System Zeros, Proceedings IEEE International Symposium on Circuits and Systems, Pages 1385–1388, May 1990

199.

198.

197. 196.

195.

194.

193.

192.

191.

190.

189.

188.

187.

186.

Publications of Michael K. Sain 185.

184.

183.

182. 181.

180.

179.

178.

177.

176.

175.

174.

173. 172. 171.

170.

345

J. Suhardjo, B.F. Spencer, Jr., and M.K. Sain Feedback-Feedforward Control of Structures under Seismic Excitation, in Nonlinear Structural Systems under Random Conditions, F. Casciati, I. Elishakoff and J.B. Roberts, Editors. Amsterdam: Elsevier Science Publishers, Pages 69–89, 1990 Michael K. Sain, Joseph L. Peczkowski, and Bostwick F. Wyman, On Complexity in Synthesis of Feedback Systems, in Volume 130, Lecture Notes in Control and Information Sciences: Advances in Computing and Controls, W.A. Porter, S.C. Kak and J.L. Aravena, Editors. New York: Springer-Verlag, Pages 352–362, 1989 Bostwick F. Wyman, Michael K. Sain, Giuseppe Conte, and Anna-Marie Perdon, On the Zeros and Poles of a Transfer Function, Journal of Linear Algebra and Its Applications, Special Issue on Linear Systems, 122/123/124, Pages 123–144, 1989 Cheryl B. Schrader and Michael K. Sain, Research on System Zeros: A Survey, International Journal of Control, Volume 50, Number 4, Pages 1407–1433, 1989 L.H. McWilliams and M.K. Sain, Qualitative Step Response Limitations of Linear Systems, Proceedings IEEE Conference on Decision and Control, Pages 2223–2227, December 1989 J. Suhardjo, B.F. Spencer, Jr., and M.K. Sain, Indicial Representation of Higher Order Optimal Controls for a Duffing Oscillator, Proceedings IEEE Conference on Decision and Control, Pages 301–306, December 1989 Bostwick F. Wyman and Michael K. Sain, Filtrations for Linear Systems, Proceedings Twenty-Seventh Allerton Conference on Communication, Control, and Computing, Pages 104–110, September 1989 Patrick M. Sain, Michael K. Sain, and Anthony N. Michel, A Method for Higher Order, Dynamical Feedback in Nonlinear Servomechanism Problems, Proceedings Twenty-Seventh Allerton Conference on Communication, Control, and Computing, Pages 159–168, September 1989 B.F. Spencer, Jr., M.K. Sain, and J. Suhardjo, On the Adequacy of Linearized Methods for Control of Nonlinear Bridge Oscillations, Proceedings Fifth International Conference on Structural Safety and Reliability, Pages 1435–1442, August 1989 Michael K. Sain and Bostwick F. Wyman, The Pole Module of a Feedback System Compensator is Contained in the Zero Module of the Resulting Feedback System, Ninth International Symposium on the Mathematical Theory of Networks and Systems, Amsterdam, The Netherlands, June 1989 Joseph A. O’Sullivan and Michael K. Sain, Optimal Control by Polynomial Approximation: The Discrete-Time Case, Proceedings IFAC Symposium on Nonlinear Control Systems Design, Capri, Italy, Pages 350–355, June 1989 Samir A. Al-Baiyat and Michael K. Sain, A Volterra Method for Nonlinear Control Design, Proceedings IFAC Symposium on Nonlinear Control Systems Design, Capri, Italy, Pages 76–81, June 1989 Cheryl B. Schrader and Michael K. Sain, Subzeros of Linear Multivariable Systems, Proceedings American Control Conference, Pages 280–285, June 1989 Cheryl B. Schrader and Michael K. Sain, Subzeros in Feedback Transmission, Proceedings American Control Conference, Pages 799–804, June 1989 Cheryl B. Schrader and Michael K. Sain, Zero Synthesis in Linear Multivariable Subsystems, Proceedings IEEE International Symposium on Circuits and Systems, Pages 541–544, May 1989 Kenneth P. Dudek and Michael K. Sain, A Control-Oriented Model for Cylinder Pressure in Internal Combustion Engines, IEEE Transactions on Automatic Control, Pages 386–397, April 1989

346

Publications of Michael K. Sain

169.

Cheryl B. Schrader and Michael K. Sain, Research on System Zeros: A Survey, Proceedings IEEE Conference on Decision and Control, Pages 890–901, December 1988 Bostwick F. Wyman, Michael K. Sain, Giuseppe Conte, and Anne-Maria Perdon, Rational Matrices: Counting the Poles and Zeros, Proceedings IEEE Conference on Decision and Control, Pages 921–925, December 1988 Michael K. Sain, Bostwick F. Wyman, and Joseph L. Peczkowski, Matching Zeros: A Fixed Constraint in Multivariable Synthesis, Proceedings IEEE Conference on Decision and Control, Pages 2060–2065, December 1988 Michael K. Sain, Joseph L. Peczkowski, and Bostwick F. Wyman, On Complexity in Synthesis of Feedback Systems, Proceedings International Conference on Advances in Communications and Control Systems, Pages 1422–1432, October 1988 Cheryl B. Schrader and Michael K. Sain, Invariant Subzeros are Not Invariant, Proceedings Twenty-Sixth Allerton Conference on Communication, Control, and Computing, Pages 189–190, September 1988 Michael K. Sain, Bostwick F. Wyman, and Joseph L. Peczkowski, Classical and Extended Transmission Zeros: The Relationship in Model Matching, Proceedings Twenty-Sixth Allerton Conference on Communication, Control, and Computing, Pages 177–186, September 1988 Michael K. Sain and Bostwick F. Wyman, The Fixed Zero Constraint in Dynamical System Performance, Linear Circuits, Systems and Signal Processing: Theory and Application, C.I. Byrnes, C.F. Martin, and R. Saeks, Editors, North-Holland, 1988, pages 87–92 Bostwick F. Wyman and Michael K. Sain, Zeros of Square Invertible Systems, Linear Circuits, Systems and Signal Processing: Theory and Application, C.I. Byrnes, C.F. Martin, and R. Saeks, Editors, North-Holland, 1988, Pages 109–114 Kenneth P. Dudek and Michael K. Sain, An Application of Tensor Algebra to ControlOriented Modeling of Cylinder Pressure in Internal Combustion Engines, Linear Circuits, Systems and Signal Processing: Theory and Applications, C.I. Byrnes, C.F. Martin, and R. Saeks, Editors, North-Holland, 1988, Pages 125–130 Kenneth P. Dudek and Michael K. Sain, A Control-Oriented Model for Cylinder Pressure in Internal Combustion Engines, Proceedings American Control Conference, Pages 2412–2417, June 1988 Michael K. Sain, Bostwick F. Wyman, and Joseph L. Peczkowski, Zeros in Plant Specification: Constraints and Solutions, Proceedings American Control Conference, Pages 1243–1248, June 1988 Michael K. Sain and Bostwick F. Wyman, Extended Zeros and Model Matching, Proceedings Twenty-Fifth Allerton Conference on Communication, Control, and Computing, Pages 524–533, October 1987 Samir A. Al-Baiyat and Michael K. Sain, Volterra Models in Nonlinear Control Design: An Application, Eighth International Symposium on the Mathematical Theory of Networks and Systems, June 1987 Bostwick F. Wyman and Michael K. Sain, Module Theoretic Zero Structures for System Matrices, SIAM Journal on Control and Optimization, Volume 25, Number 1, Pages 86–99, January 1987 Samir A. Al-Baiyat and Michael K. Sain, Control Design with Transfer Functions Associated to Higher Order Volterra Kernels, Proceedings IEEE Conference on Decision and Control, Pages 1306–1311, December 1986 Bostwick F. Wyman and Michael K. Sain, On Dual Zero Spaces and Inverse Systems, IEEE Transactions on Automatic Control, Volume AC-31, Number 11, Pages 1053–1055, November 1986

168.

167.

166.

165.

164.

163.

162.

161.

160.

159.

158.

157.

156.

155.

154.

Publications of Michael K. Sain 153.

152. 151.

150.

149.

148.

147.

146.

145.

144.

143.

142.

141.

140.

139.

138.

347

Bostwick F. Wyman and Michael K. Sain, Zero Spaces, Pole Spaces, Duals, and Inverse Systems, Proceedings Twenty-Fourth Allerton Conference on Communication, Control, and Computing, Pages 145–149, October 1986 Joseph A. O’Sullivan and Michael K. Sain, An Approximation Approach to Nonlinear Optimal Regulation, American Control Conference, June 1986 Bostwick F. Wyman and Michael K. Sain, Module Theoretic Zero Structures for System Matrices, Proceedings IEEE Conference on Decision and Control, Pages 514– 518, December 1985 Bostwick F. Wyman and Michael K. Sain, On the Design of Pole Modules for Inverse Systems, IEEE Transactions on Circuits and Systems, Volume CAS-32, Number 10, Pages 977–988, October 1985 Joseph A. O’Sullivan and Michael K. Sain, A Theorem on the Feedback Equivalence of Nonlinear Systems Using Tensor Analysis, Proceedings Twenty-Third Annual Allerton Conference on Communication, Control, and Computing, Pages 156–157, October 1985 Samir A. Al-Baiyat and Michael K. Sain, An Application of Volterra Series to the Design of Nonlinear Feedback Controls, Proceedings Twenty-Third Annual Allerton Conference on Communication, Control, and Computing, Pages 103–112, October 1985 Stephen Yurkovich, Daniel Bugajski, and Michael Sain, Polynomic Nonlinear Dynamical Systems: A Residual Sensitivity Method for Model Reduction, Proceedings American Control Conference, Pages 933–939, June 1985 Michael K. Sain and Joseph L. Peczkowski, Nonlinear Control by Coordinated Feedback Synthesis, with Gas Turbine Applications, Proceedings American Control Conference, Pages 1121–1128, June 1985 Joseph L. Peczkowski and Michael K. Sain, Synthesis of System Responses: A Nonlinear Multivariable Control Design Appproach, Proceedings American Control Conference, Pages 1322–1329, June 1985 Joseph A. O’Sullivan and Michael K. Sain, Nonlinear Optimal Control with Tensors: Some Computational Issues, Proceedings American Control Conference, Pages 1600–1605, June 1985 Samir A. Al-Baiyat and Michael K. Sain, A History of Inverse Properties in System Models, Proceedings Sixteenth Annual Pittsburgh Conference on Modeling and Simulation, Part 2: Control, Robotics and Automation, Pages 495–499, April 1985 Leo H. McWilliams and Michael K. Sain, Extending Nonlinear Models with Tensor Parameters, Proceedings Sixteenth Annual Pittsburgh Conference on Modeling and Simulation, Part 2: Control, Robotics and Automation, Pages 489–493, April 1985 Bostwick F. Wyman and Michael K. Sain, Poles, Zeros, and Lattices, in Volume 47, Contemporary Mathematics: Linear Algebra and Its Role in Systems Theory. Providence: American Mathematical Society, Pages 497–506, 1985 Bostwick F. Wyman and Michael K. Sain, A Unified Pole-Zero Module for Linear Transfer Functions, Systems and Control Letters, Volume 5, Number 2, Pages 117– 120, November 1984 Joseph A. O’Sullivan and Michael K. Sain, Nonlinear Feedback Design: Optimal Responses by Tensor Analysis, Proceedings Twenty-Second Annual Allerton Conference on Communication, Control, and Computing, Pages 864–873, October 1984 P.J. Antsaklis and M.K. Sain, Feedback Synthesis with Two Degrees of Freedom: {G,H;P} Controller, Proceedings Ninth Triennial World Congress, International Federation of Automatic Control, Volume IX, Pages 5–10, July 1984

348 137.

Publications of Michael K. Sain

R. Michael Schafer and Michael K. Sain, Computer Aided Design Package for the Total Synthesis Problem, Proceedings Ninth Triennial World Congress, International Federation of Automatic Control, Volume VIII, Pages 179–184, July 1984 136. Joseph L. Peczkowski and Michael K. Sain, Design of Nonlinear Multivariable Feedback Controls by Total Synthesis, Proceedings American Control Conference, Pages 688–697, June 1984 135. P.J. Antsaklis and M.K. Sain, Feedback Controller Parameterizations: Finite Hidden Modes and Causality, in Multivariable Control: New Concepts and Tools, S.G. Tzafestas, Editor. Dordrecht, Holland: D. Reidel Publisher, 1984, Pages 85–104 134. Kenneth P. Dudek, Michael K. Sain, and Bostwick F. Wyman, Module Considerations for Feedback Synthesis of Sensitivity Comparisons, Proceedings Twenty-First Annual Allerton Conference on Communication, Control, and Computing, Pages 115–124, October 1983 133. P.J. Antsaklis and Michael K. Sain, Feedback Controller Parameterizations: Causality and Hidden Modes, Proceedings Sixth International Symposium on Measurement and Control, International Association of Science and Technology for Development, Pages 437–440, August 1983 132–1. K.T. Yang, Howard Saz, Gary Gutting, Lee Tavis, James Kolata, Michael Sain, Peri Arnold, Frank Castellino, and Eugene Ulrich, Significant Findings: The Quest of the Research Life, Cover Article, Notre Dame Magazine, Volume 12, Number 2, Pages 47–54, May 1983 132. Michael K. Sain, With Best Regards, Editorial IEEE Transactions on Automatic Control, Volume AC-28, Number 5, Page 545, May 1983 131. Michael K. Sain, Ten Thousand, and One, Editorial, IEEE Transactions on Automatic Control, Volume AC-28, Number 4, Pages 437–438, April 1983 130. Michael K. Sain, Feedback: A Modern Parable, Freimann Inaugural Address, April 28, 1983 129. Bostwick F. Wyman and Michael K. Sain, On the Zeros of a Minimal Realization, Journal of Linear Algebra and Its Applications, Special Issue on Linear Systems, Volume 50, Pages 621–637, 1983 128. Bostwick F. Wyman and Michael K. Sain, Internal Zeros and the System Matrix, Proceedings Twentieth Annual Allerton Conference on Communication, Control, and Computing, Pages 153–158, October 1982 127. T.A. Klingler, S. Yurkovich, and M.K. Sain, A Computer-Aided Design Package for Nonlinear Model Applications, Preprints Second Symposium on Computer Aided Design of Multivariable Technological Systems, International Federation of Automatic Control, Pages 345–353, September 1982 126. Michael K. Sain and R. Michael Schafer, A Computer-Assisted Approach to Total Feedback Synthesis, Proceedings American Control Conference, Pages 195–196, June 1982 125. Michael K. Sain and Joseph L. Peczkowski, Nonlinear Multivariable Design by Total Synthesis, Proceedings American Control Conference, Pages 252–260, June 1982 124. Michael K. Sain and Stephen Yurkovich, Controller Scheduling: A Possible Algebraic Viewpoint, Proceedings American Control Conference, Pages 261–269, June 1982 123. Thomas A. Klingler, Stephen Yurkovich, and Michael K. Sain, An Application of Tensor Ideas to Nonlinear Modeling of a Turbofan Jet Engine, Proceedings Thirteenth Annual Pittsburgh Conference on Modeling and Simulation, Pages 45–54, April 1982

Publications of Michael K. Sain 122.

121.

120.

119. 118.

117.

116.

115.

114.

113.

112. 111.

110.

109.

108. 107. 106.

349

Michael K. Sain, The State of Technical Manuscripts: Computer Assisted Measurements, Editorial, IEEE Transactions on Automatic Control, Volume AC-27, Number 2, Pages 293–294, April 1982 Michael K. Sain, The State of the Transactions: A New Constraint, Editorial, IEEE Transactions on Automatic Control, Volume AC-27, Number 1, Pages 1–2, February 1982 Michael K. Sain, P.J. Antsaklis, Bostwick F. Wyman, R.R. Gejji, and Joseph L. Peczkowski, The Total Synthesis Problem of Linear Multivariable Control, Part II: Unity Feedback and the Design Morphism, Proceedings IEEE Conference on Decision and Control, Pages 875–884, December 1981 P.J. Antsaklis and M.K. Sain, Unity Feedback Compensation of Unstable Plants, Proceedings IEEE Conference on Decision and Control, Pages 305–308, December 1981 B.F. Wyman and M.K. Sain, The Pole Structure of Inverse Systems, International Federation of Automatic Control, Preprints Eighth Triennial World Congress, Volume 3, Pages 76–81, August 1981 B.F. Wyman and M.K. Sain, Exact Sequences for Pole-Zero Cancellation, Proceedings International Symposium on the Mathematical Theory of Networks and Systems, Pages 278–280, August 1981 Michael K. Sain, Bostwick F. Wyman, R.R. Gejji, P.J. Antsaklis, and Joseph L. Peczkowski, The Total Synthesis Problem of Linear Multivariable Control, Part I: Nominal Design, Proceedings Twentieth Joint Automatic Control Conference, Paper WP-4A, June 1981 Michael K. Sain and Joseph L. Peczkowski, An Approach to Robust Nonlinear Control Design, Proceedings Twentieth Joint Automatic Control Conference, Paper FA3D, June 1981 Stephen Yurkovich, Thomas A. Klingler, and Michael K. Sain, Tensor Ideas for Nonlinear Modeling of a Turbofan Jet Engine: Preliminary Studies, Proceedings Twelfth Annual Pittsburgh Conference on Modeling and Simulation, Pages 1423–1428, May 1981 V. Seshadri (and M.K. Sain), Exterior Algebra and Simultaneous Pole-Zero Placement, Directed Research, Proceedings Conference on Information Sciences and Systems, Johns Hopkins University, Pages 478–483, May 1981 M.K. Sain, Status of Computer-Aided Control System Design For Feedback Design: Algebraic Design, GE-RPI-NSF Workshop on Control Design, May 20, 1981 Joseph L. Peczkowski and Michael K. Sain, Scheduled Nonlinear Control Design for a Turbojet Engine, Proceedings IEEE International Symposium on Circuits and Systems, Pages 248–251, April 1981 Michael K. Sain, On Linear Multivariable Control Systems, Guest Editorial, Special Issue on Linear Multivariable Control Systems, IEEE Transactions on Automatic Control, Volume AC-26, Number 1, Pages 2–3, February 1981 Bostwick F. Wyman and Michael K. Sain, The Zero Module and Essential Inverse Systems, IEEE Transactions on Circuits and Systems, Volume CAS-28, Number 2, Pages 112–126, February 1981 Michael K. Sain, Short Papers: A Denouement, Editorial IEEE Transactions on Automatic Control, Volume AC-25, Number 6, Pages 1025–1026, December 1980 B.F. Wyman and M.K. Sain, The Zero Module and Invariant Subspaces, Proceedings IEEE Conference on Decision and Control, Pages 254–255, December 1980 M.K. Sain, R.M. Schafer, and K.P. Dudek, An Application of Total Synthesis to Robust Coupled Design, Proceedings Eighteenth Annual Allerton Conference on Communication, Control, and Computing, Pages 386–395, October 1980

350

Publications of Michael K. Sain

105.

Stephen Yurkovich and Michael Sain, A Tensor Approach to Modeling of Nonhomogeneous Nonlinear Systems, Proceedings Eighteenth Annual Allerton Conference on Communication, Control, and Computing, Pages 604–613, October 1980 Michael K. Sain, Quotient Signal Flowgraphs: New Insights, Proceedings, IEEE International Conference on Circuits and Computers, Page 417, October 1980 R.R. Gejji (and M.K. Sain), Reliable Floating Point Computation of Minimal Bases, Directed Research, Proceedings Nineteenth Joint Automatic Control Conference, Paper WA-8B, August 1980 Michael K. Sain and Abraham Ma, Multivariable Synthesis with Reduced Comparison Sensitivity, Proceedings Nineteenth Joint Automatic Control Conference, Paper WP-8B, August 1980 Joseph L. Peczkowski and Michael K. Sain, Control Design with Transfer Functions: An Application Illustration, Proceedings Twenty-Third Midwest Symposium on Circuits and Systems, Pages 47–52, August 1980 Michael K. Sain, Theory and Application: A Common Ground?, Editorial, IEEE Transactions on Automatic Control, Volume AC-25, Number 3, Pages 345–346, June 1980 M.K. Sain and J.L. Peczkowski, Engine Decoupling: An Example of the Use of Inverse Systems in Frequency Domain Design, Workshop on the Mathematical Theory of Networks and Systems, May 1980 M.K. Sain and A. Ma, Engine Decoupling Revisited: An Example of Robust Design in the Frequency Domain, Workshop on the Mathematical Theory of Networks and Systems, May 1980 Michael K. Sain, Abraham Ma, and Daphne Perkins, Sensitivity Issues in Decoupled Control System Design, Proceedings Twelfth Southeastern Symposium on System Theory, Pages 25–29, May 1980 Stephen Yurkovich and Michael Sain, Generating Nonlinear Models from Digital Simulations: A Tensor Approach, Proceedings Eleventh Annual Pittsburgh Conference on Modeling and Simulation, Pages 797–802, May 1980 R.J. Leake, J.L. Peczkowski, and M.K. Sain, Step Trackable Linear Multivariable Plants, International Journal of Control, Volume 30, Number 6, Pages 1013–1022, December 1979 Bostwick F. Wyman and Michael K. Sain, Essential Right Inverses and System Zeros, Proceedings IEEE Conference on Decision and Control, Pages 23–28, December 1979 R. Michael Schafer and Michael K. Sain, CARDIAD Approach to System Dominance with Application to Turbofan Engine Models, Proceedings Thirteenth Annual Asilomar Conference on Circuits, Systems, and Computers, Pages 78–82, November 1979 V. Seshadri and M.K. Sain, Loop Closures and the Induced Exterior Map, Proceedings Seventeenth Annual Allerton Conference on Communication, Control, and Computing, Pages 753–761, October 1979 R. Michael Schafer and Michael K. Sain, Frequency Dependent Precompensation for Dominance in a Four Input/Output Theme Problem Model, Proceedings Eighteenth Joint Automatic Control Conference, Pages 348–353, June 1979 J.L. Peczkowski, M.K. Sain, and R.J. Leake, Multivariable Synthesis with Inverses, Proceedings Eighteenth Joint Automatic Control Conference, Pages 375–380, June 1979

104. 103.

102.

101.

100.

99.

98.

97.

96.

95.

94.

93.

92.

91.

90.

Publications of Michael K. Sain 89.

88.

87.

86. 85.

84.

83.

82.

81.

80.

79.

78.

77.

76.

75.

74.

73.

351

Michael K. Sain and R. Michael Schafer, Alternatives for Jet Engine Control, Proceedings Propulsion Controls Symposium, NASA Conference Publication 2137, Pages 129–138, May 1979 Michael K. Sain, Remarks on an Occasion of Editorial Transition, Editorial, IEEE Transactions on Automatic Control, Volume AC-24, Number 2, Pages 153–154, April 1979 V. Seshadri and M.K. Sain, An Application of Exterior Algebra to Multivariable Feedback Loops, Proceedings Conference on Information Sciences and Systems, Johns Hopkins University, Pages 337–342, March 1979 Michael K. Sain, On Exosubsets and Internal Models, Proceedings IEEE Conference on Decision and Control, Pages 1069–1073, January 1979 B.F. Wyman and M.K. Sain, Rings of Transfer Functions, 761st Meeting of the American Mathematical Society, College of Charleston, South Carolina, Page A665, November 3–4, 1978 R.M. Schafer and M.K. Sain, CARDIAD Design: Progress in the Four Input/Output Case, Proceedings Sixteenth Annual Allerton Conference on Communication, Control, and Computing, Page 567, October 1978 Michael K. Sain, The Quotient Signal Flowgraph for Large Scale Systems, IEEE Transactions on Circuits and Systems, Volume CAS-25, Number 9, Pages 781–788, September 1978 Michael K. Sain, Miniaturization of Large Criminal Justice Systems by Generalized Linear Signal Flow Graphs, Journal of Interdisciplinary Modeling and Simulation, Volume 1, Number 2, Pages 97–122, April 1978 J.L. Peczkowski and M.K. Sain, Linear Multivariable Synthesis with Transfer Functions, Alternatives for Linear Multivariable Control, M.K. Sain, J.L. Peczkowski, and J.L. Melsa, Editors, National Engineering Consortium, 1978, Pages 71–87 R.M. Schafer and M.K. Sain, Input Compensation for Dominance of Turbofan Models, Alternatives for Linear Multivariable Control, M.K. Sain, J.L. Peczkowski, and J.L. Melsa, Editors, National Engineering Consortium, 1978, Pages 156–169 M.K. Sain, The Theme Problem, Alternatives for Linear Multivariable Control, M.K. Sain, J.L. Peczkowski, and J.L. Melsa, Editors, National Engineering Consortium, 1978, Pages 20–30 R. Michael Schafer and Michael K. Sain, Some Features of CARDIAD Plots for System Dominance, Proceedings IEEE Conference on Decision and Control, Pages 801– 806, December 1977 Michael K. Sain and V. Seshadri, Pole Assignment and a Theorem from Exterior Algebra, Proceedings IEEE Conference on Decision and Control, Pages 291–295, December 1977 Michael K. Sain, Miniaturization of Large Criminal Justice Systems by Generalized Linear Signal Flow Graphs, Proceedings International Conference on Cybernetics and Society, Pages 320–328, September 1977 Michael K. Sain, The Quotient Signal Flow Graph for Large Scale Systems, Proceedings Twentieth Midwest Symposium on Circuits and Systems, Pages 527–531, August 1977 R. Gejji, R.M. Schafer, M.K. Sain, and P. Hoppner, A Comparison of Frequency Domain Techniques for Jet Engine Control System Design, Proceedings Twentieth Midwest Symposium on Circuits and Systems, Pages 680–685, August 1977 R.R. Gejji and M.K. Sain, Application of Polynomial Techniques to Multivariable Control of Jet Engines, Proceedings Fourth Symposium on Multivariable Technological Systems, International Federation of Automatic Control, Pages 421–429, July 1977

352 72.

71.

70.

69.

68.

67.

66.

65.

64.

63.

62.

61.

60.

59. 58. 57. 56.

Publications of Michael K. Sain R.M. Schafer, R.R. Gejji, P.W. Hoppner, W.E. Longenbaker, and M.K. Sain, Frequency Domain Compensation of a DYNGEN Turbofan Engine Model, Proceedings Sixteenth Joint Automatic Control Conference, Pages 1013–1018, June 1977 P.W. Hoppner (and M.K. Sain), The Direct Approach to Compensation of Multivariable Jet Engine Models, Directed Research, Technical Report 774, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 1, 1977 R.M. Schafer (and M.K. Sain), A Graphical Approach to System Dominance, Directed Research, Technical Report 772, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, April 1, 1977 J.M. Daschbach,..., M.K. Sain, et al., Computers in Courts, Encyclopedia of Computer Science and Technology, J. Belzer, A.G. Holzman, and A. Kent, Editors, Marcel Dekker, Inc., New York, Volume 6, 1977, Pages 350–382 R.R. Gejji (and M.K. Sain), A Computer Program to Find the Kernel of a Polynomial Operator, Directed Research, Proceedings Fourteenth Annual Allerton Conference on Circuit and System Theory, Pages 1091–1100, September 1976 V. Seshadri and M. Sain, An Approach to Pole Assignment by Exterior Algebra, Proceedings Fourteenth Annual Allerton Conference on Circuit and System Theory, Pages 399–407, September 1976 V. Seshadri and M.K. Sain, Interaction Studies on a Jet Engine Model by Characteristic Methodologies, Proceedings Nineteenth Midwest Symposium on Circuits and Systems, Pages 232–237, August 1976 Raghvendra R. Gejji and Michael K. Sain, A Jet Engine Control Problem for Evaluating Minimal Design Software, Proceedings Nineteenth Midwest Symposium on Circuits and Systems, Pages 238–243, August 1976 M.K. Sain, R.J. Leake, R. Basso, R. Gejji, A. Maloney, and V. Seshadri, Alternative Methods for the Design of Jet Engine Control Systems, Proceedings Fifteenth Joint Automatic Control Conference, Pages 133–142, July 1976 Anthony J. Maloney III (and M.K. Sain), Graphics Analysis of Dominance in Jet Engine Control Models, Directed Research, Technical Report 765, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, June 24, 1976 V. Seshadri and Michael K. Sain, Multivariable System Compensation Including Characteristic Methods for Jet Engine Control, Technical Report 763, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, June 24, 1976 Peter Hoppner and Michael Sain, Data Display Improvement for a Social System Simulator, Proceedings Seventh Annual Pittsburgh Conference on Modeling and Simulation, Pages 376–381, Part I, April 1976 Raghvendra R. Gejji and Michael K. Sain, Polynomic Techniques Applied to Multivariable Control of Jet Engines, Technical Report 761, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, March 22, 1976 Michael K. Sain, The Growing Algebraic Presence in Systems Engineering: An Introduction, IEEE Proceedings, Volume 64, Number 1, Pages 96–111, January 1976 Michael K. Sain, Minimal Torsion Spaces and the Partial Input/Output Problem, Information and Control, Volume 28, Number 2, Pages 103–124, October 1975 Michael K. Sain, An Exterior View of Linear Multivariable Systems, First International Symposium on Operator Theory of Networks and Systems, August 1975 Michael K. Sain, A Free-Modular Algorithm for Minimal Design of Linear Multivariable Systems, International Federation of Automatic Control, Sixth Triennial World Congress Proceedings, Part IB, Pages 9.1–1-9.1–7, August 1975

Publications of Michael K. Sain 55.

54.

53.

52.

51.

50.

49.

48.

47. 46.

45. 44.

43.

42.

41.

353

Michael K. Sain, Applications of Modern Algebra in Engineering: Introductory Lecture Notes, Technical Report 752, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 14, 1975; also published by the Department of Electrical Engineering, Texas Tech University, Lubbock, Texas, May 1975 Michael K. Sain and John J. Uhran, Jr., The Equivalence Concept in Criminal Justice Systems, IEEE Transactions on Systems, Man, and Cybernetics, Volume SMC-5, Number 2, Pages 176–188, March 1975 J.L. Massey and M.K. Sain, Inverses of Linear Sequential Circuits, Key Papers in the Development of Coding Theory, Elwyn R. Berlekamp, Editor, IEEE Press, 1974, Pages 205–212 Michael K. Sain, Eugene W. Henry and John J. Uhran, Jr., An Algebraic Method for Simulating Legal Systems, Simulation, Volume 21, Number 5, Pages 150–158, November 1973 Eugene W. Henry, John J. Uhran, Jr., and Michael K. Sain, Interactive Computer Simulation of Court System Delays, Socio-Economic Systems and Principles, William G. Vogt, Marlin H. Mickle, and H.E. Hoelscher, Editors, University of Pittsburgh School of Engineering Publication Series, Number 10, 1973, Pages 111–120 M.K. Sain, Review of Finite Dimensional Linear Systems, by R.W. Brockett, IEEE Transactions on Automatic Control, Volume AC-17, Number 5, Pages 753–754, October 1972 James L. Massey, Michael K. Sain, and John M. Geist, Certain Infinite Markov Chains and Sequential Decoding, Discrete Mathematics, Volume 3, Numbers 1, 2, 3, Pages 163–175, September 1972 M.K. Sain, Review of On Minimal Partial Realizations of a Linear Input/Output Map by R.E. Kalman, IEEE Circuit Theory Group Newsletter, Volume 6, Number 2, Page 15, June 1972 Michael K. Sain and David T. Link, A Study of Delay in Criminal Courts, Forty-First National Meeting, Operations Research Society of America, April 1972 Eugene W. Henry, John J. Uhran, Jr., and Michael K. Sain, Interactive Computer Simulation of Court System Delays, Proceedings Third Annual Pittsburgh Conference on Modeling and Simulation, Pages 89–98, April 1972 J.J. Uhran, Jr., M.K. Sain, E.W. Henry, and D. Sharpe, Computer Model of the Felony Delay Problem, IEEE International Convention Digest, Pages 310–311, March 1972 Lin-Nan Lee and Michael K. Sain, Case Studies on a Convergence Diagnostic for the LEADICS Simulation Algorithm, Electrical Engineering Memorandum 7201, University of Notre Dame, Notre Dame, Indiana, February 1972 L.G. Foschio, J.M. Daschbach, et al. (Including M.K. Sain), Systems Study in Court Delay: LEADICS, Law-Engineering Analysis of Delay in Court Systems, Law School and College of Engineering, University of Notre Dame, Notre Dame, Indiana, Volumes I–IV, January 1972; Volume I: Executive Summary, National Technical Information Service, U.S. Department of Commerce, 5285 Port Royal Road, Springfield, Virginia, Number PB-214 595 Stanley R. Liberty and Joseph K. Murdock (and M.K. Sain), Techniques for Determining the Density of a Random Integral Quadratic Form, Directed Research, IEEE Record Fifth Asilomar Conference on Circuits and Systems, Pages 502–508, November 1971 Michael K. Sain and Stanley R. Liberty, Performance Measure Densities for a Class of LQG Control Systems, IEEE Transactions on Automatic Control, Volume AC-16, Number 5, Pages 431–439, October 1971

354 40.

39.

38.

37.

36.

35. 34.

33.

32. 31.

30. 29. 28.

27.

26.

25.

24.

Publications of Michael K. Sain Michael K. Sain and Stanley R. Liberty, Performance Measure Densities for a Class of LQG Control Systems, Preprints Twelfth Joint Automatic Control Conference, Pages 695–703, August 1971 Stanley R. Liberty (and M.K. Sain), Existence and Uniqueness for a Class of SemiNorm Constrained Optimization Problems, Directed Research, Proceedings Fourteenth Midwest Symposium on Circuit Theory, Pages 9.1.1–9.1.5, May 1971 M.K. Sain and M.V. Maguire, A Nonsymmetric Riccati Equation of Linear Quadratic Gaussian Control, Proceedings Fifth Annual Princeton Conference on Information Sciences and Systems, Page 99, March 1971 Frederick H. Burrows and Michael K. Sain, A Viskovatov Technique for Numerical Approximation of Unknown Probability Density Functions, Second Annual Pittsburgh Conference on Modeling and Simulation, March 1971 Joseph K. Murdock and Stanley R. Liberty (and M.K. Sain), Probability Density Techniques for Integral Quadratic Forms, Directed Research, Proceedings Eighth Annual Allerton Conference on Circuit and System Theory, Pages 587–588, October 1970 Michael K. Sain, Inflation in Linear Dynamical Systems, Proceedings Eighth Annual Allerton Conference on Circuit and System Theory, Pages 151–160, October 1970 S.P. Panda, L.M. Silverman, M.K. Sain, and J.L. Massey, Comments on ‘Inversion of Multivariable Linear Systems’, IEEE Transactions on Automatic Control, Volume AC-15, Number 4, Pages 489–491, August 1970 Stanley R. Liberty (and M.K. Sain), An Application of Baggeroer’s Fredholm Integral Equation Technique - Determining Probability Densities of Integral Quadratic Forms, Directed Research, Electrical Engineering Memorandum 714, University of Notre Dame, Notre Dame, Indiana, August 1970 Michael K. Sain, Matrix Identities, IEEE Transactions on Automatic Control, Volume AC-15, Number 2, Page 282, April 1970 James L. Massey and Michael K. Sain, Derivative Controllability, Proceedings Fourth Annual Princeton Conference on Information Sciences and Systems, Page 189, March 1970 R. Saeks and M.K. Sain, Sensitivity Matrices for Multiparameter Networks, IEEE Digest International Symposium on Circuit Theory, Page 86, December 1969 Michael K. Sain and Richard Saeks, The Absolute Sensitivity Matrix, IEEE Proceedings Eighth Symposium on Adaptive Processes, Pages 5-b-1 - 5-b-3, November 1969 Michael K. Sain and James L. Massey, A Modified Inverse for Linear Dynamical Systems, IEEE Proceedings Eighth Symposium on Adaptive Processes, Pages 5-a-1 5-a-3, November 1969 Michael K. Sain and Stanley R. Liberty, Some New Computational Results for Minimum Variance Controllers, Proceedings Seventh Annual Allerton Conference on Circuit and System Theory, Pages 408–417, October 1969 M.K. Sain, Review of State Space Analysis, by L.K. Timothy and B.E. Bona, IEEE Transactions on Automatic Control, Volume AC-14, Number 4, Pages 436–437, August 1969 Michael K. Sain and James L. Massey, Invertibility of Linear Time-Invariant Dynamical Systems, IEEE Transactions on Automatic Control, Volume AC-14, Number 2, Pages 141–149, April 1969 J.L. Massey and M.K. Sain, Distribution of the Minimum Cumulative Metric for Sequential Decoding, Abstracts of Papers, IEEE International Symposium on Information Theory, Page 56, January 1969

Publications of Michael K. Sain 23.

22.

21.

20.

19.

18.

17. 16.

15.

14.

13.

12.

11. 10. 9.

8.

7.

355

R. Saeks and M.K. Sain, A Filter Theoretic Approach to the Multivariable Sensitivity Problem, Proceedings National Electronics Conference, Volume XXIV, Pages 98– 103, December 1968 James L. Massey and Michael K. Sain, Postscript to ‘Inverses of Linear Sequential Circuits’, IEEE Transactions on Computers, Volume C-17, Number 12, Page 1177, December 1968 James L. Massey and Michael K. Sain, Trunk and Tree Searching Properties of the Fano Sequential Decoding Algorithm, Proceedings Sixth Annual Allerton Conference on Circuit and System Theory, Pages 153–160, October 1968 Stanley R. Liberty and Michael K. Sain, Minimal Variance Feedback Controllers: Initial Studies of Solutions and Properties, Proceedings Sixth Annual Allerton Conference on Circuit and System Theory, Pages 408–417, October 1968 Michael K. Sain and Celso de Renna e Souza, A Theory for Linear Estimators Minimizing the Variance of the Error Squared, IEEE Transactions on Information Theory, Volume IT-14, Number 5, Pages 768–770, September 1968 Luis Cosenza and Michael Sain, On the Sufficiency of Non-Randomized Control Policies for Optimizing Certain Real Stochastic Systems, Proceedings National Electronics Conference, Volume XXIV, Pages 113–118, May 1968 James L. Massey and Michael K. Sain, Inverses of Linear Sequential Circuits, IEEE Transactions on Computers, Volume C-17, Number 4, Pages 330–337, April 1968 J.L. Massey and M.K. Sain, Codes, Automata, and Continuous Systems: Explicit Interconnections, IEEE Transactions on Automatic Control, Volume AC-12, Number 6, Pages 644–650, December 1967 Guilherme J. Binelli and Charles B. Silio, Jr. (and M.K. Sain), Study of a Multivariable Classical Sensitivity Design, Directed Research, Proceedings National Electronics Conference, Volume XXIII, Pages 175–180, October 1967 James L. Massey and Michael K. Sain, Codes, Automata, and Continuous Systems: Explicit Interconnections, Proceedings National Electronics Conference, Volume XXIII, Pages 33–38, October 1967 James L. Massey and Michael K. Sain, Inverse Problems in Coding, Automata, and Continuous Systems, IEEE Conference Record Eighth Annual Symposium on Switching and Automata Theory, Pages 226–232, October 1967 Michael K. Sain, Performance Moment Recursions, with Application to Equalizer Control Laws, Proceedings Fifth Annual Allerton Conference on Circuit and System Theory, Pages 327–336, October 1967 M.K. Sain, H.C. Chen, and D.K. Cheng, On ‘A Useful Matrix Inversion Formula and Its Applications’, IEEE Proceedings, Volume 55, Number 10, Page 612, October 1967 Michael K. Sain, Realization Errors in Open-Loop Control Programs, IEEE Transactions on Automatic Control, Volume AC-12, Number 5, Page 612, October 1967 Michael K. Sain, Functional Reproducibility and the Existence of Classical Sensitivity Matrices, IEEE Transactions on Automatic Control, Volume AC-12, Number 4, Page 458, August 1967 M.K. Sain and C.R. Souza, Linear, Minimum Variance, Squared Error Estimation: Motivation and Applications, Technical Report 6710, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, April 1967 M.K. Sain, A Sufficiency Condition for Minimum Variance Control of Markov Processes, Proceedings Fourth Annual Allerton Conference on Circuit and System Theory, Pages 593–599, October 1966

356 6.

5.

4.

3.

2.

1.

Publications of Michael K. Sain Michael K. Sain and Celso de Renna e Souza, A Projection Principle for Minimum Variance Estimation, Proceedings National Electronics Conference, Volume XXII, Pages 683–686, October 1966 M.K. Sain, Cost Free Variance Control of Discrete Linear Systems, Technical Report 666, Department of Electrical Engineering, University of Notre Dame, Notre Dame, Indiana, May 1966 M.K. Sain, Control of Linear Systems According to the Minimal Variance Criterion - A New Approach to the Disturbance Problem, IEEE Transactions on Automatic Control, Volume AC-11, Number 1, Pages 118–122, January 1966 M.K. Sain, On the Control Applications of a Determinant Equality Related to Eigenvalue Computation, IEEE Transactions on Automatic Control, Volume AC-11, Number 1, Pages 109–111, January 1966 Michael K. Sain, Relative Costs of Mean and Variance Control for a Class of Linear, Noisy Systems, Proceedings Third Annual Allerton Conference on Circuit and System Theory, Pages 121–129, October 1965 Michael Kent Sain, On Minimal-Variance Control of Linear Systems with Quadratic Loss, Ph.D. Thesis, Department of Electrical Engineering, University of Illinois, Urbana, January 1965

Index

(A, B)-invariant sub-semimodule, 135 (A, B)-invariant sub-semimodule of feedback type, 135 Γ -zero semimodule, 138 i-regular, 131 jth moment, 107 k cost cumulant, 99 k-regular, 132

bilinearization, 283 Bourne relation, 131 Brownian motion, 4

filtration, 67 independent increments, 44 probability measure, 67 realizations, 43 separable metric space, 67 sigma field, 44, 67, 68 stationary Wiener process, 44 composition of motions, 191 continuous dynamical system, 174 continuous with respect to the initial conditions, 195 continuous-time dynamical system, 174 controllability Gramian, 232 controllability sub-semimodule, 137 controllable sub-semimodule, 137 converse theorem, 190 cost moment control, 5 Critic, 202 cumulant third cumulant, 109, 111 two cumulant, 103 cumulant games, 14 cumulants, 4, 102 curse of dimensionality, 202

certainty equivalence principle, 94 process noise characteristics, 94 classical Lyapunov stability results, 177 cohomology of sheaves, 155 comparison theorem, 182 complete filtered probability space, 44, 67 Borel measurable, 68, 70 Borel sets, 67, 68 experiment, 67

DDS, see also discontinuous dynamical system, 174 decentralized fixed modes, 223 digital pre-distortion, 247 discontinuous dynamical system, 174 discrete-time dynamical system, 174 divided will of Saint Augustine, 322 dynamic programming, 50, 51, 53, 54, 80, 81, 84

ABET, 315 admissible mean cost, 103 admissible mean cost function, 105 algebraic Riccati equation, 206 algebraic transfer functions, 155 approximate decentralized fixed modes, 223, 228 approximate dynamic programming, 202 associated DDS, 185 asymptotically stable, 176 attractive, 176

358

Index

boundary condition, 54, 59, 83, 87 HJB equation, 53–55, 83–85 necessary conditions, 82 reachable set, 53, 55, 83 sufficient condition, 57, 82, 83, 89 value function, 53, 55, 56, 82–84, 88, 91 valuefunction, 51 verification theorem, 81, 83, 84, 89 dynamical system, 174 Dynkin formula, 101 enhanced iterative flipping algorithm, 251 feedback information structure, 80, 89, 93 finite dimensional dynamical system, 174 finite relative degree, 165 first characteristic function, 71, 72 moment-generating function, 71 first professional degree, 313 flexibility, 224 game theory, 67 prediction and prevention, 67 courses of action, 66, 94 decision making, 67 intelligent and irregular, 67 generalized homogeneous domination approach, 265 global inverses, 155 greedy iteration, 210 H-minimum phase, 163 higher level, 66 minimax strategies, 66, 92 multi-cumulant, Pareto and minimax strategy, 92–94 multi-cumulant, Pareto and Nash strategy, 89, 94 pessimistic situations, 67 self-enforcing Nash solutions, 66 Hilbert space, 44, 67, 68 square integrable processes, 44, 67, 68 history of engineering education, 314 homogeneous domination approach, 263 image, 131 image regular, 131 infinite dimensional dynamical system, 174 integral output feedback, 44, 49

invariant, 176 invertible system, 162 kth cost cumulant control, 13 Kalman, R. E., 29 Kalman, Rudy, 145 Lagrange multiplier, 31 left inverse, 162 linear-exponential-quadratic-Gaussian, 16 linear-quadratic-Gaussian, 4 linear-quadratic-regulator, 205 lower level, 66 Pareto parameterization, 69, 94 permissible Pareto decisions, 69, 70 team cooperative profiles, 69 team Pareto decisions, 69 Maclaurin series, 48, 74 Master of Engineering, 313, 314, 317 Mayer problem, 53 initial cost, 53, 80 Mayer form, 81, 84 minimal cost variance, 4, 7 minimal cost variance control, 7 feedback MCV control, 10 open loop MCV control, 9 minimum cost variance, 38 minimum mean, 38 mixed H2 /H∞ , 99 module of poles, 147 global, 149 module of poles at infinity, 148 module of zeros, 149 generic, 151 global, 150 module of zeros at infinity, 150 moment jth moment, 107 monoid, 130 morphism, 131 image, 131 kernel, 131 proper image, 131 motion, 174 Nash equilibrium, 66, 80, 81, 102 feedback Nash equilibrium, 65, 81, 83, 89, 92, 94

Index informational nonuniqueness, 80, 81 nonlinear decentralized output feedback control linear integrator systems, 272 power integrator systems, 277 nonlinear Volterra control, 281 observability Gramian, 233 orthogonal frequency division multiplexing, 245 partial motion, 190 partial transmit sequence, 250 peak-to-average power ratio, 246 performance measure, 43–46, 48, 60, 68–71, 75, 76, 93, 94 chi-squared type, 46, 66, 71, 77, 79 cost cumulants, 49 information statistics, 75, 79, 93, 94 integral-quadratic form, 68 non negative and monotone, 75 performance cumulants, 74, 75, 80 performance distribution, 46, 47, 52, 66 performance measure statistics, 43, 44, 49, 61 pole semimodule of input type, 134 pole semimodule of output type, 133 poles, fixed, 152 policy iteration algorithm, 207 pre-controllability sub-semimodule, 137 probability density space, 67, 77 mean, 66 probability density function, 65, 66, 80, 94 product space, 68 Cartesian product, 83 product mappings, 79 proper image, 131 proportional-integral controllers, 43 steady-state tracking error, 43 quadratic decision problems, 65 large-scale distributed systems, 65 quotient fixed modes, 226 quotient system, 227 recursion equation, 31 relative degree, 165 risk-sensitve control, 5, 7, 16

359

relationship between risk-sensitive and MCV control, 8 second characteristic function, 71 cumulant-generating equations, 75 cumulant-generating function, 71 natural logarithmic transformation, 71 terminal-value condition, 72, 74 time-backward differential equation, 72, 74 semigroup, 130 semimodule, 131 sub-semimodule, 131 semiring, 131 sequence exact, 131 proper exact, 131 sheaves, 155 short exact sequence, 131 situation awareness, 75 comprehension, 75 perception, 75 projection of future status, 75 relevant attributes, 75 state of knowledge, 75 solid state power amplifier, 247 stable, 176 state semimodule, 133 state space, 174 statistical approximation, 43 statistical control, 4, 43, 44, 49–54, 62, 66, 67 admissible feedback gains, 71, 79, 80, 92 affine input, 45, 46, 60, 62 cumulant-generating function, 46 decision laws, 68, 71, 80, 92, 93 decision process, 67, 70 decision states, 67, 68, 70, 71, 93 degrees of freedom, 52 design freedom, 65 feedback gain, 46, 52, 53, 58, 60–62 finite linear combination, 65, 79, 93 levels of influence, 79, 89, 93 moment-generating function, 46 multi-performance objectives, 62 optimization, 71, 81 performance distribution, 75, 89, 93 performance index, 43, 45, 50, 52, 55, 60, 71, 79, 80, 84, 89, 92

360

Index

performance uncertainty, 43 process noise, 44, 50 product mappings, 51 quadratic performance measure, 43 regulating performance, 48 stationary Wiener process, 67 stochastic uncertainty, 44 terminal-value conditions, 47, 48, 50, 53, 59, 61, 76, 77, 79, 81, 82, 88 time-backward differential equations, 47 time-backward evolutions, 50 time-backward histories, 61 time-backward matrix differential equations, 76, 77, 89, 93 time-backward trajectories, 51 tracking performance, 43 steady, 132 stochastic difference equation, 30 stochastic differential equation, 100 stochastic multi-team games, 65 conflicts of interests, 66 hierarchical structure, 66 informational decentralization, 66 simultaneous order, 66 stochastic regulator systems, 43 strongly connected, 227 sub-semimodule, 131 theology and engineering, 321, 329 third cumulant, 109, 111 time set, 174 total synthesis design, 151 total synthesis problem, 282 traveling wave tube amplifier, 247 two cumulant, 103 uncertainty analysis, 67 adversary tactics, 66, 67

performance uncertainty, 94 surprise attacks, 66 threat prediction, 65, 67 uniformly asymptotically stable, 176 uniformly asymptotically stable in the large, 177 uniformly exponentially stable, 44, 45, 70 state transition matrix, 44, 70 uniformly detectable, 46, 49, 60 uniformly stabilizable, 46, 49, 50, 60, 75–77, 89, 93 uniformly stable, 176 utility function, 20 verification theorem, 108 vibration suppression, 43 Volterra Kernel, 286 Volterra operators, 288 Volterra Representation, 282, 284 Wedderburn–Forney construction, 146, 154 weighted homogeneity, 259 will first model, 324 fourth model, 326 second model, 324 third model, 325 wind benchmark, 118 zero, 159 zero dynamics, 161 zero module, 161 zero semimodule, 138 zeros, 146, 149, 229 generic, 146 distance from transmission zeros, 229 transmission zero, 149 zero module, 149

Systems & Control: Foundations & Applications Series Editor Tamer Bas¸ar Coordinated Science Laboratory University of Illinois at Urbana-Champaign 1308 W. Main St. Urbana, IL 61801-2307 U.S.A. Systems & Control: Foundations & Applications Aims and Scope The aim of this series is to publish top quality state-of-the art books and research monographs at the graduate and post-graduate levels in systems, control, and related fields. Both foundations and applications will be covered, with the latter spanning the gamut of areas from information technology (particularly communication networks) to biotechnology (particularly mathematical biology) and economics. Readership The books in this series are intended primarily for mathematically oriented engineers, scientists, and economists at the graduate and post-graduate levels. Types of Books Advanced books, graduate-level textbooks, and research monographs on current and emerging topics in systems, control and related fields. Preparation of manuscripts is preferable in LATEX. The publisher will supply a macro package and examples of implementation for all types of manuscripts. Proposals should be sent directly to the editor or to: Birkh¨auser Boston, 675 Massachusetts Avenue, Cambridge, MA 02139, U.S.A. or to Birkh¨auser Publishers, 40-44 Viadukstrasse, CH-4051 Basel, Switzerland A Partial Listing of Books Published in the Series Representation and Control of Infinite Dimensional Systems, Vol. I A. Bensoussan, G. Da Prato, M. C. Delfour, and S. K. Mitter Representation and Control of Infinite Dimensional Systems, Vol. II A. Bensoussan, G. Da Prato, M. C. Delfour, and S. K. Mitter Mathematical Control Theory: An Introduction Jerzy Zabczyk

H∞ -Control for Distributed Parameter Systems: A State-Space Approach Bert van Keulen Disease Dynamics Alexander Asachenkov, Guri Marchuk, Ronald Mohler, and Serge Zuev Theory of Chattering Control with Applications to Astronautics, Robotics, Economics, and Engineering Michail I. Zelikin and Vladimir F. Borisov Modeling, Analysis and Control of Dynamic Elastic Multi-Link Structures J. E. Lagnese, G¨unter Leugering, and E. J. P. G. Schmidt First-Order Representations of Linear Systems Margreet Kuijper Hierarchical Decision Making in Stochastic Manufacturing Systems Suresh P. Sethi and Qing Zhang Optimal Control Theory for Infinite Dimensional Systems Xunjing Li and Jiongmin Yong Generalized Solutions of First-Order PDEs: The Dynamical Optimization Perspective Andre˘ı I. Subbotin Finite Horizon H∞ and Related Control Problems M. B. Subrahmanyam Control Under Lack of Information A. N. Krasovskii and N. N. Krasovskii H∞ -Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach Tamer Bas¸ar and Pierre Bernhard Control of Uncertain Sampled-Data Systems Geir E. Dullerud Robust Nonlinear Control Design: State-Space and Lyapunov Techniques Randy A. Freeman and Petar V. Kokotovi´c Adaptive Systems: An Introduction Iven Mareels and Jan Willem Polderman Sampling in Digital Signal Processing and Control Arie Feuer and Graham C. Goodwin

Ellipsoidal Calculus for Estimation and Control Alexander Kurzhanski and Istv´an V´alyi Minimum Entropy Control for Time-Varying Systems Marc A. Peters and Pablo A. Iglesias Chain-Scattering Approach to H∞ -Control Hidenori Kimura Output Regulation of Uncertain Nonlinear Systems Christopher I. Byrnes, Francesco Delli Priscoli, and Alberto Isidori High Performance Control Teng-Tiow Tay, Iven Mareels, and John B. Moore Optimal Control and Viscosity Solutions of Hamilton–Jacobi–Bellman Equations Martino Bardi and Italo Capuzzo-Dolcetta Stochastic Analysis, Control, Optimization and Applications: A Volume in Honor of W.H. Fleming William M. McEneaney, G. George Yin, and Qing Zhang, Editors Mutational and Morphological Analysis: Tools for Shape Evolution and Morphogenesis Jean-Pierre Aubin Stabilization of Linear Systems Vasile Dragan and Aristide Halanay The Dynamics of Control Fritz Colonius and Wolfgang Kliemann Optimal Control Richard Vinter Advances in Mathematical Systems Theory: A Volume in Honor of Diederich Hinrichsen Fritz Colonius, Uwe Helmke, Dieter Pr¨atzel-Wolters, and Fabian Wirth, Editors Nonlinear and Robust Control of PDE Systems: Methods and Applications to Transport-Reaction Processes Panagiotis D. Christofides Foundations of Deterministic and Stochastic Control Jon H. Davis Partially Observable Linear Systems Under Dependent Noises Agamirza E. Bashirov Switching in Systems and Control Daniel Liberzon

Matrix Riccati Equations in Control and Systems Theory Hisham Abou-Kandil, Gerhard Freiling, Vlad Ionescu, and Gerhard Jank The Mathematics of Internet Congestion Control Rayadurgam Srirkant H∞ Engineering and Amplifier Optimization Jeffery C. Allen Advances in Control, Communication Networks, and Transportation Systems: In Honor of Pravin Varaiya Eyad H. Abed Convex Functional Analysis Andrew J. Kurdila and Michael Zabarankin Max-Plus Methods for Nonlinear Control and Estimation William M. McEneaney Uniform Output Regulation of Nonlinear Systems: A Convergent Dynamics Approach Alexey Pavlov, Nathan van de Wouw, and Henk Nijmeijer Filtering Theory: With Applications to Fault Detection and Isolation Ali Saberi, Anton A. Stoorvogel, and Peddapullaiah Sannuti Representation and Control of Infinite-Dimensional Systems, Second Edition Alain Bensoussan, Giuseppe Da Prato, Michel C. Delfour, and Sanjoy K. Mitter Set-Theoretic Methods in Control Franco Blanchini and Stefano Miani Stability of Dynamical Systems: Continuous, Discontinuous, and Discrete Systems Anthony N. Michel, Ling Hou, and Derong Liu Control of Turbulent and Magnetohydrodynamic Channel Flows: Boundary Stabilization and State Estimation Rafael Vazquez and Miroslav Krstic Advances in Statistical Control, Algebraic Systems Theory, and Dynamic Systems Characteristics: A Tribute to Michael K. Sain Chang-Hee Won, Cheryl B. Schrader, and Anthony N. Michel Numerical Methods for Controlled Stochastic Delay Systems Harold J. Kushner