Linear Dynamical Systems
This is Volume 135 in MATHEMATICS IN SCIENCE AND ENGINEERING A Series of Monographs and Textbooks Edited by William F. Ames, Georgia Institute of Technology A list of recent titles in this series appears at the end of this volume.
Linear Dynamical Systems A revised edition of
DYNAMICAL SYSTEMS AND THEIR APPLICATIONS: LINEAR THEORY
John L. Iastl International Institute for Applied Systems Analysis Laxenburq, Austria
ACADEMIC PRESS, INC. HARCOURT BRACE JOVANOVICH, PUBLISHERS Boston Orlando San Diego New York Austin London Sydney Tokyo Toronto
Copyright © 1987, Academic Press, Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher.
ACADEMIC PRESS, INC. Orlando, Florida 32887
Library of Congress Cataloging-in-Publication Data
Casti, J. L. Linear dynamical systems. (Mathematics in science and engineering; v: 135) Rev. ed. of: Dynamical systems and their applications. 1977. Includes bibliographies and index. 1. Linear systems. I. Casti, J. L. Dynamical systems and their applications. II. Title. III. Series. QA402.C37 1986 003 86-17363 ISBN 0-12-163451-5 (alk. paper)
87 88 89 90 9 8 7 6 5 4 3 2 1 Printed in the United States of America
To the memory 0/ ALEXANDER MIKHAILOVICH LETOV Scholar, Gentleman, and Friend
This page intentionally left blank
(ontents
xi xiii
Preface to the Revised Edition Preface to the First Edition
Chapter 1
Basic Concepts, Problems, and Examples 1.1 Dynamical Systems, Inputs, and Outputs 1.2 Internal Description of ~ 1.3 Realizations 1.4 Controllability and Observability 1.5 Stability and Feedback 1.6 Optimality 1.7 Stochastic Disturbances Notes and References
Chapter 2
7 11
13 17 19
Mathematical Description of Linear Dynamical Systems 2.1 2.2 2.3 2.4 2.5 2.6
Chapter 3
1 3 6
Introduction Dynamical Systems External Description Frequency-Domain Analysis Transfer Functions Impulse-Response Function Notes and References
21 21 27 28 30 31 33
Controllability and Reachability 3.1 Introduction 3.2 Basic Definitions
35 36
vii
viii
CONTENTS 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11
Chapter 4
58
61 62 64
68
Introduction Basic Definitions Basic Theorems Duality Functional Analytic Approach to Observability The Problem of Moments Miscellaneous Exercises Notes and References
72 73 75 81 82
83
84 85
Structure Theorems and Canonical Forms 5.1 5.2 5.3 5.4 5.5 5.6 5.7
Chapter 6
39 43 47 52 55 57
Observability/Constructibility 4.1 4.2 4.3 4.4 4.5 4.6
Chapter 5
Time-Dependent Linear Systems Discrete-Time Systems Constant Systems Positive Controllability Relative Controllability Conditional Controllability Structural Controllability Controllability and Transfer Functions Systems with a Delay Miscellaneous Exercises Notes and References
Introduction State Variable Transformations Control Canonical Forms Observer Canonical Forms Invariance of Transfer Functions Canonical Forms and the Bezoutiant Matrix The Feedback Group and Invariant Theory Miscellaneous Exercises Notes and References
88 90 91
97 99 101 104
111 114
Realization Theory 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9
Introduction Algebraic Equivalence and Minimal Realizability Construction of Realizations Minimal Realization Algorithm Examples Realization of Transfer Functions Uniqueness of Minimal Realizations Partial Realizations Reduced Order Models and Balanced Realizations Miscellaneous Exercises Notes and References
117 118
124 127 128 131 132
133 138
140 145
CONTENTS
Chapter7
IX
Stability Theory 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9
Chapter8
147 149 152 156 162 164 169 173 174 177 179
The Linear-Quadratic-Gaussian Problem 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 8.10 8.11 8.12 8.13 8.14 8.15
Chapter 9
Introduction Some Examples and Basic Concepts Routh-Hurwicz Methods Lyapunov Method Frequency-Domain Techniques Feedback Control Systems and Stability Modal Control Observers Structural Stability Miscellaneous Exercises Notes and References
Motivation and Examples Open-Loop Solutions The Maximum Principle Some Computational Considerations Feedback Solutions Generalized X - Y Functions Optimality versus Stability A Low-Dimensional Alternative to the Algebraic Riccati Equation Computational Approaches for Riccati Equations Structural Stability of the Optimal Closed-Loop System Inverse Problems Linear Filtering Theory and Duality The Separation Principle and Stochastic Control Theory Discrete-Time Problems Generalized X - Y Functions Revisited Miscellaneous Exercises Notes and References
182 185 187 190 192 196 204
214 216 219 220 227 231 233 234 235 240
A Geometric-Algebraic View of Linear Systems 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 9.10 9.11 9.12
Algebra, Geometry, and Linear Systems Mathematical Description of a Linear System The Module Structu~e of n r, and X Some System-Theoretic Consequences Transfer Functions Realization of Transfer Functions The Construction of Canonical Realizations Partial Realizations Pole-Shifting and Stability Systems over Rings Some Geometric Aspects of Linear Systems Feedback, the McMillan Degree, and Kronecker Indices
246 247 249 253 257 260 263 271 273 274 278 283
x
CONTENTS 9.13 9.14 9.15 9.16
Some Additional Ideas from Algebraic Geometry Pole Placement for Linear Regulators Multivariable Nyquist Criteria Algebraic Topology and Simplicial Complex of I: Miscellaneous Exercises Notes and References
285 288 291 292 298 310
Chapter 10 Infinite-Dimensional Systems 10.1 10.2 10.3 10.4 10.5 10.6 10.7
Index
Finiteness as a System Property Reachability and Controllability Observability and Duality Stability Theory Realization Theory The LQG Problem Operator Riccati Equations and Generalized X - Y Functions Miscellaneous Exercises Notes and References
317 319
323 325
327 330
332 335 345
347
Preface 10 the Revised Edition
When the first edition of this book, (formerly titled Dynamical Systems and their Applications: Linear Theory) was published in 1977, it presented a reasonably thorough account of the major ideas and results of linear system theory circa mid-1970s. The past decade has witnessed an explosion of interest in mathematical system theory with major advances in the understanding of systems governed by functional differential equations, n-d systems, system identification, numerical methods, and frequency-domain techniques, not to mention the creation of an elegant and comprehensive algebraic and geometric theory of linear systems. And this is just for linear systems; much more can be said about new results in bifurcation theory, chaos, fractals, and other nonlinear phenomena, some of which is treated in my volume, Nonlinear System Theory (Academic Press, 1985), a companion to the present work. The task of doing justice to this impressive body of new work in a hundred pages or less was an imposing one, and one which ultimately required many compromises. Since a detailed account of all major developments was totally out of the question, I decided to treat a few topics in some depth and then to incorporate most of the remaining results by way of augmented references and problems in the earlier chapters. Thus, the current volume includes chapter-length expositions of the algebraic and geometric theory of linear systems (Chapter 9) and the theory of infinite-dimensional systems (Chapter 10). The other topics are interwoven into Chapters 1-8, together with the correction of a number of unfortunate typos, numerical errors and, in one or two places, just plain erroneous results that marred the first edition of this book. xi
xu
PREFACE TO THE REVISED EDITION
At this time, I would like to record my thanks to numerous colleagues, friends and students who were kind enough to show me the error of my ways in the book's first edition and who served as a source of inspiration to prepare this much more comprehensive work. Their efforts and support have shaped both the content and form of this volume. John L. Casti Vienna January 1986.
Preface to the First Edition
A spin-off of the computer revolution, affecting all of modern life, is the pseudoacademic discipline of "systems analysis." The words themselves are sufficiently vague to be transferable to almost any situation arising in human affairs, yet precise enough to suggest a scientific content of sufficient depth to convince the uninitiated of the validity of the particular methodology being promoted by the analyst. The impression is often created that performing a "systems analysis" of a given situation will remove all subjectivity and vagueness from the problem, replacing fallible human intuition by objective, rational, often mechanized, "scientific" policies for future action. Valid as the above argument is for some situations, we must object to its usual mode of proof by contradiction. Implicit in the verbiage spewing forth on the general topic of systems analysis is the assumption that underlying any such analysis is a system theory whose results support the analyst's conclusions and recommendations. Unfortunately, our observations have led to the conjecture that the majority of individuals practicing under the general title of "system analyst" have very little, if any, understanding ofthe foundational material upon which their very livelihood is based. Furthermore, when this fact is even casually brought to their attention, a typical human defense mechanism is activated to the extent that the subject is brushed off with a remark such as, "Well, system theory has not yet progressed to the point where practical problems can be treated, so what are we to do when the real world demands answers?" Unfortunately, there is a germ of truth in this standard reply; but in our opinion, such a statement has an even stronger component of prejudice seeking rationality since, as noted, a majority of analysts are in no position to speak with authority as to how far system theory actually has progressed and xiii
xiv
PREFACE TO THE FIRST EDITION
what the current results do say about their problems-hence, a partial motivation for this book. While it must be confessed that, like good politics, good systems analysis is the art of the possible, it is ofprimar importance that a practitioner have a fairly clear idea of where the boundary currently lies separating the science (read: established theory) from the art (read: ad hoc techniques). In this book we address ourselves to basic foundational and operational questions underlying systems analysis, irrespective of the context in which the problem may arise. Our basic objective is to answer the question: how can mathematics contribute to systems analysis? Regarding a system as a mechanism that transforms inputs (decisions) into outputs (observations), we shall examine such basic issues as: (i) How can one construct an explanation (model) for a given input! output sequence and, if several models are possible, how can we obtain the "simplest" model? (ii) With a given model and a fixed set of admissible inputs, how can we determine the set of possible behavioral modes of the system? (iii) With a prescribed mode of observing the behavior of a process, is it possible to uniquely determine the state of the system at any time? (iv) If a criterion of performance is superimposed upon a given process, what is the best value that this criterion can be made to assume, utilizing a given set of admissible inputs?
Clearly, the above questions are far-ranging and no complete answers are likely to be forthcoming in the near future as long as we speak in such general terms. Consequently, we lower our sights in this volume and confine our attention to those systems for which there is a linear relation between the system inputs and outputs. Not only does this provide a structural setting for which a rather comprehensive theory has been developed, it also enables us to confine our mathematical pyrotechnics to a level accessible to anyone having a nodding acquaintance with linear differential equations and elementary linear algebra. Briefly speaking, the book is divided into four basic parts: introductory, structural, modeling, and behavioral. The introductory chapters (1-2) give an overview of the topics to be covered in depth later, provide motivation and examples of fundamental system concepts, and give reasonably precise definitions upon which further results are based. The structural chapters (3-5) introduce the important concepts of controllability, observability, and canonical forms. Here we find detailed information concerning the restrictions on system behavior that are imposed by purely structural obstacles associated with the way in which the system is allowed to interact with the outside world. Furthermore, such obstructions are made evident by develop-
PREFACE TO THE FIRST EDITION
xv
ment of canonical forms explicitly devised to make such system properties apparent, almost by inspection. With a firm grasp of the structural limitations inherent in a given system, the modeling chapter (6) addresses itself to the question of actually constructing a model (realization) from given input/ output behavior. A number of algorithms for carrying out such a realization are presented, and extensive attention is given to the question of how to identify "simple" models. Finally, in the behavioral chapters (7-8) we analyze questions of system dynamics. Problems associated with the stability of system behavior under perturbations of the operating environment are treated, together with the problem of choosing admissible inputs that "optimize" a specific criterion function. A student who masters the material of this volume will be well prepared to begin serious system-theoretic work either in the applications area or in graduate research in systems-related disciplines. Saratoga, California August 1976
This page intentionally left blank
CHAPTER
1
Basic (oncepls, Problems, and Examples
1.1 DYNAMICAL SYSTEMS, INPUTS, AND OUTPUTS
To create an axiomatic theory of systems, dictates of logic demand the introduction of various definitions and terminology restricting the universe of discourse to a well-defined class of objects, which may then be analyzed according to the usual methods and tools of mathematics and logic. However, to anyone who has had occasion to practice in the field of" systems analysis," such a program seems self-defeating from the very outset because an attempt to verbalize precisely what one means by the intuitively understood entity "system" seems immediately to restrict the terms to such a degree that the next" system" problem encountered no longer fits within the confines of the definition. Consequently, we shall forego definition of such a vague, but basic, concept and restrict our attention in this book to the most important subset of the class of general systems-the linear systems. Having shifted the weight from one foot to the other by introducing the qualifying adjective linear, what do we mean by a linear system? Intuitively, we think of a linear system 1: as a machine that transforms inputs to outputs in a linear way. Referring to Fig. 1.1, we envision 1: as a "machine initially in a neutral or zero state, having m input terminals, and p output terminals." At any time t, a certain signal may be presented to the input terminals, the system operates on (transforms) the given input signal, and a response is observed at the output terminals. By assuming that the inputs and outputs belong to sets in which the operations of addition and scalar multiplication
2
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES 1---------, ,
f
,
1
INPUTS 2
,p
:
'":
:
L
,
~'-{)2 '1
OUTPUTS
J
L
FIG. 1.1 The system ~.
are defined, the linearity of I: means that the input/output mapping is linear, i.e., if we let n be the set of admissible inputs, r the set of outputs, and
f:
n -+ r
(1.1)
the rule for transforming inputs into outputs, i.e., the input/output map of I:, then I: is linear if and only if
for any scalars ex, f3, and inputs WI> W2 E n. (Remark: The usual interpretation of nand r is that they are sequences of vectors from the vector spaces R" and RP, respectively, with the scalar being real numbers. As will be pointed out later, however, there is no need to be so restrictive as to always choose the real number field R since the same theory applies to any field k. This added generality can be critical in some applications of the theory, e.g., coding theory and systems governed by functional-differential equations). To illustrate the basic concepts, we consider a simple example. Example:
Industrial Production
Suppose we have three industries with output rates Xl' X2, X3' respectively. Say the outputs are measured in dollars per year. The output of each industry is used by itself, by the other two industries, and by the rest of the world-the consumers. Let aij be the value of product i required as input to produce one dollar's worth ofproductj, where i,j = 1,2,3. Let Yi be the rate at which consumers absorb the output of industry i, where i = 1,2,3. On the basis of these definitions, we have the relations 3
Xi
=
L
aijxj j= 1
+ Yi'
or, in vector-matrix form, X
= Ax
i = 1,2, 3,
+ y.
We may use the above model to determine the amount of industrial production x required to meet a given consumer demand Y, provided the
1.2
INTERNAL DESCRIPTION OF
1:
3
"technological coefficient" matrix A is known. Extensions and generalizations of the above setup form the foundation of what is usually termed "input/output" analysis in the economics literature. Often the matrix A is termed a "Leontief" matrix in honor of the founder of this branch of mathematical economics. In the language of our earlier discussion, this example has 0 = R 3 = F, f = (I - A) - 1, and the physical interpretation of the "machine" is that it is an industrial complex which transforms consumer demands into industrial products. The description of 1: given above is useful in some circumstances but is still quite limited. Among the important factors that such a setup omits are dynamical changes, stochastic effects,and, most importantly, the mechanism by which 1: transforms inputs to outputs. If the map f is interpreted as arising from physical experiments, the system analyst would like to know the "wiring diagram" of 1:, indicated by the part of 1: within dashed lines in Fig. 1.1, and not just the "black-box" behavior represented by f Consequently, we turn our attention to a description of 1: that allows us to deal with these questions.
1.2 INTERNAL DESCRIPTION OF 1:
To overcome the difficulties cited above, we introduce the concept of the "state" of 1: as a mathematical entity that mediates between the inputs and outputs, i.e., the inputs from 0 act on the state, which in turn generates the outputs in r. At this point it is important to emphasize the fact that the state, in general, is not a quantity that is directly measurable; it is introduced merely as a mathematical convenience in order to inject the notions of causality and internal structure into the description of 1:. There has been much confusion in the system modeling literature on this point and the source of much of the misunderstanding may be traced to a lack of attention to this critical point. The only quantities that have physical meaning are those that we can generate or observe, namely, the inputs and outputs. Another interpretation of the state (which is somewhat more intuitively satisfying) is that it is an amount of "information" which, together with the current input, uniquely determines the state at the next moment of time. Of course, this is a circular definition but does convey the intuitive flavor of the "state" concept. As noted above, it is probably safest just to regard the state as a mathematical construct without attaching any particular physical interpretation to it.
4
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
At this point we impose an additional assumption on :E. We assume that :E is finite-dimensional, i.e., that there exists a finite-dimensional vector space X of dimension n such that the following diagram is commutative:
f
Since nand r are spaces of m- and p-dimensional vector functions, respectively,the linear transformations G and H may be identified with n x m and p x n matrices, respectively. In order to account for the change in :E over time, we must also assume a linear transformation
F: X
-+
X
describing the law of motion in X for :E if no inputs are presented. Clearly, F may be identified with an n x n matrix. Putting these notions together, the internal (or state variable) description of :E is given by the system of differential equations
=
x(t)
F(t)x
+ G(t)u,
y(t) = H(t)x(t) .
(1.2)
in continuous time, or by the difference equations x(t
+
1)
=
F(t)x
y(t) = H(t)x
+ G(t)u,
(1.3)
in discrete time, where x E X", U E urn, y E yP, F, G, H, being n x n, n x m, p x n matrices, respectively. Here we have made the identifications U = B", Y = RP, and X = R", which we shall retain throughout the book, where U and Yare the spaces of input and output values, respectively. The connection between the internal description of E given by Eqs. (1.2) [or (1.3)] and the earlier external description of Eq. (1.1) is fairly clear. The input u(t) at a given time t is presented to:E and an output y(t) is observed. The external description/maps u(t) -+ y(t). On the other hand, in the internal description, y(t) is produced from u(t) by means of the differential (difference) equations (1.2) [(1.3)]. The internal description seems to contain far more information about :E than does the external description. In addition, the
1.2 INTERNAL DESCRIPTION OF 1:
5
dynamical behavior of 1: is easily accounted for through the concept of the state x and Eqs. (1.2) or (1.3) which governs its temporal behavior. Consider the following example of an internal description of 1:.
Example:
Water Reservoir Dynamics
The system is shown in Fig. 1.2, where rt(t), r2(t) are the rainfall inputs, Xt(t), X2(t), X3(t) the states of surface storage at locations 1-3 respectively, while the state of groundwater storage (including infiltration) is X4(t). The constant k is for surface water flow, while It and 12 are for infiltration. The
--@
r,ll l
RAINFALL INPUTS
u (t)
1,1"
.
~Y211)= .-~I
4
3
II)~
"J
£7
4,.
r2111~
l x -)(
Y,
II)
STREAMFLOW OUTPUT
\,;
9n u2 (l)
FIG. 1.2 Water reservoir network.
expression 13(X4 - X3) signifies the exchange between stream and groundwater. The outputs Yt, Y2 are the streamflow output and the contribution of groundwater to the streamflow respectively, and the quantities gttUt and gnu2 denote the water release. The continuity equations for this problem are
+ rt x2 = -12 x2 + r2 Xt = -/txt
x3 = 13(X4 -
:<4 = t,»,
+
gllUt,
gnu2, X3) - kX 3 + gttUt + gn u2 12 X2 - 13(X4 - X3)'
The outputs are Yt = kX3' Y2 = 13(X4 - X3)'
In vector-matrix form, we have .i(t) = Fx + Gu y(t) = Hx,
+
r(t),
6
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
where
F=
f-~'
-
0 11 -gll
G_
0
f
0 0
0 -1 2 0 12
~1l
-(k
+ 13 )
13
0o
I
_3 /3
]
'
0]
-g22
H =
~22'
[~ ~
~J
1.3 REALIZA nONS
The internal and external descriptions of a system give rise to the following basic questions: Given an external description off, is it always possible to find an internal, i.e., differential or difference equation, description whose input/output behavior matches)"? If yes, is this description unique and. if not, in what sense is it not unique? Obviously, these are basic questions in model building and a substantial part of the system theory literature is devoted to an analysis of these problems. . It is a happy and fortunate fact that the answer to the first question, the so-called "realization" problem, is affirmative. As we shall see in a later chapter, the construction of an internal description from a given/is actually rather easy and, in general, there are many such realizations. The problem only begins to be interesting when we impose the additional constraint that the dimension of the state space of the realization be as small as possible. On physical grounds this requirement is entirely sensible as any realization having a higher-dimensional state space contains ..something" that is not implied by the input/output map! But, since/is the only information at our disposal with which to construct the realization, it follows that these extra pieces have been imposed extraneously by hypotheses and prejudices having no relevance to the physical process generating! It will turn out that the added requirement on the dimension of the realization will eliminate the nonuniqueness in the realization (up to a change of coordinate system in the space X). Thus to each linear input/output map there corresponds a unique triple (F, G, H) ofthe smallest state dimension
1.4
7
CONTROLLABILITY AND OBSERVABILITY
realizing! Of course, the converse is trivially true also, i.e., given (F, G, H), we may uniquely constructfas the map
f: u(t) -+
f~HeF('-')Gu(S)
in continuous time and
ds
,
f: u(t) -+
L HF'-iGu(i)
i=O
in discrete time (assuming x(O) = 0). 1.4 CONTROLLABILITY AND OBSERVABILITY
The central theme that distinguishes modern mathematical system theory from classical dynamical system theory is the idea of influencing the behavior of 1: by means of inputs that are at a decision maker's disposal. Thus, rather than adopting the passive observer role of the classical theory and contenting ourselves with describing the motion of a system upon which we are able to exert no influence whatsoever, we adopt an activist point of view and regard the behavior of the system as being modifiable, subject to constraints on the manner in which interactions are allowed. These two entirely different philosophies sharply color the types of questions asked about the system, and it is clear that the activist approach must subsume the classical theory since doing nothing (no input) is always an allowable action. One of the most important aspects of a given system subject to inputs from a set n is its set of reachable states. In other words, if we assume a fixed initial state Xo, what states may be reached in any finite future time by application of input sequences from n. In the event that the entire space X is reachable, we say that 1: is completely reachable. Clearly, if 1: is given in internal form, reachability will be determined only by the matrices F and G and the set n. That the concept is not empty may be illustrated by the trivial system
X1 .= X2 =
Xl' X2
+ u(t),
in which case only states of the form Xl = 0, X 2 = 0:, Q( arbitrary, are reachable from the origin for, say, piecewise-continuous inputs u(t), t > O. A concept complementary to reachability is controllability in which we look at the question of which nonzero states may be driven to the origin in a finite time. For constant F, G (and continuous time), these two concepts are equivalent in the sense that if a state is controllable, it is reachable and conversely. For time-dependent F, G, however, the two concepts are entirely
8
I
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
independent as we shall examine in detail later. We also note that the condition of reachability/controllability in finite time may be replaced (for continuous time, linear systems with unbounded inputs) by arbitrarily short time since linearity ensures that any control that sends a state to the origin in time T may be modified to send the state to the origin in time T]«, where ex can be arbitrarily large, but finite. As an illustration of the reachability concept, we consider the growth of an economy. Example: Economic Planning
Assume that the government balances the budget so that Kp(t
Kit
+ +
1) = Kp(t) 1) = Kg(t)
+ Sp(t), + X*(t),
where Kp(t) is the private sector capital, Kg(t) the government (public sector) capital, Sp(t) the private sector savings, and X*(t) the total tax receipts of the government. We assume that the government uses two taxes: one for savings US(t) and one for consumption UC(t) and the private sector saves a fraction s(t) of disposable income. Thus Sp(t) = s(t)(1 - US(t»Y(t),
where Y(t) is the national output assumed to be obtained from a neoclassical production function !F as Y(t) = !F(Kp(t), Kg(t), (1,+ <5)IL(t»,
where L(t) is the labor force, and (I + <5) is a technical progress factor, <5 > 0, augmenting labor. Consequently, X*(t) = [US(t)s(t)
+ uC(t)(1
- s(t))]Y(t),
since consumption is C(t) = (I - UC(t»(l - s(t»Y(t).
We further assume that labor supply grows at a rate n such that L(t)
(I
= (I +
n)lL(O).
If we define lower case variables by multiplying the upper case variables by + y)-I, where 1 + Y = (I + <5)(1 + n), we obtain the discrete-time model
+ kg(t +
kp(t
I) = (l
I) = (l
+ y)-l[k p(t) + s(t)(1 - US(t»y(t)], + y)-l[k g(t) + (US(t)s(t) + uC(t)(1 -
s(t»)y(t)].
1.4
CONTROLLABILITY AND OBSERVABILITY
9
To avoid ambiguity in timing, assume that at the beginning of each period the government announces its "controls" (taxes), then consumers choose their savings rate s(t), i.e., retroactive tax legislation is forbidden. Upon fixing the production function iF (and thereby obtaining y(t)), the above model becomes a nonlinear description of a simplified economy. Controls are admissible if and only if 0 ~ US(t), UC(t) ~ 1, for all t. The basic reachability question, of course, is what levels of private and public sector capital are obtainable at any time by means of tax rates us, u". Presumably the government has certain acceptable levels and others that are desirable, and a fundamental concern is whether either of these sets is reachable from the initial level and, since the system is nonlinear, if these levels are reachable, then how long does it take and what are the tax rates that move the economy into satisfactory regions of state space. Linearized versions of similar processes will be taken up in more detail later. Another question of some concern in system problems is whether or not a given output measurement can arise from more than one internal state, i.e., if y(tt) = y(t 2 ), does this imply that x(tt) = x(t 2 )? (Recall y(t) = Hx(t).) An alternative way of stating this problem is: Given the observation y(t) and knowledge of all future inputs u(s), s ~ t, are all future states x(s) uniquely determined for s > t? If yes, then we say the state x(t) (inverse image of y(t) relative to H) is observable. If the set of observable states is all of X, then k is said to be completely observable. Clearly, this concept involves only the matrices F and H. Just as in the case of reachability, for time-dependent F, G, H, the concept of observability must be augmented by the closely related, but totally independent, notion of constructibility, A state x is said to be constructible at time t ifit is possible to identify xuniquely by means ofthe output measured up to time t under known input, i.e., the past output uniquely determines the current state. As in the case of controllability/reachability, the two concepts observability/constructibility are equivalent for constant linear systems but diverge in the time-varying case. Given the similarities in definition, one might ask whether these input/output concepts pair off in any sort of duality relationship. It turns out that the relevant pairings are reachability/observability and controllability/constructibility. Since this fact is one of the cornerstones of modern system theory, let us examine the situation in more detail. One of the fundamental results of linear system theory is that the two concepts of reach ability and observability are dual notions in the strict sense of duality of vector spaces. In other words, any result about reachability may be immediately translated into one about observability, and conversely by means of well-defined vector space operations. Initially this
10
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
seems quite surprising since, on the surface, the two concepts seem quite separate. However, when we return to our pictorial representation of I: in Fig. 1.1 and note that the describing equations (1.2) still make perfectly good mathematical sense if time is reversed, i.e., t -+ - t, then we are immediately led to conjecture that there should be no essential difference in the behavior of I: provided we interchange the roles of the inputs and outputs and let time run "backwards." Roughly speaking, this simple observation is the duality principle, which we shall explore in more detail later. On practical grounds there is some cause for concern if a system under study has either uncontrollable or unobservable states since such a situation is an indication that there are parts of the model that are forever inaccessible to outside influences and/or measurements. Obviously, this state of affairs must be viewed with some concern since such uncontrollable/unobservable pieces of I: can play no role in our interaction with the system. As we shall prove in a later chapter, if I: arises as a realization of some physically obtained input/output map f, then the minimal dimension requirement will ensure that the system will be completely controllable and observable. In fact, the two conditions are equivalent. Thus no "extraneous" pieces will occur in the internal description of a I: arising as the end result of a minimal realization process. If, however, I: is originally given in internal form, then the extra pieces must be identified (and usually eliminated) before undertaking further analysis on the system. Example: Population Migration
To illustrate a system that contains uncontrollable, as wellas unobservable, parts consider a country consisting of three regions whose population the national government influences by the creation of jobs, Let Xl' X 2, X 3 denote the population in the three regions, each having a natural growth rate measured by the coefficient aii' The government-generated jobs giU are the inputs. Assume the dynamics have the form Xl
=
qllxl,
X2 =
x3
a22 x2
=
a33 x3
+ g2 U' + g3 U.
We see that region 1 is not influenced by the government job policy. If the government measures the national population in such a way that a fraction Ci is associated with region i, then we might have an output of the form y=
ClXl
+ C3 X3'
In this case, the total population estimate is not influenced by region 2.
1.5
II
STABILITY AND FEEDBACK
The conclusion to be drawn from this trivial example is that region 1 is completely unaffected by government action, while if such action is based on an estimate of the total population, then the success or failure of a given policy in region 2 will never be noted. Thus region 1 is an uncontrollable, while region 2 is an unobservable, part of the process. For purposes offuture study, it would be wise for government analysts to drop both of these areas from their model or to modify their control and observation structure. 1.5 STABILITY AND FEEDBACK
Classically, the question of whether a given system is stable under small perturbations in either 1: itself or in the initial data has given rise to an enormous literature which is still far from giving a complete picture of the situation. Since we deal almost exclusively with linear systems in this book, our primary concern will be with perturbations of the initial data and asymptotic stability in the sense of Lyapunov (definitions later). Roughly speaking, we conceive of the origin as being the desired state of 1: and ask whether an initial disturbance x(O) = Xo =F 0 will damp out with increasing time, i.e., does x(t) ~ 0 as t ~ oo? Obviously, this question has several parts depending on whether we are operating in a passive (u == 0) or an active mode. In the first case, the whole issue depends solely on the location of the characteristic values of F (and the rate of growth of the components of F, if F is time-varying) for which one of the algebraic criteria such as Routh-Hurwicz, Lienard, Lyapunov, etc. will rapidly settle the issue. On the other hand, if interaction through inputs u(t) is allowed, then the stability question is closely tied to the controllability concept described above. For example, the system Xl
=
Xz =
Xl(O) =
Xl> Xl -
Xz + u,
xz(O)
a =F 0,
= p,
cannot be asymptotically stable in the sense of Lyapunov for any control u since the component Xl is uncontrollable and diverges to ±oo as t ~ 00. The similar looking system Xl
=
Xz =
Xl Xl -
+ U, Xz,
Xl(O) =
xz(O) =
a,
p,
may, however, be stabilized by choosing, for example, u(t) = -2x l(t). The difference between these two examples, of course, is that the first is not completely controllable while the second is. Thus, by our previous remarks, we can see that the second system not only may be driven to the origin, but that there exists a control law u(t) that will do the job as rapidly as desired.
12
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
The elementary example just discussed introduces two basic ideas. The first is that if we want to be able to stabilize :E for any initial perturbation, then it is imperative that the system be completely controllable (actually, a somewhat weaker concept-stabilizability-is sufficient and will be presented in the chapter on stability theory). The second key issue introduced by our example is the idea oi feedback. The stabilizing control law u was expressed explicitly as a function of the state variables and only implicitly as a function of t. Of all the conceptual differences between classical and modern system theory this is perhaps the most basic: control is afunction ofstate. Thus, rather than adopting the time orientation of classical dynamical system theory, we adopt the event orientation of modern control theory and say that control is a function of where we are and what the time is, and not just a function of the time. Schematically, these fundamentally different points of view are depicted in Fig. 1.3, which indicates the origin of the terminology" closed loop" for the feedback control of :E.
~y(tl (OPEN LOOP)
~
L
f
y lt )
tc LOSED LOOP I
FIG. 1.3 Open- versus closed-loop control.
Assuming that :E is controllable (and consequently stabilizable), an important question is to what degree is it possible to alter the characteristic values of F by means of linear feedback? Since the rapidity with which the perturbed state will return to the origin is governed by the characteristic root of the largest real part of the controlled system, it would be desirable to be able to have all roots well into the left half of the complex plane to ensure a rapid return to the desired level. The surprising answer to the foregoing question is that under the sole assumption of controllability (for F, G constant), a linear feedback law u(t) ::: - Kx(t) may be found such that the controlled system will have its characteristic values at any predefined locations in the complex plane. The practical impact of this amazing result is that there is no cause to worry about stability in the original design of E since it can always be assured later by addition of suitable linear feedback. As a result, the system designer has considerable flexibility in his choice of F and G, unimpeded by stability considerations. In keeping with our practice of introducing quasi-real world examples to illustrate basic concepts, we now consider the regulation of an ecological system as a typical area in which stability is an essential requirement.
1.6
13
OPTIMALITY
Example: Predator-Prey System At time t, let N I (t) be the number of prey (pests) present, N 2(t) the number of predators, and u(t) the rate of application of an ideal pesticide that kills only pests in a density-dependent manner, leaving no residue. The dynamics of this system are governed by the equations dNI/dt = (exl - PI N 2 )N I - bINlu(t), dN 2/dt = (P2 NI - ex2)N2,
NI(O) = N IO, N 2(O) = N 2 0 .
If the state ofthe system is at N I = ex 2/P2, N 2 = ext/PI' then the pest population remains at a constant level with no further application of pesticide. Thus we assume that this state is the desired target for our control program. Several questions immediately present themselves:
(i) Since u(t) must satisfy 0 :::;; u(t) :::;; U, where U is a constant representing the maximum rate at which pesticide can be applied, does there exist any allowable control law that drives k to the desired state (is the equilibrium state reachable)? (ii) Since k is nonlinear, if a feasible control exists how long does it take for k to reach the equilibrium level? (iii) If k is not reachable in finite time, is it at least asymptotically stable, i.e.,does there exist a control law u such that N 1 (t) --+ ex2/P2, N 2(t) --+ ext/PI as t --+ CXJ under application of u(t)? These are all vital questions which, unfortunately, require methods outside the main thrust ofthis book for their answers (since k is nonlinear). We shall, however, give some guidelines and hints as we proceed in order that the interested reader can see what modifications of the basic linear theory are necessary to cope with these issues. 1.6 OPTIMALITY
All of the system concepts considered thus far-dimension, controllability, observability, stability etc.-have been basically of a qualitative nature, relatively independent of specific numerical values of parameters, i.e., the particular property remains unchanged upon small perturbations in the parameters describing k. Historically, however, control theory has been developed as a branch of the calculus of variations in which the primary concern was the minimization of an integral subject to various side conditions. Paradoxically, it has been only in the past two decades that the more basic concepts described above have been explicitly recognized and studied independently of the concept of optimization. Using ex post
14
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
facto reasoning, it certainly seems apparent that all of the preceding qualitative properties of 1: should be well understood before superimposing any type of numerical performance measure (e.g., it is of little consolation to obtain an ..optimal" control law if it yields an unstable closed-loop system). But once these basic issues have been decided and flexibility still remains in the admissible input set n, it then makes perfectly good sense to speak of the input from n which is optimal with respect to a given performance index. Since this book is devoted almost exclusively to the linear theory, we shall consider only those criteria that lead to linear feedback control laws; namely, those that are quadratic in the states and controls. The most general criterion of this type is J = (x(T), Mx(T»
+
iT
[(x, Qx)
+ 2(x, Su) + (u, Ru)J dt,
where M, Q(t), S(t), and R(t) are n x n, n x n, n x m, m x m matrices, respectively. Often, for convenience and to ensure nondegeneracies, we shall impose additional restrictions on M, Q, S, and R, e.g., M, Q ~ 0, R > 0, S = 0 are the most common. The range of integration may be either finite or infinite. The physical interpretation of J is clear. We desire 1: to remain at the level x = O. The matrix Q measures the penalty attached to deviation from the desired state. Since application of control is not free, the matrix R measures the cost of influencing the system with the control u. (The remaining matrix S has a less transparent character and can usually be eliminated through a change of variable in either n or r.) Since we often want to ensure that 1: terminates near the origin, the matrix M is also included to measure this cost if T < 00. As illustration of a situation where a cost function such as J is appropriate, we consider a somewhat extensive water pollution model. Example: Storm-Sewer Control We consider the watershed basin depicted in Fig. 1.4, where solid lines denote sewer elements, dashed lines gutters or pipes, hatched boxes sewersystem output points, triangles conduits, and arrows the direction of drainage in each subcatchment. The system has four conduits with the three dark triangles being control points at which the flow may be stored or diverted to the receiving waters. The flow through the system is controlled by the orifice openings at the control points and the overflow is controlled by variable height weirs. The objective function is chosen to minimize the weighted overflows from the system. If the different weighting factors are chosen on the basis of pollutant concentrations combined with a knowledge
]5
1.6 OPTIMALITY
:TI
©
-l® I I
iii
(2~
® (3)
t:S
®
/
FIG. 1.4 The storm-sewer system.
of the receiving waters, then the overall effect on the receiving waters may be minimized. For example, overflow from a control point in a subcatchment located in an industrial region with high traffic density may contain a higher percentage of pollutants than that for a residential section. Let the functions F 1(r), F 2(t ), F 3(t) represent the inflows of the system at points 1, 2, 3, respectively. The variables VI' V2 , V3 represent the overflows to be diverted to the receiving waters, while q l ' q2' q3 are the outflow volumes through the controllable orifices. The flows over the variable height weirs are governed by • _
V; -
3/2
CWi
hi (t),
where cwo is an empirical coefficient and h, the depth of flow over weir i. The flows through the orifices are given by
qj =
c o i r / ( d i ) 1/ 2 ,
where co; is an empirical coefficient, r, the radius of the opening, and d, the depth of water above the center line ofthe orifice. The depth d, is governed by
d. = F i I
-
co i r / ( di ) I/ 2
AMJ
-
cw i hl/2
'
where AMJ is an area-depth relationship determined by the dimensions of the conduit immediately upstream 'of the control point. The objective is to determine ri(t), h.{t) to minimize J
=
f~
[tlZiCwih;/2(S)]dS,
where z, is the weighting factor, T the final time, and to the initial time. In addition to the state equations, we must have
16
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
which says that the depth of flow over the weir cannot be less than zero nor greater than the depths of water in the reservoir. Also, rj(rj - R j)
~
0,
stating that the orifice radius cannot be negative nor exceed a maximum value and dMj - DJ ~ 0, which provides that the depth of water above the control point cannot be negative or greater than some preset maximum value. By considering a modified cost function and by making some standard transformations, the foregoing nonlinear optimal control problem may be restated as a linear regulator problem. In the differential equation for the depth, the area-depth relationship may be assumed linear. Consider the state variables to be the depth of water at various control points and the flow through the orifice at each control point. Then the state equations take the form did j = l/k j[ F j(t ) - co,v/dl/ 2 - cw ,ht /2]
qj
=
co J / dl /2 •
Let Yi = d//2; then Yi = f\(t) - 1/kjco,r/yl/4 - 1/k jcw ,ht /2, • _ qj - Coirj 2Yj1/4 ,
h F- i (t ) -- F i /k j"C-b i -- Co, 21 / 4 . were Now we define the control components in terms of fluid flows as follows. Let where N is the number of control points in the system. Then the state equations are Yi =
t, -
qj
k jU 1+N'
=
Uj - Ui+N,
which is a system of the form
x=
with F ==
Fx
+
Gu
°and the t, as disturbances. Once optimal control trajectories
u/ and Ur+N are determined, the optimal values of the physically adjustable quantities hi' Wi are obtained as h j*
= [k jUj*/c w.]2/3,
rj* = [kjUr+N/Co,yrl/4r/2.
1.7
17
STOCHASTIC DISTURBANCES
Note that ri* is expressed as a function of the state Yi*' Since the state variable is easily measured, no difficulty is presented. A cost function that represents the original cost function, but that also penalizes rapid changes in control, may be given by
=
J
fT [(x, Qx) + (u, Ru) + (u, Su)] dt, 10
where x = [Ylq)', and Q, S represent the penalty with respect to depths or rate of change of control. The R matrix represents the relative weighting factors applied to overflows at the different control points. This problem with derivative constraints may be rewritten as a standard regulator problem by means of the transformations
x,
R 1 = S,
= [xlu)"
F,=[~
~J
Qi =
[~ ~J
The final system dynamics are
xt
= Ftx, + GtUt,
and the cost function is J =
fT[(X" Q,xd + (u"
R,u t )] dt.
to
1.7 STOCHASTIC DISTURBANCES
Many important physical systems are so complex that many assumptions and approximations are needed to bring them within the purview of the framework presented above. Naturally, there arises the question of how much of the flavor of the original problem is retained in the mathematical approximation. This is clearly a far-ranging question possessing no uniform answer. One of the ways to account for the various inaccuracies in translating the real world into a tractable mathematical formulation is to assume that the idealized system is subjected to random disturbances of one type or another. Since our major objective in this book is to provide an introductory account of modern linear system theory, we shall refrain from a detailed discussion of the myriad ways such disturbances may be introduced, and content ourselves with presenting only those approaches that dovetail with deterministic system theory through the so-called separation principle. Consider the internal description of I::
x = Fx + Gu,
18
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
and now assume the input u to be a stochastic vector whose purpose is to account for uncertainty in the system dynamics. In other words, the mathematical idealization of the original system is
x=
Fx,
and we introduce a stochastic input vector u to account for the various uncertainties present in passing from the real world to the mathematical model. Furthermore, assume that the observations of the system are also corrupted by additive noise, i.e., y
= Hx + v,
where v is a stochastic vector of observation errors. A natural question to pose is: Given the corrupted observations y(t) over an interval 0 ~ t ~ t I and the statistics of the noise processes u and v, what is the "best" estimate of the current state x(t I)? If we agree to measure" best" in the sense of minimizing the mean square deviation of the true state from the estimate over the interval 0 ~ t ~ t 10 this problem leads directly to the so-called Kalman filter. We shall give all details later, but at this point we note that the duality principle mentioned earlier manifests itself in this problem by the complete equivalence between the filtering problem just posed and a totally deterministic control process of the type discussed in the last section. This remarkable fact, that an inherently stochastic problem can be reformulated as a totally deterministic control process, is a consequence of one of the cornerstones of modern stochastic control theory-the separation principle. In order to describe heuristically the content of the separation principle, let us recall the actual mechanism involved in a feedback control process. Since the control law is a function of the state x and the time t, we must first measure the state (or perhaps the output y for output-feedback control), then compute the optimal control according to the prescription appropriate for the problem at hand. In the stochastic case, the situation is complicated by the fact that the measured quantity y is corrupted by noise and so, in general, we cannot expect that the deterministic procedure based on exact knowledge of the state or output will be of any use. The question that arises is whether or not it would be mathematically permissible, for the purposes of computing the control, to pretend that the optimal statistical estimate of the state is the true state. The answer to this question is known as the separation principle, which asserts conditions under which such an approach will be valid. It is called the separation principle since its application enables one to "separate" the procedures of estimation and control in the sense that the statistical estimate of x is made, then this estimate is input to a deterministic controller in order to determine the optimal control. The diagrammatic form is shown in Fig. 1.5. In general nonlinear problems, of course, such a
19
NOTES AND REFERENCES u
FIG. 1.5 Separation principle.
separation of estimation and control is invalid and leads to erroneous results. However, under the assumptions of the principle, essentially consisting of linear dynamics, quadratic costs, and Gaussian noise processes, separation is legitimate resulting in a dramatic reduction in the complexity of the problem. A simple illustration of the foregoing type of process is provided by the reservoir system discussed in Section 1.2. The rainfall inputs rb t z are stochastic quantities whose statistics are given for the particular area under study. The stream outputs Yl, Y2 may be subject to noises in the sense that inaccuracies in the inherent measuring equipment may produce uncertainty in the true flow. Under these conditions, it is quite reasonable to estimate and control the entire reservoir system on the basis of the ideas following from the separation principle. NOTES AND REFERENCES
Section 1.2
The water reservoir example has been adapted from
Szollosi-Nagy, A., State Space Approach to Hydrology, Symp. Math. Modelling Hydrology, University College, Galway, Ireland (April 1974).
Additional examples of the use of linear systems tools for water resource problems are found in Wood, E., editor, "Real-Time Forecasting/Control of Water Resource Systems." Pergamon, Oxford, 1980.
Section 1.4
Further details on the uses of system theory in economics are
given by Aoki, M.. Local controllability of a decentralized economic system, Rev. Econom. Studies, 51-63 (January 1974).
Other papers and books dealing with economics, control, and system techniques include Baumol, W., "Economic Theory and Operations Analysis." Prentice-Hall, Englewood Cliffs, New Jersey, 1965. Gale, D., "The Theory of Linear Economic Models." McGraw-Hili, New York, 1960.
20
BASIC CONCEPTS, PROBLEMS, AND EXAMPLES
Dobell, A. R., Some characteristic features of optimal control problems in economic theory, IEEE Trans. Automatic Control AC-14, 39-47 (1969).
A fascinating use of linear systems ideas for investigating the rise and fall of civilizations a la Toynbee is given in Lepschy, A. and Milo, S., Historical event dynamics and 'A study of history' by Arnold Toynbee, Scientia 111,39-50 (1976).
The concept of a feedback control law has been explicitly used for at least a century. The great physicist J. C. Maxwell suggested a mechanism for governing the Watt steam engine based precisely on the feedback principle. Many further aspects in the development of the feedback concept to its current refined form are treated in the book Section 1.5
Bellman, R., and Kalaba, R. (eds.), "Mathematical Trends in Control Theory." Dover, New York, 1964.
The predator-prey example was first presented by Goh, B., Leitrnann, G., and Vincent, T., Optimal control of a predator-prey system, Proc, 14th Internat. Conqr. Entymology, Canberra, Australia (August 1972).
Further examples of systems concepts in ecology are given in Slobodkin, L., "Growth and Regulation of Animal Populations." Dover, New York, 1980.
Section 1.6
For more details on the storm-sewer example, see the paper by
Winn, C. B., and Moore, J., The application of optimal linear regulator theory to a problem in water pollution, IEEE Trans. Systems, Man, & Cybernet, SMC-3, 450-455 (1973).
A good account of the separation principle, its many uses, and its restrictions is given by
Section 1.7
Astrom, K. J., "Introduction to Stochastic Control Theory." Academic Press, New York, 1970.
See also the references on filtering theory cited in Chapter 8.
CHAPTER
2
Mathematical Description of Linear Dynamical Systems
2.1 INTRODUCTION
Before plunging into the deep waters of mathematical system theory, it is important to have a clear and unambiguous notion of the mathematical objects being discussed. Consequently, this short chapter is devoted primarily to a collection of definitions precisely delineating what we mean by the words "dynamical," "linear," and "constant" as applied to system theory. In addition, we also take this opportunity to introduce a few other descriptions of a linear system which are also useful in a variety of contexts. Of particular note in this regard will be the so-called frequency domain description which will be utilized later, especially in the chapters on realization and stability theory.
2.2 DYNAMICAL SYSTEMS
Prior to giving the technical definition of what is meant by a dynamical system, let us briefly review the basic features explored in the first chapter. We regard 1: as a device that transforms inputs into outputs. This processing may take place continuously over time or it may occur only at discrete moments. In either case there is a time set T associated with the system. At each moment of time t E T, 1: accepts an input u(t) E V, a fixed set of input values (from say Rm) and produces an output y(t) E Y, a fixed set of output 21
22
2
MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS
values. In general, we demand that the complete input function u(t), t 1 < t ~ t 2 , belong to the set n. Since the output of L depends, in general, on both the present input and the past history of L, we say that the present output depends on the state of L, where the state is intuitively interpreted as that part of the present and past history of L that is required in order to determine current and future outputs, i.e.,the current state is an internal attribute of L that determines the present output and affects future outputs. Note that we do not necessarily demand that the state be the minimal amount of information required to predict the effect of the past on the future. The adjective "dynamical" reflects the fact that both the current state x(t 1) and the input segment u(t), t 1 < t ~ t 2, are necessary and sufficient to determine the state x(t 2)~ t 1 < t 2' Of course, this implies that T has a preferred direction (T is ordered) and we assume that the ordering is set up so that the past precedes the future. The basic definition of a dynamical system is now given. Definition A A dynamical system L is a mathematical concept defined by the following axioms: (i) There is a time set T, a state set X, a set of input values U, a set of input functions Q = {u: T -+ U}, a set of output values Y, and a set of output functions r = {y: T -+ Y}. (ii) T is an ordered subset of the real numbers. (iii) Q satisfies the conditions (a) Q is nonempty. (b) An input segment u(t), t 1 .< t ~ t 2 in Q is restricted to (t 1, t 2) n T. If u, U' E nand t 1 < t 2 < t 3' there exists u" E Q such that u"(t) = u(t), t 1 < t ~ t 2 and u"(t) = u'(t), t 2 < t ~ t 3 • (iv) There is a state transition function qJ: TxT x X x Q -+ X
whose value is the state x(t) = qJ(t; r, x, u) in X, resulting at time t E T from the initial state x(r) E X at initial time rET, under action of the input u E Q. tp has the following properties: (a) qJ is well defined for all t 2: r but not necessarily for all t < r. (b) qJ(t; t, x, u) = x for all t E T, X E X, U E Q. (c) For all t 1 < t 2 < t 3 we have qJ(t 3 ; t 1 , x, u) = qJ(t 3 ; t 2 , qJ(t 2 ; t 1 , x, u), u) for all x E X, U E Q. (d) If u, u' En and u(t) = u'(t), r < t ~ T, then qJ(l; r, x, u) = qJ(l; r, x, u'). (v) There is a readout map 1'/: T x X y(t) = I'/(t, x(t)) at time t,
-+
Y that assigns the output
2.2
23
DYNAMICAL SYSTEMS
Some of the items introduced in this formal definition often appear under different names in the literature. For example, the pair (t, x), t E T, X E X is called an event in :E. The set T x X is often called the phase space. (The state space X is also sometimes called a "phase space," particularly in physics and engineering, but this is archaic and ambiguous. It refers to the special case: state = position + momentum.) The state transition function cp has many names: trajectory, motion, orbit, flow, solution, etc. The "sernigroup" property (iv), part (c) is of particular interest. Basically, it states that it is totally irrelevant whether one regards the system as having arrived at the state at time t 3 by a direct transition from the state at time t 1, or by first going to an intermediate state at t 2, and then considering the system as having been "restarted" from the state at time t 2 and moving to the state at time t 3' In either case, the semigroup property asserts that the system will arrive at the same state at time t 3 • Figure 2.1 depicts the situation graphically. It is clear that the semigroup property is a generalized statement of uniqueness for the solution of the state transition equation.
FIG. 2.1 The semigroup property for state transitions.
As an example of the semigroup property, consider the transition function cp given by
corresponding to the scalar differential system
x=
lXX,
x(r) =
x
with the input u == O. The semigroup property is
which, when translated for the above example, is
an expression of the familiar law for addition of exponents. Clearly, this interpretation extends to the matrix exponential as well.
2
24
MATHEM ATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS EXERCISES
1. Use the semigroup property to establish the addition law for the sine and cosine functions, i.e., sin(s cos(s
+ I) = + I) =
sin s cos I cos s cos I
+ cos s sin I, -
sin s sin I.
2. What does the semigroup property say about the solution of the Riccati equation r(O)
= O?
The foregoing definition of a dynamical system is much too vague and general to supply the basis for deep mathematical theorems and practical applications. It is useful primarily as a vehicle to focus our thoughts on the basic objects comprising a system and to enable us to see which concepts need refinement in order to arrive at definitions of greater mathematical and applied interest. To quote a dictum of the mathematician Adolph Hurwicz: "To generalize is easy, to particularize is what is difficult." We must lower our sights and restrict attention to various special classes of systems to make headway in our quest for detailed understanding. The first restriction we impose on a system ~ in a given state is that the output response to a given input segment be independent of the particular time interval in which the input takes place. In other words, that the structure of ~ not change with time. More precisely, we state the next definition. A dynamical system ~ is constant (time invariant) if and only
Definition B
if (i) T is an additive group (as a subset of R), i.e., if t l' (a) t 1 + 12ET (b)
(II
+ (2 ) + 13 =
II
12 , t 3 E
T, then:
+ (12 + (3),
(c) 0 E T and 0 + I = I for all lET (d) For each t E T, there exists an element - t E T such that t
+
(-1)=0.
(ii)
n is closed under the shift
operator z": u
u'(t) = u(t
-+
u' defined as
+ r)
for all t, rET (iii) cp(t; r, x, u) = cp(t + s; t + s, x, z'u) for all sET (iv) The map 17(1, .): X -+ Y is independent of I. Many of the basic results presented in succeeding chapters will be independent of the constancy of ~ and we shall point this out as we go along.
2.2
25
DYNAMICAL SYSTEMS
However, constant systems are the easiest to study and, generally speaking, the main results take their simplest and most definitive form for this class of systems. Given a differential equation description of a system as
x=
f(x, u, r),
the definition of constancy amounts to little more than saying that independent of explicit dependence on t (f is autonomous), i.e.,
f
=
f
is
jt». u).
Requirement (iv) imposes the same autonomy on a given system output function. In the matrix description given for linear systems in Chapter 1, the constancy assumption means that the matrices F, G, H are constant. Our next refinement is to impose some restriction on the "complexity" of the state space of 1:. Definition C A dynamical system 1: is finite-dimensional if and only if X is a finite-dimensional vector space. Thus dim 1: ~ dim X.
The major assumption that distinguishes the systems studied in this book is linearity. Intuitively, this means that if the system begins in a neutral state (x = 0, say), and if Yt(t), Y2(t) are the responses from two inputs Ut(t), U2(t), then the response from the input ut(t) + U2(t) is Yt(t) + Y2(t), i.e., the "superposition" principle is valid. The next definition states this more precisely. Definition D
A dynamical system 1: is linear if and only if
(i) X, U, n Y, r are vector spaces over R. (ii) The map cp(t; r, " .): X x n --. X is R-linear for each t, r (iii) The map 1](t,'): X --. Y is R-linear for all t E T.
E
T.
Definitions A-D have been entirely set-theoretic and algebraic in the concepts introduced and, as a result, are of no particular use if we wish to employ the various tools from analysis in our own investigation of systems. To remedy this situation, we introduce the notion of continuity by means of Definition E. . Definition E
A dynamical system 1: is smooth if and only if
(i) T = R, the real numbers. (ii) X and n are topological vector spaces, i.e., the concept of an open set may be introduced in X and n. (iii) The transition map cp is such that (r, x, u) --. cp(.; r, x, u) defines a continuously differentiable map f: T x X x n --. {continuously differentiable functions from T --. X}.
26
2
MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS
The topology referred to in (ii) may be taken to be any of the topologies induced by imposing a metric (or norm) on the spaces X and Q. The most familiar metric is that of square integrability over a finite subset [t l' t 2] of T. In this case II
u(t) 11 2 = fl'(U(t), u(t» dt, 11
where (" .) denotes the vector inner product. A similar definition holds for II x(t) II. In this case, the open sets in X and Q are defined to be open balls in the neighborhood of the origin, e.g., V, = {u(t): II u(t) II < s, s > O} c Q,
is a family of such open sets. With the open sets of X and Q defined, continuity of tp is then determined in terms of the usual notions of inverse images of open sets being open (or, alternatively, images of closed sets being closed). In order to have Definitions A-E make contact with the usual mathematical model of a system as a set of differential equations, we have Theorem 2.1. Theorem 2.1 Let I: be a smooth dynamical system. Then the transition function cp of I: is a solution of the differential equation dxldt = f(t, x, u). PROOF
(2.1 )
The theorem follows immediately from property (iii)of Definition E.
Thus we see that the smoothness assumption on I: suffices to show that I: may be represented by a set of differential equations, a fact not at all obvious from the earlier definitions. Hence, rather than defining I: by the set of equations (2.1), we have deduced from abstract principles that, under Definitions A-E, I: must be describable by a differential system. It is also clear that if we specialize to a constant, linear, finite-dimensional I:, then the appropriate equations are (2.2) dxjdt = Fx + Gu, yet) = Hxit), as given ab initio in Chapter 1. EXERCISE
1. (a) What are the conditions needed to show that a given transition function may be represented by the difference equation
x(t
+
1) = Fx(t)
yet) = Hx(t).
+ Gu(t),
2.3
27
EXTERNAL DESCRIPTION
(b) May any of the assumptions concerning qJ be dropped in the discretetime situation? 2.3 EXTERNAL DESCRIPTION
Classically, a system was described by its input/output behavior obtained during the course of a laboratory experiment involving some physical mechanism whose workings were to be explained. As pointed out in Chapter 1, the "state variable" or internal description of ~ given in Definitions A-E is, chronologically speaking, a recent arrival upon the system theory scene. The traditional (and still useful) "external" description is our next objective. Before giving the formal definition, however, let us recall the intuitive notions discussed earlier. We are given an initial event (r, x) and an input segment u(t) that acts Q.p. ~ to produce the output segment y(t), r < t s: t i- Thus we have a map f: u(t)
--+ y(t),
where, in general, f depends on both r, the initial time, and x, the initial state. Assuming the structure of ~ given in Definition A, we clearly have [f(u)] (t) = '7(t, qJ(t; r, x, u)),
On the other hand, any family of functions ft. x' which are compositions of the functions tp and '7 and which satisfy the parts of Definition A relative to qJ and '7, define a dynamical system in the input/output sense. Intuitively speaking, we conceive of the transition function qJ as being an object that defines the physical mechanism under study. In classical physics this function is the object of our study and all that is known to us are the domain and range of the map f, i.e., the sets n, U, r, and Y. Our experiments consist of selecting inputs from Q, applying them to ~, and measuring the outputs r. In this way we hope to deduce a unique qJ describing the experimental results and, failing this ultimate goal, to isolate the class of possible mechanisms tp. In the chapter on realization theory, we shall elaborate further on this theme. Our formal definition of the external description of ~ is given as follows. Definition F
~
is said to be given in external (input/output) form if
(i) There exist sets T, U, Q, Y, and r satisfying all properties of Definition A. (ii) There exists an index set A indexing a family of functions
:F
=
(fa: T
x Q --+ Y,
(X
E
A}.
28
2
MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS
Eachj, is called an input/output function and has the properties: (a) There is a map J1: A --+ T such that fa(t, u) is defined for all t 2 J1(a).
(b) Let r, t E T and r < t. If u, U' E Q and 1I(t') = u'(t'), r < t' :s; t, then fit, u) = fit, u') for all a such that r = J1(a). Definition F formally defines the experimental setup described above. Property (ii), parts (a) and (b), merely tells us when the experiment a began and that two experiments should be labeled differently if they yield different results under seemingly identical circumstances. The definitions of constancy and linearity for system l: described externally follow immediately from Definitions Band D. 2.4 FREQUENCY-DOMAIN ANALYSIS
As the historical origins of modern system theory are in the domain of electrical engineering, specifically circuit theory, it is perhaps surprising that so little flavor of this subject has been observed in our treatment thus far. It is doubly surprising since the vast majority of the extant literature (particularly elementary textbooks) is replete with examples, notations, and philosophy from this field. Obviously, our feeling is that the somewhat specialized ideas of Nyquist plots, Bode diagrams, frequency-response curves, et al., useful as they are for certain applications, are not of sufficient generality or applicability to warrant a central role in the modern mathematical theory of systems. Basically, the subject has outgrown its parentage and all current trends indicate that the tail will continue to wag the dog for the foreseeable future. Nonetheless, in the study of constant, linear, dynamical systems, some of the classical concepts are still of great utility and we shall not hesitate to call on them as needed. Thus in this section we present a brief review of the traditional frequency-domain ideas that will be utilized later. We consider the time-invariant linear system
x=
Fx
+ Gu,
(2.3)
y = Hx,
and examine the effect of choosing inputs u of the form
where u is a constant m-dimensional vector. By standard results from differential equations, x(t) = eF1xO
+ f~eF(I-S)GU(S)
ds,
2.4
29
FREQUENCY-DOMAIN ANALYSIS
where Xo = x(o). Thus y(t)
= HeFtxo + f:HeFlt-S)GU(S) ds,
and we see that the output of any linear system may be decomposed into two distinct parts: (a) the zero-state response LHeF1t-S)GU(S) ds,
(2.4)
corresponding to Xo = 0, and (b) the zero-input response corresponding to u(r) == 0. For the time being, we consider only the zero-state response and ask about the output behavior of I: under sinusoidal inputs as t --+ 00. Instead of considering the process as beginning at a fixed time t = 0, we look at the output at a variable initial time r as t --+ 00, i.e., y(r, r) = H f>(t-SlGU(S) ds.
Our interest is in
Iimt~oc
y(r, r), Substituting u(t) = e1iOJtu, we have
y(r, t) = H felIiOJ-FlseFtG dsu = P(r, t)u.
It is not difficult to show that lim Pit; t) = eiOJrMUw)u (if it exists) as t --+ 00, where M(iw) is an m x m matrix depending only on w. The important point here, of course, is that the steady-state output is also a sinusoid of the same frequency as the input. If u had been composed of a sum of sinusoids N
u(t) =
I
eiOJjtuj'
1=1
then, by linearity, the output of I: would also have been such a sum of the same frequencies. We immediately see the relevance of this result for electrical circuit theory-if we desire the response of I: to any input u, we first decompose the signal into its constituent frequencies, determine the response of I: to each frequency, then add them together properly weighted. Since such
30
2
MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS
frequency analyses are part-and-parcel of any electrical engineer's arsenal of analytic weapons, such an approach to linear systems seems quite natural. There are pitfalls, however, since even for simple systems the matrix M may not exist. EXERCISE
1. Consider the system
x = x + u, y = x.
Prove that if limt _
oc
Pit, r) exists, then it has the form stated in the text. 2.5 TRANSFER FUNCTIONS
For modern system theory, a more important object for study than the frequency-response function is the so-called transfer function, relating the inputs and outputs of ~. Assume that the components of the input u are functions satisfying
i = 1, ... ,m, for some (1 < 00. Let us denote by O/I,:?£, OJ/, the Laplace transforms of the vectors u, x, y, respectively. Then taking the Laplace transform of (2.3), we have s:?£(s) - :?£(O) = F:?£(s) + GO/I(s), OJ/(s) = H:?£(s).
Thus OJ/(s) = H(sI - F)-I[I(O)
or, if x(O)
+
GO/I(s)],
= 0, OJ/(s) = H(sI - F)-IGO/I(S).
(2.5)
The object Z(A) = H(U - F)-IG relating the Laplace transforms of the input and output of ~ is called the transfer function of E, Thus for every A that is not a characteristic value of F, Z exists and is a p x m matrix function orA. Let us now relate the transfer function Z to the steady-state frequencyresponse matrix M(iw). We have the following theorem.
2.6
31
IMPULSE-RESPONSE FUNCTION
Let the characteristic values of F be denoted by AI' A2 , ... , An
Theorem 2.2 and let
0(0
= max Re(A j )
< O.
1 =5i~n
Suppose u(t)
= e1sotu, where Re So >
0(0'
Also, assume that x(O)
= O. Then
lim Yet, t) = Z(SO)elsofU • t~o::
That is, M is the special case of Z where PROOF
So
is purely imaginary.
Left as an exercise.
As a result of Theorem 2.2 we see that the transfer function Z is a more salient object for system theory so, in subsequent chapters, we shall not return to the frequency-response function, although it is well to keep in mind the origins of Z in terms of the frequency response. EXERCISES
1. (a) Compute the transfer function associated with the system
(b) What is the significance of the fact that some of the elements of F, G, and H do not appear in Z(A)? 2. Show that if we allow direct output observation, i.e., y = Hx + Ju, then the transfer function Z(A) may have entries with the degree of the numerator equaling that of the denominator. (Hint: Consider Z(A) = J + H(M - F)-IG and expand in powers of IjA.) 3. Compute the transfer function matrix for the system
0 0 1 0 0 0 F= 0 0 0 0
0 0 0 0 0 0 0 0 0 1 0 0 0 1 0
0
1
1 0 G= 0 1 1 0
H =
[~
0 0
2.6 IMPULSE-RESPONSE FUNCTION
Given the linear system
x=
F(t)x
y = H(t)x,
+ G(t)u,
x(t o) = xo,
1 0 0 0 0 0
~l
32
2
MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS
it is easily verified that x(t)
=
+
It
to
where the matrix
=
s < t,
F(t)
Thus yet) = H(t)
+
It
H(t)
to
If the initial state Xo = 0, we then have the zero-state response yet) =
It
H(t)
~
=
It
Wet, slues) ds,
~
where Wet, s) is the so-called impulse response, since it measures the observed output due to a "delta-function" input. When the input u == 0, t ~ to, then yet) = H(t)
is the zero-input response. Thus, as in the case of constant systems, we see that the output of any linear system may be decomposed into two distinct parts corresponding to zero initial state and zero input. This is an important consequence of linearity which shall be exploited many times in succeeding chapters. MISCELLANEOUS EXERCISES
1. A representation of ~ in the form of (2.3)determines Z(A) uniquely. Is the converse true? Consider the two systems
Xl =
x2 =
u(t),
X2 Xl
+ u(t),
yet) = x 2 (t )
and dx/dt = -x y(t) = x(t)
+ u(t),
in this regard. Prove that the components of Z{A) are rational functions of A, and that the denominator of each component equals the characteristic polynomial of F. What interpretation can be attached to the numerator of each entry? 2.
33
NOTES AND REFERENCES
3. Let Z(A) be given as the transfer function of an abstract constant linear system as per Definitions A, B, and D.lfthe components of Z are not rational functions of A, show that this implies the state space of I: is not finite-dimensional. NOTES AND REFERENCES
Sections 2.2-2.3 The definitions and concepts introduced here follow the scheme given in Chapter One of Kalman, R., Falb, P., and Arbib, M., "Topics in Mathematical System Theory," McGrawHill, New York. 1969.
The semigroup property of the state transition map plays a central role in the abstract theory of dynamical processes. An encyclopediac treatment is the classical work by Hille, E., and Phillips, R., "Functional Analysis and Sernigroups," Vol. 31. American Math. Society, Providence, Rhode Island, 1957.
Another excellent reference for the use of semigroups in dynamical processes is by Sibirsky, K. S., "Introduction to Topological Dynamics." Noordhoff, Leyden, 1975.
A discussion of various aspects of nonlinear semigroup theory may be found in Crandall, M., Semigroups of nonlinear transformations in Banach spaces in "Contributions to Nonlinear Functional Analysis" (E. Zarantonello, ed.). Academic Press, New York, 1971.
Section 2.4 A good introduction to the basic concepts behind the frequencydomain formulation of linear system problem is by Polak, E., and Wong, E., "Notes for a First Course on Linear Systems." Van NostrandReinhold, Princeton, New Jersey, 1970.
See also Rosenbrock, H. H., "State-Space and Multivariable Theory." Wiley, London, 1970. Wonharn, W. M., ..Linear MultivariabJe Control: A Geometric Approach." Springer, Berlin, 1974.
A provocative article comparing frequency versus time-domain approaches is Horowitz, I. C, and Shaked, V., Superiority of transfer function over state variable methods in linear-time-invariant feedback system designs, IEEE Trans. Automatic Control AC-20, 84-97 (1975).
34
2
MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS
Sections 2.5-2.6 A modern treatment of the Laplace transform approach for engineering analysis is Zeman ian, A. H., "Distribution Theory and Transform Analysis." McGraw-Hili, New York, 1965.
Further results are given by Zadeh, L., and Desoer, c., "Linear Systems Theory." McGraw-Hili, New York, 1963. Rubio, J., "The Theory of Linear Systems." Academic Press, New York, 1971. Barnett, S., "Introduction to Mathematical Control Theory." Oxford U. Press, Oxford, 1975.
A number of important processes evolve in discrete, rather than continuous, time. In these cases, the system output function is 1
y(t) =
L HP-iGu(i -
I).
i= 1
Thus the impulse response function in discrete time is the quantity
nric. Forming the discrete-time Laplace transforms u(z)
=
L U(t)Z-I, 00
1=0
00
y(z) = Ly(t)Z-I, 1=0
we immediately obtain the discrete-time transfer function analogous to Z(A). The important point to note in this regard is that the discrete-time transfer matrix is also composed of rational functions and, as a result, is amenable to treatment by the same algebraic techniques as are used for the continuous-time case; only the interpretations of the results differ due to the different time sets.
CHAPTER
3
tontrollability and Reaehability
3.1 INTRODUCTION
The most basic objective of any control process is to transfer the motion of a dynamical system from one point (or set) in state space to another in some desirable fashion subject to various restrictions. After a mathematical model of a given process has been constructed, the first step of the analysis must be to establish precisely what can and cannot be done within the limitations imposed on the admissible inputs. Thus it should come as no surprise that we begin our mathematical study of systems by establishing conditions for given systems to be controllable in one sense or another. For example, consider the simple electrical network shown in Fig. 3.1. We let Xl be the magnetic flux in the inductor while X2 is the electric charge on the capacitor. The input u is a voltage source. For simplicity, assume Lie = R 2 = 1. Then the differential equations for the system are 1
dx;
dt = - L X l + u(t), dX2
dt =
-
1
C X 2 + u(t).
If we let
35
3
36
CONTROLLABILITY AND REACHABILITY
we see that
Thus the input u affects only the state Xl' while X2 cannot be influenced by the applied voltage source. As a result, only a subset of the state space may be reached by varying the voltage u. R
R
ult)
FIG. 3.1 A simple electrical network.
Even in this trivial example, it is not difficult to see that if L # C, then any magnetic flux Xl and electric charge X2 may be obtained through application of a suitable input voltage u. In other words, b) selecting the capacitance and inductance "at random," the circuit has probability one of being controllable to any state (Xl> X2), i.e., in two-dimensional (L, C)-space, only the onedimensional line L = C corresponds to circuits that are not controllable to an arbitrary prescribed state. Later we show that this situation is representative of all constant linear systems in the sense that if the components of F and G are selected at random, "almost every" resulting system will be controllable. Conversely, the class of uncontrollable systems forms a set of zero measure within the set of all constant linear systems. This point is made more precise in Section 3.9. 3.2
BASIC DEFINITIONS
Let I: be an arbitrary dynamical system as defined in Chapter 2 and assume there exists a state x* and an input u* such that qJ(t; r, x*, u*)
= x*
for
all t, rET,
t ~ r,
The state x* will be called the neutral state. (If X and n have additive structure, we may take x* = 0, u* = 0, where 0 has its usual meaning. Throughout this book, we shall consider only systems I: with this extra property.) Definition 3.1 An event (r, x) is controllable if and only if there exists a t, r < t < 00 and a u E n (both t and u may depend on (r, x» such that qJ(t; r, x, u) =
o.
I: is completely controllable if it is controllable for every event (T, x).
3.2
37
BASIC DEFINITIONS
In other words, an event is controllable if and only if it can be transferred to the neutral state in finite time by application of some admissible input function u. Pictorially, we think of the path from (r, x) to [r, 0) as the graph of a function defined over [r, t] (see Fig. 3.2).The definition of the "dual" concept reachability is suggested by reflecting the graph about the line r. In words, this is stated in the following definition.
x
---'------t----+--_t-------l.---T 5
T
T
FIG. 3.2 Controllability and reachability.
Definition 3.2
<
00
and au
An event (r, x) is reachable if and only if there is an s, s < r (both sand u may depend on (r, x)) such that
En
x
=
q>(r; s, 0, u).
1: is completely reachable if it is reachable for every event (r, x). NOTE Controllability and reachability are entirely different concepts. They coincide only in special cases, ope of which is when 1: is a constant, continuous-time, linear system. The reader can easily verify this point by consideration of the scalar system
x=
g(t)u(t),
where g(t) is any continuous function such that g(t) == 0, t < 0. Then the system is seen to be controllable for any event (0, x), but not reachable.
Example:
Macroeconomics'
A highly simplified version of a common economic situation will illustrate some features of the reachability issue. A country has the short-run economic objective of full employment without inflation (internal balance) and balance of international payments (external balance), which must be accomplished through policy instruments such as changes in the interest rate and in the budget deficit. Thus the country has a central bank which controls interest rates, and a legislative body (congress) which controls changes in the government deficit. While it is politically difficult to combine these institutions into
38
3
CONTROLLABILITY AND REACHABILITY
a single controlling agency, it is possible to establish general directives for them to follow. The question is whether this system can be controlled. To study this problem, we define the following variables: domestic production (i.e., income of consumers), aggregate expenditures, C(t) aggregate consumption, S(t) aggregate savings, I(t) domestic investment, M(t) imports of foreign goods and services, K(t) net capital outflow, T(t) net taxes of transfers, G(t) government expenditures for goods and services, B(t) net surplus in international balance of payments. Y(t)
X(t)
All of these variables are annual rates in period t, and are deflated to a uniform price level. Define the additional variables
E YF
exports of goods and services (assumed constant), the full employment, no inflation level of domestic production (assumed constant), r(t) domestic interest rate, rF foreign interest rate (assumed constant). Four accounting identities link these variables:
x=
Y=C+S+T, B = E - M - K,
B
C + I + K + G, = .y - X.
Further, the following linear relationships have been found empirically to be roughly valid:
S=
1X 1 Y -
1X 0,
M =
/31
Y-
/30'
+ 'Yo,
I = 'Y1 r
Defining the state and control vectors as B(t))
x(t) = ( Y(t) ,
r(t
u,(t) = ( Di:
+ +
1) - r(t)) 1) - D(t) ,
where D = G - T (net government deficit), the above relations give the dynamical model x(t
where
+
1) = x(t)
+ Au(t),
3.3
39
TIME-DEPENDENT LINEAR SYSTEMS
From a given initial state x(O), the planner desires to steer the economy to the target x* =
[~J
where the international payments are in balance and a level YF of outputs is achieved yielding internal balance. For this example, it does not take deep thought to see that with unrestricted changes in tax rate and deficits, any desired terminal state can be reached if the matrix A is nonsingular. Since this will be the case for almost every set of values of the parameters <5 10 P10 Y1' 0(10 the complete reachability property is "stable" with respect to changes in these parameters. 3.3 TIME-DEPENDENT LINEAR SYSTEMS
We shall study the linear system dxfdt
=
F(t)x
+ G(t)u,
(3.1 )
or its discrete-time counterpart x(k
+
1) = F(k)x
+ G(k)u,
(3.2)
where the matrix functions F(t), G(t), (F(k), G(k» are defined for all t(k) and are bounded in every bounded interval of T. For the time being, we restrict our attention to system (3.1). We have already seen that (3.1) has the solution cp(t; to, xo, u) =
(t, to)xo
+
Jt
where the matrix (t, s) satisfies the equation o(t, s)/ot
and
=
F(t)(t, s),
possesses the properties that:
(i) (t, s) = (t, u) = (t, T)( T, 0") for all t, T,
U.
Hence,
(3.3)
40
3
CONTROLLABILITY AND REACHABILITY
It follows immediately from (3.3) and the properties of <1> that an event (T, x) is controllable if and only if x = - 1'<1>(T, s)G(s)u(s) ds.
t >
T
holds for some control u(t) E n. Let ~(T) c X represent the set of states that are controllable at time T. Then the following facts are consequences of the definition of controllability and the linearity of L.
Theorem 3.1
~(T)
is a linear subspace of X.
Now let {e;} be a basis for ~(T). Then there is a number t \ (T, ej) such that the event (T, ej) can be transferred to (t\, 0). Define tdT) = maxj{t\(T, ej)}. Then we have the result:
Theorem 3.2 control.
Every event (T, x) in C6(T) can be transferred to ((1,0) by some
An important consequence of (3.3) is Theorem 3.3.
Theorem 3.3 From every event (T, xo) in (T, ~(T)), it is possible to reach at, or before, t 1 (T) every motion which passes through (T, x.) in (T, C6(T)). In other words, Theorem 3.3 tells us that we may transfer any controllable state at time T to another controllable state at time t 2 ;:::>: t I (T), and a trajectory beginning at the event (T, xo) outside (T, C6(TH can never enter the set (t, ~(t)), t ;:::>: T. Theorems 3.1-3.3 will be used in proving the main properties of the set of controllable (or reachable) events. We now define the symmetric, nonnegative definite linear transformation W(T, r) by
W(T, t) = 1'<1>(T, s)G(s)G'(S)<1>'(T, s) ds.
(3.4)
The rank of W will playa central role in our discussions. Clearly, the rank of W is nondecreasing in t and is bounded above by n (= dim X). Thus there is some time t I(T) beyond which W has constant rank.
EXERCISE
l. Show that the number t 1(T) defined by W is the same as the number t 1(T) occurring in Theorems 3.2 and 3.3.
3.3
41
TIME-DEPENDENT LINEAR SYSTEMS
Let 9t'[W] and feW] denote the range and null space of Jv, respectively. Then the basic controllability theorem is as follows.
Theorem 3.4 The event (r, x) is controllable for some t > r. PROOF
(Sufficiency) Let W(r, t)y
=
if and
only
if x E 9t'[W(r, t)]
x for some t > r. The control
u = - G'(t)'(r, t)y will take (r, x) to (t, 0) as can be seen by substituting u into (3.3). (Necessity) Since W is symmetric, we have the orthogonal state space decomposition x = 9t'[W(r, t)] EB %[W(r, t)]. Because of linearity and the previous theorems, we need only show that no state in .¥[W(r, t)] is controllable. . Assume that x =F 0 E %[W(r, t)] and that the event (r, x) is controllable. Then we have 0= (x, W(r, t)x)
=
f"G'(s)'(r, s)x11 2 ds.
(Here we use the notation (', .) to denote the usual inner product in R".) Since the integrand is nonnegative, we must have G'(s)'(s, r)x
=0
for
almost all s E [r, t].
Since (r, x) is assumed controllable, we must have
x = - f(r, s)G(s)u(s) ds for some u. Thus
2 f(X, -(r, s)G(s)u(s)) ds.
0< IIxl1 =
However, the right side vanishes identically which yields the desired contradiction.
Corollary ~(r) = 9t'[W(r, t 1 ) ] , where t 1 is any value of t for which W(r, r) has maximal rank. The following is the corresponding reachability theorem.
Theorem 3.5 An event (r, x) is reachable some s < r, where W(s, r)
=
if and only if x E 9l[W(s, r)]
f(r, u)G(u)G'(u)'(r, u) da.
for
3
42
CONTROLLABILITY AND REACHABILITY
REMARKS (1) If G(·) is zero on (-00, r), we cannot have reachability, and if G(.) is zero on (r, (0), we cannot have controllability. (2) For F, G constant, W(r, t) = W(2r - t, r) and the integrals defining Jv, W depend only on the difference of the limits. Thus for constant l: an event (r, x) is reachable for all r if and only if it is reachable for one r ; an event is reachable if and only if it is controllable.
Since the rank condition on W (or W) may not always be easy to verify, we now present a sufficient condition for controllability of a time-varying linear system. This condition is much easier to verify than that of Theorem 3.4 but fails to be necessary. Theorem 3.6 Let F(t) E cr>. G(t) E en-Ion n x m matrix functions Qi(t) by
[0,00].
Define the sequence of
Qo(t) = G(t), Qi+ I(t) = F(t)Qi(t) - Qi(t),
i
=
0, 1, ... , n - I.
Then the linear system (3.1) is completely controllable at time ~(t)
t
if the matrix
= [Qo(t)/QI(t)1 .. ·IQn-I{t)]
has rank nfor some time t > r. PROOF
We have x(t) = {'P(r, s)G(s)u(s) ds,
(3.5)
where 'P(r, r) = - I.
o'Pjot = - 'P(t, s)F(t),
Integrating the right side of (3.5) by parts, we obtain n-I
x(t) = i~O (_I)i+ IQi(r)
t i
it t
+ (- l ]" t'P(s)Qn(S)
(s - r)i - i - '- u(s) ds
it s
(r - s)n-I
(n _ 1)! u(r)dr ds.
(3.6)
Hence, representation (3.6) shows that if the condition of the theorem is satisfied for some t 1 > r, then the rows of'P(r, t I)G(r)arelinearly independent over [r, t d, i.e., l: is controllable. EXAMPLE
Consider Hill's equation with a forcing term z(t)
+ (a - b(t))z(t) = g(t)u(t),
3.4
43
DISCRETE-TIME SYSTEMS
where b(t) is an analytic periodic function and a constant. Putting Xt{t) =
z, X2(t) = z, we have Xt(t) = X2(t), x2(t) = - [a - b(t)]xt(t)
+ g(t)u(t).
It may be that the periodicity of b(t) influences in some interesting way the controllability properties of (*). However, computing ~(t) we find
g(t)J
-g(t) ,
which implies that (*) is controllable at any r < t such that g(t) ¥- O. Thus the periodic system (*) apparently has no controllability properties which are a consequence of its periodicity. EXERCISES
1. Show that the condition of Theorem 3.5 is also necessary if F(t), G(t) are real analytic functions of t. 2. Show by example that the pair (F(t), G(t)) may be completely controllable for each fixed t, but still not be completely controllable. 3.4 DISCRETE-TIME SYSTEMS
Now we briefly consider the discrete-time, nonstationary system (3.2) and the associated reachability and controllability questions. In connection with this investigation, we define a real n x n matrix-valued function cp by . {F(k)F(k - 1)· .. FU cp(k,}) = I,
+
I)FU),
undefined,
k ?: i. j = k + I, j>k+1.
The function cp is the discrete-time analog of the continuous-time function <1>. We have . x(k; ko, xo, u)
=
cp(k - 1, ko)xo
+
k-t
I
cp(k - l,j
+
I)GU)uU).
j=ko
We observe that in contrast to the continuous-time situation, it is possible for the set of all solutions to (3.2) with u == 0 to lie in a proper subspace of R" (e.g., let F(k) == 0). Thus the possibility of such pointwise degeneracy means that theories developed for continuous-time systems (where such degeneracies cannot occur) may not be in 1-1 correspondence with their
44
3
CONTROLLABILITY AND REACHABILITY
discrete-time analogs. The main reachability/controllability result for discrete-time systems is given in the following theorem.
Theorem 3.7 A necessary and sufficient condition for (3.2) to be completely reachable at time i in M steps is that rank[G(i - 1)1q>(i - 1, i - I)G(i - 2)1" ·Iq>(i - 1, i = n. PROOF
+M +
I)G(i - M)]
(Sufficiency) Let
Rk(i - 1) = [G(i - 1)1q>(i - 1, i - I)G(i - 2)1" '!q>(i - 1, i - k + I)G(i - k)], and suppose rank RM(i - 1) = n. Then the solution to (3.2)at time i, starting in the zero initial state at time i - M, is
l
u(i - 1)
x(i; i - M, 0, u)
= RM(i -
1) u(i
~
2)
j
~
RM(i - I)U M(i).
(3.7)
u(i - M)
Define the n-dimensional vector VM(i) by the relation
Then
which shows that we can solve for VM(i) and obtain the appropriate control sequence needed to reach any given XM(i). (Necessity) Suppose rank RM(i - 1) < n but that (3.2) is completely reachable in M steps at time i. Then there exists a vector '7 =F 0 in R" such that '7'R M (i - 1) = O. Hence, premultiplying both sides of (3.7) with '7' gives '7'x(i; i - M, 0, u) = 0 for any u. Since (3.2)is completely reachable at time i, we choose the control sequence {u(i - M), ... , u(i - I)} such that xU; i - M, 0, u) = '7. Then '7''7 = 0, contradicting '7 =F O. REMARKS (l) The criterion of Theorem 3.7 is also a sufficient condition for complete controllability of(3.2) at time i - M. However, it is not necessary unless F(k) is invertible on the interval i - M + 1 ~ k ~ i-I. This is the pointwise nondegeneracy condition referred to above. (2) The proof of Theorem 3.7 shows that complete reachability at time i in M steps implies the ability to reach any fixed state at time i from any given state (not just the origin) at time i - M.
3.4
45
DISCRETE-TIME SYSTEMS
(3) Complete reachability in M steps at time i implies complete reachability in N steps at time i for all N ~ M. This is false if reachability is replaced by controllability unless F(k) is invertible for all k ~ i + M. Example: National Settlement Planning An area in which discrete-time reachability questions play an important role is in governmental planning for national settlement policies. Several different approaches have been proposed for dealing with this sort of problem, some of them falling into the basic framework considered in this chapter. We describe one of these "system-theoretic" approaches. The government objective is to promote a desired migratory process by differential stimulation of the employment market. The state equations for the model are x(t v(t
+ 1) = + 1) =
Kx(t) Mv(t)
+ (I - M)v(t), + u(t) + z(t),
where the vector x(t) E W represents the population distribution at time t, v(t) E R" the distribution of job vacancies at time t, u(t) E W the distribution of government stimulated job vacancies, and z(t) E R" the distribution of spontaneously occurring vacancies. The matrix K is a diagonal matrix whose elements reflect the natural population growth rates within a region, while M is a migration matrix with elements mij being the probability that a job vacancy in regionj will be filled by someone living in region i; i.] = 1, ... , n. The problem, of course, is to choose u(t) so that x(t) (and possibly v(t)) follow some desired course. The budgetary and fixed immigration constraints on the choice of u(t) are given by (i) u(t) ~ 0, (ii) (u(t), r(t» ~ b, (iii) Ilu(t)11 ~ (I, t = 1,2, ... , T. Here 11·11 denotes some appropriate vector norm (e.g., [1), with r(t) being a function giving the total resource (jobs) available to be offered regionally by the government at period t, b being the total number available. By introducing the new vectors s(t)
=
L~t)J
w(t)
X(t )]
= [ v(t)
,
y(t) =
[z~)J
it is possible to rewrite the above model in the form w(t
+
1)
=
Fw(t)
+ Gs(t) + y(t),
46
3
CONTROLLABILITY AND REACHABILITY
where
Constraints (i)-(iii) restrict the region of admissible inputs s(t). Actually, on the basis of more detailed analysis, for purposes of determining reachable sets it suffices to replace inequalities (ii) and (iii) by the corresponding equality. (Physically, this fact is fairly obvious but requires a surprising amount of analysis to prove.) However, the restriction of the admissible inputs does impose added mathematical complications in determination of the reachable set. We shall take up some of these "complications" of the basic problem in a later section. An interesting question appears if one considers the discrete system (3.2) as arising from (3.1) by discretizing the time axis. For this case, in many physical processes it is natural to assume that the input u is a piecewiseconstant function, i.e., u(k)=<J.n ,
n
s. k
c n
»
o,
n,k=1,2, ....
Such an assumption is called sampling in the engineering literature since the continuous system's input and output are "sampled" at discrete-time instants of length (J. Suppose that the continuous system is completely controllable. Since the introduction of sampling is a restriction on the controls, it cannot possibly improve controllability. The question is "can sampling destroy controllability?" The answer is given by the next theorem. Theorem 3.8 Suppose that system (3.1) is constant and completely controllable. Then a sufficient condition for the sampled system (3.2) arising from (3.1) to be completely controllable also is that Im[Aj(F) - AiF)]
=1=
2nqj(J,
q
= ±1, ±2, ....
where
Re[Aj(F) - AiF)] = 0, lfm = I, this condition is also necessary. PROOF
See the references cited at the end of the chapter.
The intuitive meaning of the above result is that the periodicity inherent in sampling must not interact with the natural frequencies of the system to be controlled if controllability is to be retained. Note also that the condition can always be satisfied by choosing the sampling frequency Ij(J sufficiently large, i.e.,making a sufficientlyclose approximation to the original continuous system.
3.5
47
CONSTANT SYSTEMS
3.5 CONSTANT SYSTEMS
Often in problems of practical concern we must assume that the system under investigation is constant, since otherwise it may be impossible to determine a model for it: in an arbitrary nonconstant system, past measurements may reveal nothing about future behavior. As one might suspect, restricting our attention to constant systems results in major simplifications in the form ofthe foregoing results and also suggests several new subproblems which we shall investigate. In addition to the practical and computational aspects associated with a thorough study of the constant case, a detailed understanding of constant systems serves to point out what features to expect from a general theory of controllability and to suggest appropriate methodological tools for such a development. Thus there are several compelling reasons for a careful analysis of constant linear systems. The simplest and most useful criterion for complete controllability for a continuous-time, constant linear system ~ is Theorem 3.9. Theorem 3.9 The constant linear system (3.1) is completely controllable and only if the n x nm matrix
if
has rank n. Corollary 1 If ~ is completely controllable, then for any r, x, and s > 0, there is an input u that transfers the event (r, x) to (r + s.D), PROOF Let rc have rank n and assume ~ is not completely controllable. Thus there exists a state XI "# 0 such that XI E %(W(O, tl))' We fix t l and show, as in Theorem 3.4, that
since
Differentiating this relation n - 1 times with respect to t, and setting t = t I yields i = 0, 1, ... , n - 1,
(3.8)
which implies that X I is orthogonal to the columns of rc. But rc has n linearly independent columns. Thus Xl = 0, a contradiction for the chosen t r- But, since t 1 was arbitrary, ~ is controllable over any interval [0, t 1]'
48
3
CONTROLLABILITY AND REACHABILITY
To prove the converse, let 1: be controllable and assume rank CC < n. Then there exists a nonzero vector Xl that satisfies (3.8). By the HamiltonCayley theorem, (3.8)is actually true for all i. Therefore,
which implies that x ' E A'TW(O, t 1)], contradicting the controllability of 1:, since W must be nonsingular (Corollary to Theorem 3.4). REMARK Corollary 1 may seem somewhat surprising to practically minded engineers since it is a well-known fact that real control systems cannot be controlled in arbitrarily short intervals of time. However, the difficulty is one of practice and not theory. Theorem 3.9 assumes that the matrices F and G are known to an arbitrary degree of accuracy, in fact, that they are known exactly and the effect of the control input u is immediately felt by the system L, i.e.,there are no uncertainties or time lags in the control loop. When we apply the theory, we must be concerned about the accuracy of our data and the relevance of the simultaneity assumption before appealing to Theorem 3.9. EXAMPLES
(1) Consider the system
F =
[~ ~J
G=
[::1
We investigate the conditions on gl, g2' that imply controllability. The matrix CC is
which is of rank 2 if and only if g2(2g l - g2) -# O. (2) A less trivial example is provided by considering the longitudinal equations of motion for a VTOL-type vehicle hovering at constant altitude. These are
d dt
Ug
-O(g
U
Xu
x
q
e
o u, o
0 Xu 1
Mu
0
o
o
0 0 0
o o u,
o
1
o -g
0
Ug
0
U
o x o q o e
wl
+
0
0 15+
0
M6
0
0
0
3.5
49
CONSTANT SYSTEMS
where ug the longitudinal component of gust velocity, u velocity perturbations along the x-axis, x position along the x-axis, q pitch rate, () pitch attitude, [) control stick input, M s control stick amplification factor, M u speed stability parameter, M 9 pitch rate damping, M q control sensitivity, Xu longitudinal drag parameter, g gravitational constant, IXg wind gust break frequency, WI longitudinal component of wind speed. Clearly, the controllability of this process is unaffected by the wind speed. Thus the relevant controllability matrix is '{;=
[~.
0 0 0 M.M.
M.
0
0
-gM.
0 M"M. M.M.
0
-gM.(X u
-
M.l
-gM.(X u
-gM. 2
-gMhMu
+ M.M/
M.M/
-
M.)(X u
-gM.(X u -gM.(X u
-
M.lM u
-
M.( -gM u
-
+ M/l
M.l
M.M.(gM" - M/l
+ M/l
Hence, we see that the aircraft system is not completely controllable for any values of the parameters IXg , Xu, M u , M q , a situation more or less obvious from a glance at the dynamics, as ug is unaffected by the other states and by the control. Some additional useful corollaries of Theorem 3.9 follow. A constant system L ~ (F, G, -) is completely reachable if and only if the smallest F-invariant subspace of X containing the columns of G is X itself.
Corollary 2
Corollary 3 A constant system L = (F, G, -) is completely reachable if and only if there is no nontrivial characteristic vector of F which is orthogonal to every column of G. Corollary 4
The set of reachable states of L is a subspace of X. More precisely, the set of reachable states is the subspace of X generated by the columns of
-«
3
50 Corollary 5
CONTROLLABILITY AND REACHABILITY
Consider the extended controllability matrix
-F
0 I
0
-F
0 0
0 0
I
't=
2
of dimension n x n(n if ~ has rank n 2 •
+m
0 0 0
0 0
I
0
G
-F
G
0
0 0 0
0
0 0
0
G
G
G
0
0 0
0 0
0 0
0 0
- 1). Then .E is completely reachable
if and only
PROOF In the matrix 16, add F times the first block row to the second, then add F times the new second block row to the third, and so on. The result is a block matrix similar (by column operations) to
[
1,._ ".
:],
On.n(n- I)
which has rank equal to rank <:t
+ n2
-
n.
The conclusion follows immediately. An important system-theoretic interpretation of the foregoing corollaries is given by Theorem 3.10. Theorem 3.10
The state space of .E may be written as the direct sum
X=X 1EBX 2 , which decomposes the system dynamics as
dx-fdt = FllXI + F I 2 X 2 dX2/dt = F 22 X2 ' The subsystem (F 11> G 1,
-)
+ G1u(t),
is completely reachable.
PROOF Define X 1 to be set of reachable states of .E. This is an F -invariant subspace of X. Hence, since X is finite-dimensional, XI is a direct summand of X. Since every state in X 1 is reachable and every column of G is in XI' the F -invariance of X 1 implies F 21 = 0, which implies the above equations of motion. REMARKS (l) X 2 is not intrinsically defined since it depends on an arbitrary choice in completing the direct sum. Thus the set of all unreachable
3.5
51
CONSTANT SYSTEMS
(or uncontrollable) states does not have the structure of a vector space. In general, the direct sum cannot be completed in such a way that F 12 = O. (2) In problems in which control enters, only the space X 1 has meaning. It is imperative to isolate the space X 1 at the beginning of an analysis since control problems are nontrivial only with respect to the subspace X 1 and it may be of much lower dimension than the original state space X. The theorems for discrete-time systems are analogous to the continuoustime versions if F is nonsingular. However, some complications do arise in the singular case. Thus, for completeness, we state the basic results without proof.
Theorem 3.11 The discrete-time constant system k = (F, G, -) is completely reachable if and only if the matrix
has rank n. Theorem 3.12 A state x of the system k described in Theorem 3.11 is controllable if and only if x is contained in the space generated by the columns of the matrix
where F(-k)G
= {x: Fkx =
g for so~e
column g of G}.
Theorem 3.13 In a real, discrete-time, finite-dimensional linear constant system k = (F, G, -), a reachable state is always controllable and the converse is also true if det F =I' O. EXERCISE
1. In Corollary 5 to Theorem 3.9, show that the extended controllability matrix '6' may be transformed to
<6=
G 0 0
I -F
0
0
0
0 G 0
0 I -F
0 0 0 0 G I
by means of elementary row and column operations.
0 0 0
0 0 0
-F
G
52
3
CONTROLLABILITY AND REACHABILITY
3.6 POSITIVE CONTROLLABILITY
Until now we have examined the controllability/reachability issue under the hypotheses that: (a) the input signals u(t) are piecewise-continuous functions but otherwise unrestricted and (b) the entire state space X is of primary interest to us. Clearly, in a large number of interesting practical cases one or the other of these conditions (or both) may be unreasonable or undesirable. In the next few sections, we shall examine several important cases of controllability in which restrictions of one type or another are imposed upon either the input space O or the state space X. Perhaps the most frequently occurring restriction in practice is to demand that the input functions assume only nonnegative values. Thus the input space n is restricted to the subset
n+
= {u(t): u is piecewise continuous for all t and u(t) 2 O}.
The problem of positive controllability then becomes the question of whether or not 1: is controllable/reachable with inputs restricted to n+. It is clear that if 1: is positively controllable, then necessarily a controllable state X o has the form
k = 1, ... , m, i = 0, I, ... , where gk represents the kth column of G. Obviously, each positively controllable system is controllable but not conversely. Let us consider some examples. EXAMPLES
(I)
The system
x=u
(x, u scalars)
is controllable, but it is not positively controllable. The states which may be transferred to the origin by inputs from n+ lie on the negative real axis. (2) The controllable system
x
=
u,
is also not positively controllable. The initial states Xo = {x(O), x(On which may be transferred to the origin by controls in n+ are given by the conditions /10' /11 > 0,
where
9 =
G)'
Fg =
(~}
Thus only interior points of the fourth quadrant are positively controllable to the origin.
3.6
53
POSITIVE CONTROLLABILITY
(3) The system 'X I
x2
cos () + X2 sin () = - X I sin () + x 2 cos ()
= XI
a<
+ u,
() #- kn,
k = 1, 2, ... ,
is positively controllable. Before proceeding to an analysis of the positive controllability problem, we digress for a brief moment to consider controllability as a problem in the theory ofmoments. We recall that the state X(t) of'E has the representation
The condition x(t d =
a is equivalent to the equation -eFflxO = {leF(fl-SlGU(s) ds.
If u is a one-dimensional (scalar) control, the numbers [eF11xo]j, where [zl denotes the ith component of the vector z, are the "moments" of the function u with respect to the system function [eF (I, - S)G1 . A necessary and sufficient condition for solvability of the" problem of moments" is that the functions [eF(ll -SlG]i'
i = 1,2, ... , n,
be linearly independent, i.e.,
for all s and for any z, [z] = 1. This condition is completely equivalent to Theorem 3.9 (Proof?). To apply the foregoing result to the problem of positive controllability, we consider a single-input (m = I) system L
x=
Fx
+ guo
Define [a(t)]_ = {a(at ),
a(t) < a(t) ~
a, a.
Then the arguments given above show that a necessary and sufficient condition for positive controllability is the requirement that a~s~t
for all z, small.
IIzll = 1. Clearly,
(3.9)
this condition may be violated if t is sufficiently
54
3
CONTROLLABILITY AND REACHABILITY
To obtain more explicit criteria, we examine the quantity
VJ(t) = z/eF(I-S)g more closely. Since F satisfies its own characteristic equation, we have F" + cxlF n- 1 + cx n- 2 + .. , + cxnI = O. 2F
Multiplying by (_l)nz'eF(I-S) on the left and by 9 on the right, we obtain
t/J(n)(t) - cxlt/J(n-I)(t) + ... + (.,-l)n-Icxn_It!i(t) + (_l)ncxnt/J(t)
= o.
The initial conditions are
= z'q, t!i(s) = z'Fg,
t/J(s)
t/J(n-I)(S) = (_l)n-Iz'Fn-lg. Returning to condition (3.9), we finally obtain the main result.
Theorem 3.14 Necessary and sufficient conditionsfor a single-input system L to be positively controllable are the matrix I{j = [gIFgl" ·IFn-lg] has rank n; (2) F has only complex or pure imaginary characteristic roots; (3) the number t is sufficiently large. (1)
As a result, we see that a single-input system cannot be positively controllable for all t. Also, no single-input system of odd order may be positively completely controllable. EXERCISES
1. (Multi-Input Positive Controllability) Show that a necessary and sufficient condition for a multi-input system (m > 1) to be positively completely controllable is that the set of vectors (-I)P+ J FP gv, fJ = 0, 1, ... , v = 1, ... , m, form a nonnegative basis for R", where gj denotes the ith column of G, i = I, ... , m. (Definition: The vectors bl, b 2 , •.• .b, form a nonnegative basis in the subspace that they generate if any vector 5 from the subspace may be written as
,
fj =
L fJi b;,
j~
1
fJi
~ 0.)
2. Show that every controllable system may be made positively controllable by adding an additional control um+ I with vector gm+ I = - Li~ I cxig;, CXi > O.
3.7
55
RELATIVE CONTROLLABILITY
3.7
RELATIVE CONTROLLABILITY
Occasionally the requirement of being able to transfer L from any initial state xo to the origin is too strong. What we actually desire is to transfer x o to some appropriate subspace X' of X. For example, X' may represent some set of equally desirable states, a set of terminal states, and so forth. More precisely, we have the following definition.
Definition L is called controllable relative to the subspace X' = {x: Kx = O} if for every state Xo, there exists a number i < rx) and a piecewise continuous control u(t), 0 ~ t ~ i, such that Kx(t) = O. Since every controllable state Xo has the representation m
Xo =
n-l
L1 LOIlki F igb
k=
i
(3.10)
e
the states controllable to X' at t are given by KrjJ(t)xo
=
m
n-l
L L Ilki K rjJ(t )F ig
k>
k= 1 i=O
where rjJ(t)xo is the state at time t if L begins in state Xo. Since rjJ(t) is nonsingular for all t, we immediately have Theorem 3.15. Theorem 3.15 and only if
The system L is controllable relative to the subspace X' rank[KGIKFGI" ·IKFn-1GJ
if
= rank K,
i.e.
rank KC(J = rank K. EXERCISE
1. Show that in the case of an (n - I)-dimensional subspace X': k'x = 0, the condition of Theorem 3.15 takes the simpler form i = 0, I, ... , n - I,
Example:
j = L 2.... , m.
Urban Traffic Flow
The problem of regulating the flow of urban traffic in a street or freeway network provides a good illustration of a situation in which the notion of relative controllability may playa role. Consider the rectangular network depicted in Fig. 3.3. We assume that the network is oversaturated, i.e., at one or more intersections traffic demand
3
56
FIG. 3.3
CONTROLLABILITY AND REACHABILITY
Urban traffic network.
exceeds capacity. Let Xj(t) be the number of cars waiting at intersection i, and let u;(t) denote the number of cars leaving intersection i during the green light. If we assume that the travel time between two intersections is small compared to the waiting time, then the dynamics of the process are reasonably well described by the equations x(t
+ 1) =
x(t)
+
Gu(t)
+
q(t),
where the vector q(t) has components qj(t) representing the external traffic arriving at intersection i during period t. It is clear from Fig. 3.3 that the flows u 3' U6' U9' and u 1 0 are flows out of the network. The control matrix G takes the form
-1 51
G=
0 0 0 0 0 0 '1
0
0 -I 52
0 0 0 1"2
0 0 0
0 0 -1 0 0 0 0 0 0 0
0 0 0 -1 54
0 0 0 0 1"4
0 0 0 0 -1 55
0 1"5
0 0
0 0 0 0 0 -1 0 0 0 0
0 0 0 0 1"7
0 -1 0 0 57
0 1"8
0 0 0 0 0 -1 58
0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -1 0 - -1
The elements I"j and s, denote the percentage of cars turning, either right or left, and going straight ahead, respectively.
3.8
57
CONDITIONAL CONTROLLABILITY
On psychological grounds, it is reasonable to impose the control constraints
u,
~
Uj(t)
u;
~
i
=
1, ... ,10,
where M i and Vi represent the minimal and maximal number of cars that may move during a given psychologically acceptable green time. The basic problem is now quite simple: given an initial state x(O), assumed to be an oversaturated condition, is there a control policy u(t) which transfers x(t) to an undersaturated region within a prescribed time T? Thus we see that the subspace f is chosen to be the smallest subspace of X containing the set of undersaturated states of X.
3.8 CONDITIONAL CONTROLLABILlTV
We have already seen that the set of controllable states forms a subspace that is generated by the columns of ctJ. In many cases, however, we are given a particular subspace .if of X and would like to determine whether or not every state in .Ii is controllable and, moreover, we desire a simple test to answer this question. This situation is particularly important in those processes for which we have either a priori knowledge about or influence over the initial state Xo and can assert that it belongs to some distinguished subspace of X. From the representation of controllable states (3.10), the following criterion for conditional controllability is obtained. Theorem 3.16 The system L is conditionally controllable from a subspace j{{xo = My, y E X} if and only if
rank[M I G I FG I·· ·\Fn - 'G]
= rank[G I FG I·· '!Fn-1G],
i.e.,
rank[MI'1&"] EXAMPLE
Let L = (F, G, -), where 1 -I
-1 -5
o
°2 -1
= rank '1&".
3
58
CONTROLLABILITY AND REACHABILITY
Assume that the initial states lie in the plane X4 = 0, X3 space AI is given by those points x = My, y E R 4 , where
M=[~
o
= O. Then the sub-
! H]. 0 0 0
Substituting these entries into Theorem 3.16, we see that the rank condition is satisfied. Thus ~ is conditionally controllable from AI. Notice, however, that had we chosen the plane x I = X 2 = 0 as the region of interest, then the rank condition would fail. 3.9 STRUCTURAL CONTROLLABILITY
In an attempt to obtain a more realistic methodology for studying system structure, we now turn to controllability questions that are dependent only on the internal connections of the process under study, and not on the specific numerical values of the system parameters. For definiteness, let us assume that the entries in the system matrices F and G are either fixed zeros or arbitrary nonzero parameters. Such a supposition is more consistent with reality since, in practice. system parameter values are never known exactly; however, the positions of fixed zeros in the structure are often known due to choice of a particular coordinate system (e.g., time derivative of position is velocity, etc.) or because physical connections between certain parts of a system are absent. From a computational point of view, the assumption of "fuzzy" parameters and "true" zeros is also desirable since digital computers can only recognize integers with exact precision. Hence it is of interest to study basic system properties that rely on numerical decisions of only the "zero/nonzero" type. We now outline an approach to the study of controllability utilizing such ideas. We first define precisely the notions of "structure" and "structural questions." Definitions A structural matrix F is a matrix having fixed zeros in some locations and arbitrary, independent entries in the remaining locations. A structured system ~ = (F, G, -) is an ordered pair of structured matrices. The two systems ~ = (F, G, -), f = (F, G, -) are structurally equivalent if there is a one-to-one correspondence between the locations of their fixed zero and nonzero entries. A system ~ = (F, G, -) is called structurally controllable if there exists a system structurally equivalent to ~ which is controllable in the usual sense.
3.9
59
STRUCTURAL CONTROLLABILITY
The foregoing definitions show that if there are N arbitrary nonzero entries in 1:, then associated with 1: is the parameter space R N and every system structurally equivalent to 1: is represented by a datum point r ERN. The properties of interest for this discussion will turn out to be true for all data points except those lying on an algebraic hypersurface in RN . To be more precise, consider a finite set of polynomials t{Jj E R[A], A = (AI"'" AN)' Then the point set V of common zeros of t{Jj(A) in RN forms what is called an algebraic variety. If V ¥ R N , then V is called proper and if V ¥ 0, V is nontrivial. A data point r E R N is typical relative to V if r E Vo, the complement of V in R N and any property n determined by the data points of R N is called generic if the set of data points for which n fails to be true forms a proper algebraic variety in R N • Thus generic system properties can be expected to hold for almost every data point in R N , i.e., they hold for all typical data points. EXAMPLE
Consider the n x nm controllability matrix C(j
=
[GIFGI· .. IP-IG].
Let t{J(A.) be the polynomial in N = n2m indeterminates AI, ... , AN defined as the sum of the squares of all possible nth order minors of C(j. Clearly, any data point r E RN such that t{J(r) = 0 implies that rank C(j < n, i.e., that 1: is not completely controllable. Hence, to show that controllability is a generic property, we need only show that V
is a proper variety in R arbitrarily.
N
•
=
{r: t{J(r)
= O}
But, this is trivial since the entries of~
can be chosen
The above example shows that it is the inclusion of" structure" into the controllability problem that makes it possible for complete controllability to fail, independently of parameter values. We now introduce the basic technical condition for studying structural controllability. Definition An n x s matrix A (s ~ n) is said to be of form (r) for some I ~ t ~ n if, for some k in the range s - t < k ~ s, A contains a zero submatrix of order (n + s - t - k + I) x k.
For example, the matrix
o
o
0 0 0 0 0 0 0 0
x x x x x x x x x x x x x x x
60
3
CONTROLLABILITY AND REACHABILITY
is of form (4) with k = 5, while x 0 0 0 0 x 0 000 x
0
0
0
0
x x x x x x x x x x is also of form (4), but with k = 4. The importance of form (1) is then seen in the following basic result. Lemma For any t, I .::;; t < n, rank A < t for every r ERN has form (r), PROOF·
if and
only
if A
See the references at the end of the chapter.
The connection between structural controllability and form (t) is now given by Theorem 3.17. Theorem 3.17 The system 1: = (F, G, -) is structurally uncontrollable and only if the extended controllability matrix I
0
0 0
-F
G
I
0 0
0
0
-F
G
0 0
0 0
0 0
G
?6'=
is ofform (n
2
0
if
o o -F G
o
0
I
0
-F G
).
PROOF The result follows from Corollary 5 to Theorem 3.9 plus the fact that 'ti has rank n 2 (generically) if and. only if ~ is not of form (n2 ). This last fact follows from the preceding lemma.
Computationally, the importance of this result cannot be overemphasized since the determination of the form of a matrix requires only that the computer be able to distinguish between zeros and nonzeros. Thus, in contrast to the usual controllability result that depends on finding the rank of f(j ~ notoriously unstable numerical operation), determination of the form of ?if or, equivalently, finding its generic rank, may be carried out with no numerical error.
3.10
61
CONTROLLABILITY AND TRANSFER FUNCTIONS
3.10 CONTROLLABILITY AND TRANSFER FUNCTIONS
Recall that the transfer function of the linear time-invariant system ~ (=(F,
G,
-»
x=
Fx
+ Gu
is given by the polynomial matrix
assuming, that x(O) = O. An interesting and important question to ask is: How we can deduce simple conditions on Z that imply the controllability of E or, conversely, given that ~ is controllable, what structural features does this impose on Z? As Z is the Laplace transform of the matrix eFtG (since x(t) = J~ eF(t-S)Gu(s) ds), it is natural to conjecture that, in view of the controllability condition given by the rank of'b', a similar type of linear independence condition on Z will be the appropriate "frequency-domain" version of Theorem 3.9. As substantiation of this "hunch," we have the next theorem. Theorem 3.18 The system ~ is completely controllable rows of(J..I - F)-IG are linearly independent.
if and only if the
(Necessity) Let ~ be controllable, fix t, and let VI, v2 , •.. , vn be the linearly independent rows of the matrix eFtG. Let VI, V 2 , ••• , vn be their transforms. Assume these transformed rows are dependent, i.e., there exist constants CI, C2"'" c., not all zero, such that PROOF
for all J. not equal to a characteristic value of F. Thus the vectors are dependent for all t since the Laplace transform is invertible (its null space is the zero vector). This implies the vectors VI, v2 , ••• , un are also dependent contradicting the original assumption. Hence (J..I - F)-IG has linearly independent rows. . (Sufficiency) Sufficiency is demonstrated in a similar fashion by reversing the above argument. SPECIAL CASE If ~ has a single input (m = I), Theorem 3.17 simplifies to the condition that no entry of the vector (J..I - F) - 19 is reducible, i.e., if
i
= 1,2, ... ,n,
is the ith entry of (AI - F)-I g, then Pp.) and q(J.) (the minimal polynomial of F) have no common factor, i = 1, 2, ... , n.
62
3 EXAMPLE
Let 1: be given by
=
F
[~
Then (fj =
Since rank of 1: is
(fj
CONTROLLABILITY AND REACHABILITY
-2J g=Gl G-2J
-3 '
[gIFg] =
I
-1 .
= 1 < n,1: is not completely coritrollable. The transfer function
Z(2)
2(2 + 2) + 1)(2 + 2) = . ). + 2 [ (2 + 1)(2 + 2) (2
Hence, cancellations occur that again show that 1: is not controllable. 3.11 SYSTEMS WITH A DELAY
A valid criticism that is often voiced against the use of ordinary differential equations to model real control systems is that such a model assumes that the action of a given controlling input is "instantaneously" felt by the system. It is manifestly true that this situation never occurs for any real system: control takes time! Thus the validity of an ordinary differential equation model is highly dependent on the time constants of the process. Even though our objectives in this book are to deal with the pure differential (or difference) equation case, we now offer a brief excursion into the differential-delay equation world in order to exhibit some of the features of these processes. It will be seen that for several basic questions the results and methods parallel the" instantaneous" case. Consider the single-input system with time lag
x=
Fx
+ Bx(t
- r)
+ gu(t),
(3.11 )
where F, B, g are constant matrices of appropriate sizes. We further assume that B is expressible in the form B = gc'. Thus the columns of B are collinear with g. To motivate the above class of linear systems, notice that if the nth order equation
3.11
63
SYSTEMS WITH A DELAY
is written in vector form, we obtain
y(t) = Fy(t)
+ By(t
- r)
+ ilu(t),
0 0
0 0
0 0
0
0
where
F=
0 0
1 0
0 1
0
0
0
!XZ
!X3
B=
g=
P.
pz
Hence, the columns of B are collinear with g. Since a differential-delay system is defined by prescribing an initial function over the interval [- r, OJ, rather than by giving only the value x(O), we must slightly modify our definition of controllability. The new definition follows.
Definition System (3.11) is completely controllable if for every T> 0, and for every piecewise-continuous function cp defined on [ - r, OJ, there exists a piecewise-continuous control u such that x(t) vanishes on [T, T + r]. Notice that the condition B = gc' implies that for complete controllability it suffices to have x(T) = 0 since we can choose the control u defined by u(t) = -c'x(t - r)fort > T.Then the system will reduce to x = FX,x(T) = 0, for t > T.
Theorem 3.19 System (3.11) is completely controllable if and only if(F, g) is completely controllable (i.e., if the system x = Fx + gu is controllable). (Sufficiency) If _(3.11) is completely controllable by taking T < r, we see that for every initial function cp there exists a control u such that we shall have x(T) = 0 for the system PROOF
o<
x=
Fx(t)
+ gc'cp(t
- r)
+ gu(t),
x(O) = cp(O).
Thus there is a control v such that we shall have x(T) = 0 for the system
x = Fx(t) + gv(t),
x(O) = cp(O)
(choose v(t) = u(t) + c'cp(t - r)). Hence (F, g) is completely controllable. (Necessity) Let (F, g) be controllable. For 0 < T < r there exists a control v such that x(T) = 0 for the system
x=
Fx(t)
+ gv(t),
x(O) = cp(O).
64
3
CONTROLLABILITY AND REACHABILITY
Choose u(t) = v(t) - c'¢>(t - r), Then the solution to (3.11) with initial function cp will vanish for t = T. Now let T > r and let k be such that (k - I). :$ T < kt, Then there exists v such that y(T) = 0 for the system
y=
Fy(r)
+ gv(t),
y«k - I).) = x«k - I).).
We now choose u(t) = v(t) - c'xtr -.) for (k - 1). :$ t < T and u(t) arbitrary for 0 :$ r < (k - I) r. As a result of Theorem 3.19, we see that the time lag term plays no role whatsoever in establishing controllability. However, we note that this is true only because of the assumption on the structure of B. In general, the situation is far more complex. MISCELLANEOUS EXERCISES
1. For fixed F, show that the minimum number of inputs that ~ must have in order to be controllable equals the number of nontrivial invariant factors of F. Apply this result to the system ~ whose F matrix is
-~ - ~ ~ ~j
6 -14
r
-1 -5
2 1 . -1 0
How does this relate to the example given in Section 3.8? 2. The origin is said to be invariant with respect to control in the direction P (II P II = 1) if (p, x(t)) = 0 for any u En. Prove that the origin is invariant in the direction p for the single-input system
x = Fx + gu,
m = 1,
if and only if (p,g)
=
(p,Fg)
= ... =
(p,FO-1g)
= O.
3. The origin is said to be autonomously controllable in the direction P» relative to the control u,. if it is controllable relative to U v and invariant to controls Uj' j i= v, Show that the origin is autonomously controllable relative to u,. in the direction p; if and only if 0-1
L I(p,., Fig,,)I i= 0
i=O
0-1
and
L
m
L
;=0 j=l.i,/,v
I(pj' Fig j ) I =
o.
65
MISCELLANEOUS EXERCISES
4. Let matrix
IX
be the degree of the minimum polynomial of F. Show that the
has the same rank as ~. More generally, let
Prove that if rank
5.
~. + I =
rank ~.,
then for all s
rank
= rank ~ •.
~s
~
k
Prove that the following statements are equivalent:
(a) The pair (F, G) is completely controllable. (b) No column of G is orthogonal to any characteristic vector of F. (c) The smallest invariant subspace of F which contains all the columns of G has dimension n. (d) If n(z) is an arbitrary monic polynomial, there exists an n x m matrix K such that det(z! - F - GK) = n(z).
If q'(z! - F)-IG == 0 for all z, then q is the zero vector. The identity q'eFIG == 0 on an interval t I < t < t 2 is possible only for q = O. 6. (Minimal Energy Controllability) (a) Let u(t) be any control that transfers the state x(t o) = X o to x(t l ) = XI and let u*(t) be the special control (e) (f)
that accomplishes the same transfer (here # denotes the Moore-Penrose generalized inverse). Prove that
f
l l II
to
u(t) 11 2 d~ >
fl'
II
u*(t) 11 2 dt,
to
unless u(t) = u*(t) almost everywhere on [to, t l ] (11·11 denotes the inner product norm for the finite-dimensional vector space Q). (b) Show that the minimum control energy E necessary to transfer Xo to Xl (assuming such a transfer is possible) is given by
66
3
CONTROLLABILITY AND REACHABILITY
7. (a) Let I: be a linear time-varying system with 0 a compact, convex set in R". Show that if x(to) = xo, the reachable set 9l(t 1) at time t 1 ~ to is compact, convex, and continuously dependent on t 1(b) (Bang-Bang Principle) Let 0 0 be a compact subset of 0, the convex hull of which coincides with the convex hull of O. Let 9l o(t1) be the reachable set for u E 0 0 , Show that 9l o(td
= 9l(t 1).
(Thus, if 0 0 = 00, only controls in 0 0 need be examined to determine the reachable set at time t 1') 8. Let I: be a time-invariant system such that 0 is a bounded set containing u = O. Prove that the set of controllable states is open if and only if I: is completely controllable. 9. Consider the nth order time-invariant linear system
Po
i= O.
Show that this system is completely controllable. The time-invariant system I: such that
10.
x=
Fx
+ qu,
is called controllable with an arbitrarily small control iffor any e > 0, and any two states X O and x', there exists a control u(t), satisfying lu(t)1 ~ s, which transfers I: from X O to Xl in a finite interval of time. Show that I: is controllable with an arbitrarily small control if and only if (a) I: is completely controllable. (b) The characteristic roots of F are purely imaginary. 11. (a) Consider the equations of motion of a point lying in a fixed plane and moving with a given circular orbit. The system dynamics are
x=
ra",(t)~,
m
where m = mo + m 1(t ), v is the gravitational constant, r the radius vector of the point, X the generalized momentum corresponding to the polar angle t/J, and ar(t), a",(t) the projections of the velocity vector relative to the radius direction and the direction orthogonal to it, respectively. Let z l' Z2, Z3 be the deviations of the state coordinates from their values along the circular orbit, i.e., Zl = r - rO,z2 = r,z3 = X - Xo·Formulatethe linear system for the variables Zl' Z2' Z3' (Hint: A change of coordinates is useful.) (b) Is the linearized system completely controllable?
67
MISCELLANEOUS EXERCISES
12. (a) Let I: be a time-varying system such that the elements of F and G are analytic and periodic functions of t with period w. Prove that for I: to be controllable on [0, t 1] it suffices that the matrix ~(w)
=
[G(O)leFwG(O)I·· 'leFn-'WG(O)]
have rank n. (Hint: Use Floquet's theorem. See Miscellaneous Exercise 2 in Chapter 7.) (b) Show by counterexample that the analyticity assumption on F and G may not be relaxed. (Hint: Consider the scalar system
x= -
tiJ(t) [I
+ t/!(t)Rr 1Rx + qu,
where R is an n x n constant matrix with sufficiently small elements, g a constant n-vector, and t/!(t) a scalar function of period to = 1 whose graph is shown in Fig. 3.4. Assume that g, (I + R)-l g, ... , (I + R)-(n-1)g are linearly independent. Now show that this system satisfies the above rank condition on ~(w) but is still not completely controllable.)
• t
o
2
FIG. 3.4 Graph of l/J(t).
13. Consider the time-varying linear system I: such that
x = a(t) [Fx + Gu], where F, G are constant and a is continuous and bounded. Show that if rank ~ = [GIFGI·· ·IFn -
1G]
= n,
then I: is completely reachable. 14. If the time-invariant system
x = Fx + Gu is controllable, then show that there exists a matrix (feedback law) K such that
x=
(F - GK)x
+ gjv(t)
is also controllable where q, is any nonzero column of G. (Here K depends, in general, on g;.)
68
3 CONTROLLABILITY AND REACHABILITY
15. If F is an n x n constant matrix, show that the two matrices [GIFGI·· ·IFn-1G]
and
have the same range spaces. 16. Suppose that for all constant k, 0 det[g
~
k
~
1, and a fixed vector e we have
+ kelF(g + ke)I·· ·IP-l(g + ke)]
=1=
O.
Does it then follow that
will be positive definite for all t 1 > 0 and all k(t) such that 0 ~ k(t) ~ 1? 17. Consider the transfer function matrix (U - F)-lG = 2(..1). Show that a necessary and sufficient condition for 2(..1) to be invertible is that the matrix
[! have rank in
+
G FG 0 G
Fn-1G P- 2G
FnG Fn-1G
0
G
2 nG
F F2n~
1
IG
P-1G
1)m. NOTES AND REFERENCES
Section 3.1 Many additional examples and insights into the controllability question are discussed in the paper by Kalman, R., Mathematical description oflinear dynamical systems, SfAM J. Controll, 152-192 (1963).
Section 3.2 The basic definitions and their algebraic implications are extensively examined by Kalman, R., "Lectures on Controllability.and Observability." C.I.M.E., Bologna, Italy, July 1968; Edizioni Cremonese, Rome, Italy, 1969, pp. 1-149.
The economic example has been adapted from McFadden, D., On the controllability of decentralized macroeconomic systems: The assignment problem. in .. Mathematical Systems Theory and Economics" (H. Kuhn, ed.), Vol. II. Springer-Verlag, Berlin and New York, 1969.
Section 3.3
The results follow
Kalman, R., "Lectures on Controllability and Observability." C.I.M.E., Bologna, Italy, July 1968; Edizioni Cremonese, Rome, Italy, 1969, pp. 1-149.
NOTES AND REFERENCES
69
For a treatment of controllability for time-varying coefficient matrices (Theorem 3.6) see also Gabasov, R., and Kirillova, F., "The Qualitative Theory of Optimal Processes." Nauka, Moscow, 1969. (Eng\. trans\., Dekker, New York, 1976.)
Section 3.4 The controllability/reachability results for discrete-time systems are taken from Weiss, L., Controllability, realization, and stability of discrete-time systems, SIAM J. Control 10, 230-251 (1972).
A more detailed treatment of the national settlement strategy problem can be found in Mehra, R., An optimal control approach to national settlement system planning, RM-75-58, International Institute for Applied Systems Analysis, Laxenburg, Austria, 1975.
The sampling result, Theorem 3.8, was first proved by Kalman, R., Ho, Y. C., and Narendra, K., Controllability of linear dynamical systems, Contr. Diff. Eqs. I, 189-213 (1963).
Section 3.5 An interesting survey of the historical origins of the controllability concept and the genesis of the basic results is presented by Kalman, R., "Lectures on Controllability and Observability." C.I.M.E .. Bologna, Italy. July 1968; Edizioni Cremonese, Rome, Italy, 1969, pp. 1-149.
The VTOL example is taken from Kleinman, D., Baron, S., and Levison, W., A control-theoretic approach to manned vehicle systems analysis, IEEE Trans. Automatic Control AC-I6, 824--832 (1971).
Section 3.6
The problem of positive controllability is treated by
Gabasov, R., and Kirillova, F., "The Qualitative Theory of Optimal Processes." Nauka, Moscow. 1969. (Engl. trans\.: Dekker, New York, 1976.)
Somewhat more general versions of the positive controllability question are given by Brammer, R., Controllability in linear autonomous systems with positive controllers, SIAM J. Control 10, 339-353 (\ 972).
See also Maeda, H., and Shinzo, K., Reachability, cbservability and realizability of linear systems with positive constraints, Elec. Comm Japan, 63,35-42 (\982). Schmitendorf, W. and Barmish, 8., Null controllability of linear systems with constrained controls, SIAM J. Control & Optim., 18, 327-345 (1980). Perry, T. and Gunderson, R., Controllability of nearly nonnegative linear systems using positive controls, IEEE Tran. Auto. Cont., AC-22, 491 (1977). Heymann, M. and Stern, R., Controllability of linear systems with positive controls: geometric considerations, J. Math. Anal. Applic., 52, 36-41 (1975).
3
70
CONTROLLABILITY AND REACHABIUTY
It is important to emphasize the point that the positive controllability
results above only ensure that the origin can be attained in some finite time. This is in contrast to the standard situation in which complete controllability in some finite time implies controllability in an arbitrarily short time. For results on arbitrary-interval positive controllability, see Jacobson, D., "Arbitrary Interval Null-Controllability with Positive Controls," Council for Scientific and Industrial Research, Pretoria, South Africa, 1976 (preprint).
Sections 3.7-3.8
The theory of relative and conditional controllability is
covered by Gabasov, R., and Kirillova, F., "The Qualitative Theory of Optimal Processes." Nauka, Moscow. 1969.
The urban traffic example, along with other control-theoretic aspects of traffic flow is treated in the report by Stroebel, H., Transportation, automation, and the quality of urban living, RR-75-34, International Institute for Applied Systems Analysis. Laxenburg, Austria, 1975.
The concept of structural controllability was first introduced using graph-theoretic arguments by
Section 3.9
Lin, C. T., Structural controllability, IEEE Trans. Automatic Control, AC-19. 201-208 (1974).
Our approach follows Shields, R. W.• and Pearson. J. 8., Structural controllability of multi-input linear systems, IEEE Trans. Automatic Control, AC-21, 203-212 (1976).
This paper also includes an algorithm, suitable for computer implementation, which may be used to determine form (t) for an arbitrary n x s matrix A. It is of some interest to note that the basic mathematical results used to establish the lemma on form (t), as well as the proof of Theorem 3.17, are found in much earlier papers of Frobenius and Konig. See Konig, D., Graphak es Matrixok, Mat. Lapok. 389, 110-119 (1931). Konig, D., "Theorie der endlichen und unendlichen Graphen." Leipzig, 1936. Frobenius, G .• Uber Matrizen mit nicht negativen Elernenten, Berlin Akad. 23,456-477 (1912).
A fundamental dictum of practical systems analysis is that "control takes time." Thus differential-delay equations are the real substance of applied systems analysis. Such a principle is especially apparent in problems from the social sciences where long time lags are more likely to be encountered than in engineering or physics. A good account of some of these matters is given by Section 3.11
El'sgol'ts, L., and Norkin, S., "Introduction to the Theory and Application of Differential Equations with Deviating Arguments." Academic Press, New York, 1973.
NOTES AND REFERENCES
71
Another basic work is by Bellman, R., and Cooke, K., .. Differential-Difference Equations." Academic Press, New York, 1963.
The controllability result cited in the text may be found in Halanay, A., On the controllability of linear difference-differential systems, in" MathematicalSystems Theory and Economics-II" (H. Kuhn and G. Szego, eds.), Vol. 12. SpringerVerlag, Berlin and New York, 1969.
See also Mallela, P., State controllability (and identifiability) of discrete stationary linear systems with arbitrary lag, Math Modelling, 3, 59-67 (1982). Artstein, L., Linear systems with delayed controls: A reduction, IEEE Tran. Auto Cant., AC-27, 869-879 (\982). Klarnka, J., On the controllability of linear systems with delays in the control, IntI. J. Control, 25, 875-883 (1977).
CHAPTER
4
ObservabilityIConstructibility
4.1 INTRODUCTION
Most modern control processes operate on the basis of feedback control mechanisms, i.e.,the controlling inputs to the system are generated by values of the state. Consequently, they implicitly assume that all values of the internal state variables may be measured at any instant of time. In most practical situations this is not the case. As a result, in order to maintain control, regulators must include a component for state determination. The state determination mechanism has two different types of data to determine the state: (a) knowledge of the system structure, e.g., transition map, output map, dimension, etc., and (b) knowledge of actual inputs and outputs of the system. In this chapter, we shall be concerned with development of results that ensure that data of type (b) may be used to obtain good estimates of the unknown state of the system. In passing, we note that in modern engineering practice it is usually assumed that data of type (a) are given a priori. When this is not the case and the data of type (a) must somehow be inferred from input/output information, then we have an adaptive control problem. The theory of adaptive systems is much talked about, but very little has been accomplished. In the nonadaptive control problem (where data on the 72
4.2
73
BASIC DEFINITIONS
system structure are given), dynamical properties of the system are assumed to be exactly known and it remains "only" to determine the instantaneous state. This is relatively easy, for structural data represents a very large amount of information, stemming from centuries of work in physics and chemistry. A machine that could provide adaptive control for arbitrary systems could also replace human beings in scientific experimentation and model building! In this chapter, we shall distinguish two kinds of state determination problems: (i) the observation problem, where the current state x(r) is to be determined from knowledge offuture outputs {y(s), s ~ r}, and (ii) the reconstruction problem, where the current state x(r) is to be determined from knowledge of past outputs {y(s), s :$; r}. In the first case we observe future effects of the present state and try to determine the cause. In the second, we attempt to reconstruct the present state without complete knowledge of the state transitions. 4.2 BASIC DEFINITIONS
As in the case of controllability/reachability, our principal definitions will be in terms of a certain event (r, x) being observable/constructible. We begin with observability. Definition 4.1 An event (r, x) in a real, continuous-time, finite-dimensional linear system ~ = (F( '), -, H(·)) is unobservable if and only if H(s)cI>(s, r)x = 0,
for all r
:$;
s<
00.
Here cI> is the transition matrix associated with F(· ). The motivation for this definition is clear: the "occurrence" of an unobservable event cannot be detected by looking at the output of the system after time r. Our second concept, constructibility, complements observability just as controllability complements reachability. The precise definition is stated next. Definition 4.2 With respect to the system (r, x) is unconstructible if and only if H(u)cI>(u, r)x = 0,
~
= (F(·), -, H(· )), the event
for all a
:$;
r <
00.
The motivation for this definition is suggested by statistical filtering theory. Basically, it says that the state of a system at time r cannot be determined uniquely by the system output up to time r if the state is unconstructible.
4
74
OBSERVABILITY/CONSTRUCTIBILITY
The following alternate definition of observability is often used to provide more "physical" motivation for the observability/constructibility concept. Definition 4.1' The event (r, x) is said to be observable if there exists a finite time t ~ r such that knowledge of the input u(s) and output y(s) over the interval [r, t] suffices to determine the state x uniquely. (Remark: There is no loss of generality in assuming u(s) == 0, for all s ~ r, in this definition.)
A similar definition holds to define a canstructible event. Use of the alternate definitions above then enables us to deduce the earlier definitions as consequences. For mathematical reasons, however, we prefer to retain Definitions 4.1 and 4.2 as our primary definitions.
Example: Pharmacokinetics The observability/constructibility problem is especially well illustrated by the problem of determining the concentration of a drug in a patient's body based on measurements of, say, concentrations in the urine. Specifically, we consider here the problem of cardiac patients who receive the drug digitoxin and metabolize it to digoxin. Since there is a rather fine edge between the lethal amount of digitoxin and the amount necessary to have a therapeutic effect on the patient, it is of great importance to be able to determine accurately the amount present in the body when contemplating additional doses. Thus enters the problem of observability/constructibility. The multicompartment model used to describe the kinetics and metabolism of digitoxin is shown in Fig. 4.1. Here X represents the digitoxin compartment of the body, Y the digoxin compartment, S 1 and S 2 urinary excretion sinks, S3 and S4 nonurinary sinks, while k 1, k 2 , k 3 , k 4 , and k s are diffusion rate constants.
FIG. 4.1
Multicompartment structure for digitoxin metabolism.
It is more or less standard practice to assume that when a dose of digitoxin is given, approximately 92 %of the dose is immediately taken up by compartment X and that about 85 % of the remaining 8 % is instantly taken up by compartment Y.
4.3
75
BASIC THEOREMS
The dynamics of the drug concentrations in the various compartments are assumed to be given by the equations
x=
.$1
=
-(k l
+ k 2 + k 4)X,
k l X,
.$2
Y= .$3
= k 3 Y,
k 2X - (k 3
= k 4X,
+ ks)Y,
.$4
= k s Y.
The initial conditions are X(O) = 0.92D,
Y(O)
=
(0.85)(0.08)D,
SI(O) = S2(0) = S3(0) = S4(0) = 0,
where D represents the given dose. If it is assumed that only urinary excretions of digitoxin and digoxin can be measured, then the basic question is: given measurements of SI(t), S2(t), is it possible to determine uniquely the initial state and, in particular, to determine the unknown initial dosage D? Clearly, this is a problem of observability as given in Definition 4.1. On the basis of the results to follow shortly, we shall see that with this system it is not possible to identify uniquely the initial state with only urinary excretion measurements. Thus the above system will be seen to be not completely observable. Note, however, this does not necessarily imply that D cannot be identified. It simply means that it is not possible to uniquely reconstruct the entire initial state from measurements of SI(t) and S2(t) alone. 4.3 BASIC THEOREMS
From the preceding definitions it is possible to deduce the following criteria for observability and constructibility in much the same manner as in the derivation of the analogous criteria for controllabilityjreachability.
Theorem 4.1 Let ~ = (F(·), -, H(·)) be a real, continuous-time, finitedimensional linear dynamical system. Then an event (r, x) is (a)
unobservable ifand only ifx ~ ker M(r, t)for all t > r, where M(r, t)
(b)
=
{cI>'(s, r)H'(s)H(s)<1>(s, r) ds;
unconstructible if and only if x
E
ker M(s, r)jor all s < r, where
M(s, r) = fcI>'(a, r)H'(a)H(a)(a, r) do, PROOF (a) x E ker M(r, t) ~ H(s)(s, r) = 0 for all follows similarly.
t
~ s ~ t. Part (b)
4
76
OBSERVABILITY/CONSTRUCTIBILITY
As in the last chapter, in the case of a constant, continuous-time linear system, the two notions of observability/constructibility coincide and we have a simple algebraic criterion. Theorem 4.2 If I: = (F, -, H) is a finite-dimensional, constant, continuoustime linear dynamical system, then I: is completely observable/constructible if and only if the matrix (9 = [H'IF'H'I" ·1(F,)"-lH'] has rank n. (Remark: We say that I: is completely observable/constructible whenever 0 is the only unobservable/unconstructible state.) PROOF
Word-for-word analogy to Theorem 3.9. (l)
EXAMPLES
Let F=
Then C1l{t, r)
=
diag(e 2 (t yet) =
[~~J t ),
H
=
[~ ~
e2(t- t») and
[2(te-~)
X2
oJ
1
if xC') =
c:). X2
Thus knowledge of yet) over an interval determines X2 0; however, there is no way to determine Xl O from the values of yet) over any interval t ~ r. Thus system I: = (F, -, H) is unobservable (more precisely. states of the form x' = (Xl' 0), Xl arbitrary, are unobservable). (2) (Satellite Problem) Consider the linearized dynamics of a particle in a near circular orbit in an inverse square law force field. Assuming that the distance from the center of the force field and the angle can both be measured, we have
o1 o
0 0]
0 2w 0 1 ' -2w 0 .0
1 0 0 OJ 0 1 0
H= [ 0
with y = Hx. Here ta is the angular velocity of the satellite, Yl the radial measurement, and Y2 the angular measurement. The observability matrix (9 is i o 0 0 3w 2 o 0 2 -2w _w 00100 (9= 0 1 0 0 0 o 0
r0 0 0 1 2 w
o
0
This matrix has rank 4 so I: is observable (and constructible).
4.3
77
BASIC THEOREMS
In an attempt to minimize measurements, we might consider not measuring the angle Y2' In this case, H = (l 0 0 0) and
&
~ [~ ~ I -f'J.
which has rank 3. Thus, without angular measurements, the system is not observable. In a similar way, we see that if radial measurements Yl are eliminated, :E will still be observable. In correspondence with the controllability decomposition Theorem 3.10, we have the following.
Theorem 4.3 The state space X of a real, continuous- or discrete-time, n-dimensional, linear, constant system :E = (F, -, H) may be written as a direct sum X = Xl EB X 2 with the equations of :E being decomposed as dx-fdt =
FllXb
dX2/dt = F 21Xl + F 22 X2, y(t) = H 2x 2 (t). PROOF Begin by defining X 1 as the set of all unobservable states of :E. Then proceed as in Theorem 3.10.
In discrete time, the foregoing results are expressed by the following definitions.
Definition 4.3
The discrete-time linear system x(k
+
1) = F(k)x(k)
y(k)
= H(k)x(k)
+ G(k)u(k),
(4.1)
is completely (N-step) observable at time a: ifand only if there exists a positive integer N such that knowledge of y(a: + N - 1) and u(a:), u(a: + 1), ... , u(a: + N - 2) is sufficient to determine x(a:) uniquely.
Definition 4.4 System (4.1) is completely (N-step) constructible at time a: if and only if there exists a positive integer N such that any state at time a: can be determined from knowledge of y(a: - N + 1), y(a: - N + 2), ... , y(a:) and u(a: - N + 1),... , u(a: - 1). Note that constructibility differs from observability in that in the former case we determine the" present" state from" past" data, while in the latter case we determine a "past" state from" future" measurements.
78
4
OBSERVABILITY/CONSTRUCTIBILITY
The next theorem is the main result for discrete-time systems. Theorem 4.4 System (4.1) is completely (N-step) observable at time o: only if the matrix
[H'(rx)IZ'(rx, rx)H'(rx
+
1)1·· ·IZ'(rx
+N -
2, rx)H'(rx
if and
+ N - 1)J
has rank n, where
. {F(k)F(k - 1)··· FU Z(k,j) = I,
+
k ~j, j = k + 1, j>k+l.
I)F(j),
undefined,
PROOF Identical in form to the proof of Theorem 3.7 on controllability/ reachability.
REMARK The above condition is only sufficient for complete constructibility. It becomes necessary, as well, only if the matrix F(·) is nonsingular over [rx, rx + N - 1]. Thus pointwise degeneracy would force the "present" state to be zero regardless of "past" values of y.
Example:
Input/Output Economics
Consider the very simplified dynamic Leontief system in which the production period is measured in discrete-time units. The system dynamics are
+
x(t
1) = Ax(t) .+ Ml(t),
where the production matrix has the form
a2
0 0
0
G3
0 0 0
0
0
an
0
F=
at
0
0 0
aj
~
O.
The vector x(t) represents the various products of the economic complex, with xn(t) being the finished product and Xj(t), i = 1, ... , n - 1, being intermediate products. The matrix M is assumed to be a diagonal matrix with nonnegative elements M
=
diag(ml' m2"'" m n).
The vector l(t) is the labor input to the process.
4.3
79
BASIC THEOREMS
Assume that on the basis of knowledge of the finished product xn(t) we desire to determine the level of production of the intermediate products. Thus we desire to construct the current state xlr), t ~ n, based on output measurements of xn(t). Clearly, the measured output of the economic process is given by y(t) = xn(t) = Hx(t),
where
H = (0 0· .. 0
I).
Appealing to Theorem 4.4, we compute the observability matrix
0
0
0
n
n-.
;=2
0
0
(9=
0
0
anan- t
0 an 0
0 0
0
Thus we see that the economic process is completely constructible if and only if a, # 0, i = 2, 3, ... , n. The above result also illustrates the pointwise degeneracy situation rather well since the matrix A could be singular without destroying the constructibility property. This would happen if at = O. If, however, any a., i # 1, were zero, then the system would not be completely constructible. Example:
Economics and Air Pollution
We now generalize the last example to illustrate the inclusion of nonindustrial sectors. As noted earlier, input/output analysis is a good tool for estimating the environmental and personal amenity effects of changes in the economy. The gross urban environment may be divided into several sections: the natural, community services, the sociocultural, the economic, and so on. The general problem is to show the economy's effects on the individual's environment both directly and indirectly. The first step in such an analysis is to develop linkages between particular economic activities and the affected systems. In this example, we shall look at the relation between industrial output and particulate emissions into the atmosphere. The basis approach is to postulate a dummy "atmospheric particulate matter" sector for the industrial sector. The input/output mechanism then provides a way to exhibit both the inputs and (undesired) outputs of this dummy sector.
80
4
OBSERVABILlTY/CONSTRUCTIBILlTY
We begin by expanding the original industrial sectors i = 1,2, ... , N to include antipollution activity sectors j = N + 1, N + 2, ... ,M, one for each pollutant of concern. We define output rate of industry i at time t, i = 1, ... , N, output of anitpollution activity sector i. expressed as the rate at which pollutantj is reduced,j = N + 1,... , M, rit) rate at which pollutant j is released to the air, di(t) rate of demand for industry product i to consumers, government, and export.
Xi(t) xJ{t)
The technological coefficients are aik ail ali a'm
for
input of product i required for a unit output of product k, input of product i required for a unit reduction of pollutant I, output of pollutant 1 per unit output of product i, output of pollutant 1 resulting from a unit reduction in pollutant m, i, k = 1, ... , N, I, m = N + 1,... , M.
The input/output relations are N
M
I
I
Xi(t
+
1) =
aikXk(t) + ailx/(t) k=1 /=N+1
x,(t
+
1) =
I
N
i=1
aIiXi(t)
+
+ di(t),
i
= 1, ... ,N,
M
I
m=N+1
a'mXm(t) - rl(t),
1= N
+
1,... ,M.
A plausible question to ask in the foregoing context would be whether it is possible to identify the rate at which pollution is being reduced solely on the basis of outputs from the industrial sector, i.e., we measure industrial outputs Xi(t), i = 1,... , N, and attempt to determine the pollution sector. Mathematically, the above question is equivalent to having measurements y(t)
=
Hx(t)
with x(t) = (x 1(r), ... 'XN(t), XN + 1(r),... ,XM(t))', and H = [INIOl
The system matrix F associated with the process is F = [Aik Ali
Ail ], Aim
where A ik = [aik], Ail = [ail], etc. The question of identifying the pollutant reduction may now be answered through appeal to Theorem 4.4. It is easily seen that the solution hinges critically on the properties of the matrix Ai/, linking the two sectors.
4.4
81
DUALITY
4.4 DUALITY
The reader has undoubtedly noticed a striking similarity between the definitions of the matrix functions Wand M and the functions Wand M (Theorems 3.5 and 4.1). In other words, controllability is "naturally" related (in some way) to constructibility, while observability is the natural counterpart of reachability. The most direct procedure for making this precise is to convert the integrand of W into the integrand of M. For fixed r and arbitrary real tx, the appropriate transformations are G(r
+ tx) --+ H'it
- z),
F(r
cJ>(r, r
+ tx) --+ F'(r
+ tx) --+ cI>'(r
- tx, r),
- tx).
(4.2)
Thus we take the mirror image of the graph of each function G(.), H( '), F( .) about the point t = t on the time axis, and then transpose each matrix. For controllability and constructibility, the parameter tx ~ 0, while tx $ for reachability and observa:bility. For constant systems, transformations (4.2)simplify to
°
G
--+
H',
F --+ F'.
(4.3)
The duality relations (4.2)-(4.3) are clearly one-to-one, the inverses being H
--+
G',
F
--+ F'
for constant systems and
Hit - tx)
--+
G'it
+ tx),
F(r - tx)
--+
F'(r
+ «)
for time-varying systems. In view of these remarks, we can give criteria for observability and constructibility in terms of reachability and controllability and vice versa. For example, we have Theorem 4.5. Theorem 4.5 The pair of matrix functions F(t), H(t) define a completely observable system:t at time r if and only if the matrix/unction F*(t) = F'(2r - r), G*(t) = H'(2r - r) define a completely reachable system:t* at time r.
We shall return to the "duality principle" in Chapter 6, where it will be used to help establish the canonical decomposition theorem for linear systems. This foundational result states that any finite-dimensional, linear, dynamical system may be decomposed into the four disjoint parts: (1) completely controllable/completely observable, (2) completely controllable/unobservable, (3) uncontrollable/completely observable, and (4) uncontrollable/unobservable. In Chapter 6, we shall thoroughly examine the philosophy stimulated by this basic theorem and the techniques that have been developed to find part (1)from input/output data.
82
4
OBSERVABILITY/CONSTRUCTIBILITY
EXERCISES
1. Prove that dim ker M(r, r) equals (a) n - dim range M(r, t), (b) n - rank[H'IF'H'I·· ·1(F')"-lH'] for all t > r. 2. Show that the system I: = (F, -, H) is completely constructible if and only if the system I:* = (F', H', -) is completely controllable. 4.5 FUNCTIONAL ANALYTIC APPROACH TO OBSERVABILITY
It is possible to attack the observability question from an entirely different point of view than that taken above and, as a result, to obtain additional insight into the basic concepts involved. In this section we utilize some elementary facts from functional analysis and convex sets to prove the timevarying version of Theorem 4.2 for single-output systems. As a consequence of this line of attack, we shall also obtain some results on the classical problem of moments similar to those presented in the last chapter. We begin with a slight generalization of our earlier definition of observability.
Definition 4.5 Let q be an n-dimensional vector. Ilqll = 1. Then we say the single-output system I: = (F, -, h) is observable in the direction q at time t 1 if there exists a measurable function ~(t) such that (q, xo)
= {' hx(t; xo)~(t)
(4.4)
dt
for any Xo E X. (Here x(t; xo) is the state at time t given that the system began in state Xo and no input was applied.) Since x(t; xo) = C1l(O, t)xo, where
t)~(t)
dt Xo'
(4.5)
We impose the constraint
IW)I 5, L
for all t
in order to seek a number L for which the observability problem has a solution. It is clear that equality holds in Eq. (4.5) if and only if for some L the direction q lies in the set Q(L)
= {q:q = {l
IW)I5, L}.
(4.6)
4.6
83
THE PROBLEM OF MOMENTS
Generally speaking, the problem of observability for the direction q has a solution if and only if the distance p(L)
= min Ilq - iill ijeQ(L)
from the set Q(L) to the direction q equals zero for some L. Since Q(L) is a convex, closed set for each L, we may use the min-max theorem. We have max (z, (q - ii)) = max min (z, (q - ii)).
1JtL) = min ijeQ(L)
11%1/51
11%1/51
ijeQ(L)
Substituting the expression for ii from (4.6), and replacing the operation of min over ii by the operation of min over ~, I~ I ~ L, we have p(L)
= max
11%1151
(z,
q- L
f
ll l
0 .
l(t, 0)h'1 dt).
In order that for every q there exist a number L for which p(L) = 0, it is necessary and sufficient that (z, '(t, O)h')
'1= 0
(4.7)
for all z, II z II ¥- O. Indeed, if for some z we have (z, '(t, O)h') = 0, then the direction q = az is not observable: p(L) = lal > O. If (4.7) is satisfied, then setting L =
Ilqll! min fl'I(Z, '(t, O)h') I dt, 11%1151
0
we obtain p(L) ~ 0, i.e., the system is observable. Note that in the constant case we have {t,O)
=
eFI,
which, coupled with (4.7), implies Theorem 4.2. 4.6 THE PROBLEM OF MOMENTS
If we pass to a coordinatewise description of Eq. (4.4) or (4.5), then the condition
f
ll
q =
0
'(t, O)h' W) dt
(4.8)
for solvability of the problem of observability in direction q may be treated as a problem of moments and we again arrive at (4.7). Let us sharpen this result. We integrate the right side of (4.8) n times by parts. Leaving the algebraic details to the reader, we arrive at the following result.
4
84
OBSERVABILITY/CONSTRUCTIBILITY
Theorem 4.6 The observable directions of the system ~ those, have the representation
=
(F, -, h), and only
n-l
q =
L IllF')ih',
111;1 <
i=O
00.
EXERCISES
1. Prove Theorem 4.6, both (a) directly and (b) by an appeal to duality and the results of Chapter 3. 2. Formulate and prove the multioutput version of Theorem 4.6. MISCELLANEOUS EXERCISES
1. Show that the following conditions are equivalent: (a) (b) (c) (d)
The constant system ~ = (F, -, H) is completely observable. The identity HeFlxo == 0 (t > 0) is possible only for xo = o. No characteristic vector x of F satisfies the condition Hx = O. There are no nonzero vectors 9 such that the expression H(aI - F)-lg is identically zero for a not a characteristic root of F. (e) For every pair of numbers t 1 and t 2 > t 1, the matrix
is positive definite. 2. The variable x is assumed to satisfy the differential equation x(t) + x(t) = O. If the values of x(t) are known for t = n, 2n, 3n, . . . , can x(O) and X(O) be uniquely determined from this data? 3. Show that the constant system
x=
Fx
+ Gu
is completely controllable if and only if the system
x=
-r
F'x,
y= G'x is completely observable. 4. Show that the constant system ~ = (F, -, H) can be completely observable only if the number of outputs (number of rows of H) is greater than or equal to the number of nontrivial invariant factors of F'.
85
NOTES AND REFERENCES
5. The direction q, system
I qI
= 1 (at the point x = 0) of the single-output
x=
Fx,
y = h'x is called T -indifferent to the observation y if for all initial conditions of the form xo = aq,
1rx.1
<
00,
the output y(l) does not depend on rx. for any i in the interval [0, T]. Show that the direction p, II p II = 1, is T-indifferent to y if and only if
= '" = p'(F n - 1)'h = O. 6. The set of directions Pj' II p II = t,} = 1, ... , r (at the point x = 0) of the p'h
=
p'F'h
system
.x
= Fx
Yj
= h/x
with outputs is called autonomously T-observable if each direction Pj is T-observable relative to the output y j and T -indifferent to observations in the other outputs. Prove that the collection of directions Pj' II Pj II = I,} = 1, ... , r, of the system x = Fx are autonomously observable relative to the outputs Yj if and only if n-l
n.-l
k=O
k=O
L Ip/(Fk),hj I > 0,
L Ip;'(Fk)'hjl =
0,
for i =1'}, i.] = 1, ... , r. NOTES AND REFERENCES
Section 4.1
The adaptive control discussion is pursued in greater detail by
Kalman, R., Falb, P., and Arbib, M., "Topics in Mathematical System Theory." McGraw-Hili, New York, 1969.
A somewhat different, but most illuminating, discussion is found in Bellman, R., "Adaptive Control Processes: A Guided Tour." Princeton University Press, Princeton, New Jersey, 1961.
Section 4.2
The basic definitions follow from
Kalman, R., "Lectures on Controllability and Observability." C.I.M.E., Bologna, Italy, July 1968; Edizioni Cremonese, Rome, Italy, 1969, pp. 1-149.
86
4
OBSERVABILITY CONSTRUCTIBILITY
See also the work Sontag, E., On the lengths of inputs necessary in order to identify a deterministic linear system, IEEE Tran. Auto. Cont., AC-25, 120-121 (1980).
The example on pharmacokinetics is taken from Jeliffe, R., et al., A mathematical study of the metabolic conversion of digitoxin to digoxin in man. USC Rep. EE-347, Univ. of Southern California, Los Angeles, 1967.
A related paper is Vajda, S., Structural equivalence of linear systems and compartmental models, Math. Biosci., 55, 39-64 (1981).
Section 4.3 The satellite example, along with numerous other problems of engineering interest, is discussed by Brockett, R., .. Finite-Dimensional Linear Systems. " Wiley, New York. 1970.
The discrete-time results are taken from Weiss, L.. Controllability, realization. and stability of discrete-time systems. SIAM J. Control 10, 230-251 (1972).
See also Hamano, F. and Basile, G., Unknown input, present state observability of discrete-time linear systems, J. Optim. Th. Applic.; 40, 293-307 (1983). Delforge, J., A sufficient condition for identifiability of a linear system, Math. Biosci.; 61, 17-28 (1982).
A deeper discussion of the input/output economics example is found in Tintner, G .. Linear economics and the Boehm-Bawerk period of production, Quart. J. Econ. 88. 127-132 (1974).
See also the book by Helly, W., "Urban Systems Models." Academic Press, New York. 1975.
for a treatment of numerous social problems possessing system-theoretic overtones. Section 4.4 The first published statement of the concept of observability and of the duality principle is given by Kalman, R.. On the general theory of control systems, Proc. lst IFAC Congr., Moscow. 481-492 (1960).
Sections 4.5-4.6 An extensive treatment of the observability question from the functional-analytic point of view is found in Krasovskii, N. N .• "Theory of Controlled Motion." Nauka, Moscow, 1968.
NOTES AND REFERENCES
87
See also the treatment by Gabasov, R.. and Kirillova, F., "Qualitative Theory of Optimal Processes." Nauka, Moscow, 1971. (Engl. transl.: Dekker, New York, 1976.)
A detailed treatment of the classical moment problems and its many ramifications in mathematics and science is given by Akhiezer, N., "The Classical Moment Problem and Some Related Questions in Analysis." Hafner, New York, 1965.
CHAPTER
5
Structure Theorems and Canonical Forms t
5.1 INTRODUCTION
One of the basic tenets of science, in general. and mathematical physics, in particular, is that fundamental system properties should be independent of the coordinate system used to describe them. In other words, the properties of a system or process which we are justified in.calling "basic" should be invariant under application of an appropriate group of transformations. For example, the invariance in form of Maxwell's equations under the Lorentz group of transformations is a central aspect of relativity theory. Similarly, the invariance of the frequency of an harmonic oscillator when viewed in either a rectangular or polar coordinate system illustrates the fundamental nature of this system variable. In mathematical system theory, one of the primary objectives is to discover properties about systems which are, in some sense, fundamental or t In previous chapters it has been our policy to motivate and illuminate the basic theoretical results with numerous applications from diverse areas of human endeavor. The current chapter, however, is intended primarily for" connoisseurs" of system theory and, as a result, is almost exclusively theoretical in character with the exception of some numerical examples. The reader whose tastes do not run toward the theoretical can safely proceed to Chapter 6 at this point without loss of continuity; however. for the sake of broadening one's scientificand mathematical culture, we do recommend that this chapter at least be skimmed before proceeding to the following material.
88
5.1
INTRODUCTION
89
basic to all systems of a given class. Of course, the decision as to what constitutes a "basic" property is to a certain degree a subjective one determined by the tastes and motivations of the analyst. However, coordinatefree properties certainly have a strong claim to being regarded as basic system properties and to a large extent our discussions in this chapter will be devoted to an examination of such system features. In order to isolate coordinate-free system properties, it is necessary to specify a particular group of transformations <§ for study. Our problem then reduces to a study of the structural features ofthe system I: when it is subjected to transformations from <§. For each transformation 9 E <§, I: will assume a different form dependent on the particular g. Hopefully, some of these forms will be particularly simple in the sense that the structural features of I: that are invariant under <§ will be evident by inspection. Roughly speaking, such a simple form of I: will be called canonical. Thus our general plan of action is to: (i) isolate certain system-theoretic features f!IJ as candidates for" basic" quantities; (ii) specify a particular transformation group <§ that is to act on I:; (iii) investigate whether the features f!IJ of (i)are invariant under transformations from <§ and, if so, determine appropriate canonical forms to exhibit the features f!IJ explicitly. At this point, it is natural to ask about the utility of canonical forms. Aside from their aesthetic aspects, do such forms have any practical utility? The answer to this question is that the canonical forms have considerable practical value, particularly in so-called system identification problems. The point to be noted is that, in general, the canonical forms represent I: in such a way that a minimal or nearly minimal number of parameters are used. Since the canonical form possesses the same "essential" structure as the original problem, we see that for system identification problems it is highly desirable to make use of canonical forms in order to reduce to a minimum the number of unknown parameters characterizing the system. Naturally, there are several different canonical forms and" basic" properties for a given system I:, depending on the particular set f!IJ and transformation group <§ selected by the system analyst. In this chapter, we shall confine our attention for the most part to cases for which f!IJ = {controllability! reachability} or f!IJ = {observability/constructibility}, <§ = {linear transformations in X} (state variable transformations), <§ = {linear coordinate transformations in X and n}, or <§ = {linear transformations in X and n plus state variable feedback}.
90
5
STRUCTURE THEOREMS AND CANONICAL FORMS
5.2 STATE VARIABLE TRANSFORMATIONS
We begin our investigation into the invariant properties of a linear system by considering the time-varying system ~ = (F(t), G(t), H(t» under the action ofthe general linear group ~ = GL(n) (which is the group of real, nonsingular n x n matrices). If we consider a fixed member T E GL(n), its action on ~ is given by
x
-+
Tx,
F(·)-+ TF(·)T- t ,
G( . ) -+ TG(·),
H(·)
-+
H(·)T- t . (5.1)
Our interest is in determining properties of ~ that remain invariant under GL(n). Our first result in this direction is as follows. Theorem 5.1 Let ~ = (F(·), G('), -) be completely reachable and/or completely controllable. Then the new system obtained from ~ by the state space change of basis (5.1) is also completely reachable/controllable. PROOF We prove the result in the case of reachability. By Theorem 3.5, ~ is completely reachable if and only if the matrix
Jot(s, T) = f(T' a)G(a)G'(a)'(T, a) do is positive definite for some s < verified that
T.
Jot(s, T)
Under the transformation (5.1), it is easily -+
TJot(s,
-vi:
i.e., W(s, T) is congruent to the transformed W: Standard results from matrix theory show, however, that the property of positive-definiteness of a matrix is preserved under congruence. Thus if Jot(s, t) is positive definite for some s < T, then so is the Jot associated with the transformed system. The proof for controllability is completely analogous, utilizing the matrix W of Theorem 3.4.
»
Corollary If ~ = (F(·), -, H( . is completely constructible/observable, then so is any system obtainedfrom ~ under the action ofthe state space transformations (5.1). PROOF
The proof is the same as for Theorem 5.1 using the matrices M,
M of Theorem 4.1.
Theorem 5.1 shows that two representations of the same system are either both completely controllable/reachable or they both fail to possess these
5.3
91
CONTROL CANONICAL FORMS
properties. An interesting question that arises is whether or not the transformation T:X ~ X is uniquely determined by the two system representations ~ = (F, G, H),:t = (P, G, fl). The answer is given by Theorem 5.2. Theorem5.2 Given two constant systems ~ = (F, G, H), :t linear transformation T: X ~ X is uniquely determined by~,:t :t are completely controllable (or completely observable).
= (F,G, fl), the if and only if~,
PROOF Assume~,:t are completely observable. Consider first the case m = 1, i.e., G is a single column vector g. By complete controllability, we know that the matrix
is nonsingular. Also, by the properties of transformation (5.1), it is easily verified that
so ~ is also nonsingular. Thus for m = 1 we find that T is uniquely determined as
In the case m > 1, it makes no sense to speak about the inverse letting r = rank G, we form the matrix
of~.
Thus
C = [GIFG I···' Fn-'G] and let D = ce. Then D is n x n and is nonsingular since ~ is completely controllable (see Exercise 4 of Chapter 3). Hence, TC = C implies TCe = TD = and we find that
cc
5.3 CONTROL CANONICAL FORMS
Since the properties of controllability/reachability-observability/constructibility satisfy our requirements for being" basic" system attributes in that they remain invariant under a change of coordinate system in X, we now seek to simplify the controllability/observability criteria for constant systems with a special choice of basis in X. The system theory literature has seen the introduction of several different "canonical" representations of completely controllable/observable systems,
92
5
STRUCTURE THEOREMS AND CANONICAL FORMS
each representation being championed for a different purpose. Clearly, an uncountable number of such representations are possible. Our treatment will be confined to an account of the two most prevalent (and seemingly useful) such representations. The first is the so-called control canonicalform, which is a generalization of the companion matrix form of classical linear algebra. The second form we study is based on the well-known Jordan canonical form of a single matrix. This is the Lur'e-Lefschetz-Letov canonical form, introduced to extend the Jordan form to the case of a pair of matrices (F, G).
Consider first the case of a single-input constant system Let us write the characteristic polynomial of F as
~
= (F, g,
-).
Introduce the set of vectors
en-l = Fg en = g.
+ (1.lg,
Then the set {eJ forms a basis for X if and only if (F, g) is completely controllable. The proof of this observation is immediate since {el, e2,"" en} is a triangular linear combination of the vectors {g, Fg, ... , Fn-lg}. Thus {el,"" en} form a basis for X because, by complete controllability, the same is true for {g, Fg, ... , F"-lg}. Utilizing the vectors {el,' .. , en}, we have the so-called control canonical form.
Theorem 5.3 In the basis {el"'" eJ, the matrices F and 9 have the representations
F=
o o o
o o
o
0
I
o 1
o 0
o o 1
0
o o o
o
o g=
o o 1
PROOF Compute Fel, Fe2"'" Fen and note that Fe, = -(1.nen, by virtue of the Cayley-Hamilton theorem ('"F(F) = 0).
5.3
93
CONTROL CANONICAL FORMS
The origin of the control canonical form is closely related to the standard trick of converting the nth order linear differential equation (5.2) to a system of n first-order equations dx-fdt = X2' dX2/dt = X3'
(5.3)
The passage from (5.2) to (5.3) is accomplished by introducing the state variables i
= 1,2, ... ,
n.
(5.4)
There are three points that Theorem 5.1 improves upon regarding the transformation (5.2) --+ (5.3) and the classical discussions surrounding it: (i) It is not immediately apparent that (5.2) represents a completely controllable system. (ii) It is sometimes believed (and even taught) that (5.4) is the only way to convert (5.2) into the state variable form. This is false. The system 1:, which represents the minimal realization of the input/output function u(t) --+ yet) defined by the equation n
m
;=0
;=0
L (Xi diy/dt i = L Pi d'uldt',
m
< n, (Xo = l ,
(5.5)
is completely controllable and, as a .result, also admits the control canonical form. The difference between (5.5) and (5.2) is expressed through H. In the first case
H = [t while in the second case n-l
y=
L PiXi+l,
;=0
0···0],
94
5
STRUCTURE THEOREMS AND CANONICAL FORMS
(iii) Equation (5.4),which expresses the state variables in terms of derivatives, works well for (5.2) but it is not valid, in general. For example, it does not work for (5.5). Thus what is important here is the control canonical form and not the special formula (5.4). Now we turn to the system-theoretic extension of the Jordan canonical form, the Lur'e-Lefschetz-Letov canonical form. Recall from linear algebra that any n x n matrix A may, in an appropriate coordinate system, be expressed as
A =
0 A2
[A'~ 0
0
~l
0
Ak
where each Jordan block Ai has the form
Ai
=
il
0 Ai I
[A~ 0
0
i
= 1,2, ... , k,
0 Ai
the Ai being the characteristic roots of A. The generalization is stated next. Theorem 5.4 If the single-input system (F, g, -) is completely controllable, then there exists a unique nonsinqular matrix T such that the matrix F = TFT- 1 and the vector g = Tg have the Lur'e-Lefschetz-Letov form
Fl
-
0
F=
g- -
.
[
o o o
[::]
. ,
gk
where the F j are Jordan blocks with nj rows and columns of the form
Fj
~ [At
1
0
Aj 1 0
0
...
Jl
Aj
j = 1,2, ... k,
5.3
95
CONTROL CANONICAL FORMS
and the vectors gj,j = 1,2, ... , k have nj components and are given in the form
o o
k
I
o
nj = n.
j= 1
1 The numbers A.j and nj are determinedfrom the characteristic polynomial ofF by k
XF(Z) =
IT (z -
ic)"i.
j= I
PROOF We sketch the necessary steps, leaving the reader to fill in the details as exercises. The general plan is to use the chain of implications:
(i) The controllability matrix rc associated with the control canonical form is nonsingular. (ii) rc nonsingular implies that the equations TF - FT = 0,
Tg = d,
admit a unique solution T for every vector d. (iii) For any other completely controllable pair (F, g) such that
there exists a unique nonsingular matrix T such that
9=
Tg.
(iv) Step (ii) implies the Lur'e-Lefschetz-Letov form. Note that Step (iii) is necessary to ensure uniqueness of the transforming . matrix T. The multi-input versions of these canonical forms are similar. For example, the control canonical form is given by the next theorem. Theorem 5.3' Let 1: = (F, G, -) be a completely controllable constant linear system. Then there exists a nonsinqular matrix T such that the transformed system! = (F, G, -) is given by
G=
TG,
96
5 STRUCTURE THEOREMS AND CANONICAL FORMS
where
kl
o o
col~mns
I
0
o
,, ,,
x
·, , ·:, 0
X,X
a.
1__ •
••
X
X
x
...
_
I
0
xix
X
,, ,, ,
x
...
:, 0
o
k l rows
k 2 rows
·
X
--- --------- --- --- --- t----------- -.. - --- --_.. -:- ---- ---- ..... ------ ----
.,
, ,
: m'
,: : :1
~-T-fT---'
O!', .. .. .. .. -:, :, ,,
m - m' columns ,
columns ,
~,:
..
:
1
r-~- - - ~- - - ~
.,,
,, ,
:,
:,
~
~
,
:, ,,
,
k 2 rows x
x
x
..
:, ,,
The numbers {k j } are positive integers such that k l + k 2 + ,.. + k s = n, while m' = rank G < m. The elements marked "x " in theforms P, G represent invariants of the action and are determined by F, G and the transformation T.
We shall not prove this result here since we will discuss it in some detail in a later section. It should be noted, however, that the form for G simplifies
5.4
OBSERVER CANONICAL FORMS
97
in the generic situation in which G is of full rank (m = m'). The numbers k, are identical to the well-known Kronecker indices for a pencil of matrices as we will elaborate in Section 5.6. EXERCISES
l. Prove Theorem 5.2 according to the scheme of implications given in its statement. 2. Define a relation 9i on completely controllable pairs (F, g) by the rule
if and only if there exists a nonsingular matrix T that F 2 = T FIT - 1, g2 = Tg 1 • Show that (a) 9i is an equivalence relation, i.e., it is reflexive, symmetric, and transitive. (b) If (F, g) is completely controllable, then so is any pair in the same equivalence class as (F, g). (c) If (F. g) and (F, g) are two completely controllable pairs, then there exists a nonsingular matrix T such that (F, g) 9i (p, g). (We shall greatly extend this result in a later section to include a much richer set of transformations than just basis changes in the state space x.) 3. Determine the Lur'e-Lefschetz-Letov form for the multi-input case.
5.4 OBSERVER CANONICAL FORMS
According to the duality principle described in the last chapter, we may associate a "dual" canonical form to each of the controller forms presented in the previous sections. It is clear that the same procedures employed in deriving the control canonical form and the Lur'e-Lefschetz-Letov form may be employed for obtaining. a corresponding observer form by making the transformations F --. F'. G -. H'. Rather than boring the reader with this repetitious exercise, we content ourselves solely with a presentation of the result. The observer canonical form, corresponding to the form of Theorem 5.3, is given by the next theorem. Theorem 5.5 Let the constant system I: = (F, -, H) be completely observable. Then there exists a coordinate transformation T in X such that the
98
5
system!. form
STRUCTURE nrnOREMS AND CANONICAL FORMS
= (P, -,R) = (TFT- 1, -,HT-1)assumes the observer canonical
.
J1.1 columns
o
0 1 0
o
0
x x
x
_ _ _ _ _ _ _ .....
o --- -----
o o
""
,
o o
0 0
o
0
x :, , x : x
.,,, .,
... __ • __ .. __ ... -1
.
o
x ' 0 0 x . 1 0
o
,,
x : 0 0
-- .. --_ ..-- ------:--------- --- --- --_
..........
x x x . -- -:,,, ,,
..
..
, ......... --- -- -_ ............. --- --- -------------- --- --_ ... -- - ----- -----
.
J1.2 columns
(Here, p' = rank H ~ p, while the positive integers {J1.i} satisfy the same 1 J1.i = n.) constraints as the {kJ, i.e.,
D=
The Lur'e-Lefschetz-Letov observer form is obtained in a similar manner by an appeal to duality and the controller form. EXERCISES
1. Explicitly write the Lur'e-Lefschetz-Letov observer canonical form. 2. In either the observer canonical form or in the Lur'e-Lefschetz-Letov observer form, what happens to the input matrix G under the change of coordinates T in X? Is it possible to simplify the canonical form further by use of additional transformations, e.g., in O?
5.5
99
INVARIANCE OF TRANSFER FUNCTIONS
5.5 INVARIANCE OF TRANSFER FUNCTIONS
Earlier we saw that the input/output characteristics of a constant linear system having m inputs and p outputs could be described by a p x m matrix W(t), the impulse-response matrix, or its Laplace transform Z(A), the socalled transfer function matrix. A basic structural question is to determine what aspects of the transfer matrix remain unchanged when the system 1: is transformed in one way or another. In this section we confine our attention to the group of state space coordinate transformations T. Since the transfer function relates the output to the input spaces without any direct connection through X, we would suspect that a change of basis in X would leave Z(A) invariant. In fact, if this were not the case, it would imply that the input/output behavior of 1: was dependent on the coordinate system chosen in X, implying that the input/output map is not an intrinsic feature of the system. Since in many cases, the mapfis our only experimental information about 1:, such a situation would completely destroy any attempt to construct an internal model of the observed process. Thus our objective in this section is to validate the feasibility of model building by proving the in variance of Z(A) under coordinate changes in X. Recall the definitions of Wand Z. We have the system 1: such that
x=
+ Gu,
Fx
y=Hx. An elementary calculation gives the connection between y and u as
y(t) = H when x(O)
=
J~eF(r-')GU(S)
ds,
O. If every component of u equals zero except uJ{t), i.e.,
o o u(t) =
Ole)
o 1
+--
jth position,
o o = {o/e)e j , 0,
on an interval 0 elsewhere,
~
t
~
e
100
5
STRUCTURE THEOREMS AND CANONICAL FORMS
then y(t)
= HeFt(l/t;) {e-FSGe j ds,
Passing to the limit as s
--+ 0,
t
~
s > O.
we obtain the relation t ~ O.
y(t) = HeFtGej'
Generally speaking, the (i, j)th element of the matrix W(t)
= HeFtG
gives the ith component of the output y;(t) corresponding to the unit impulse input u(t) = c5(t)ej' where c5 is the Dirac measure of weight 1 concentrated at t = O. The matrix W(t) completely defines all connections between the inputs and outputs of a controllable and observable system. Thus, for arbitrary input u(t), we have y(t) =
f~
W(t - s)u(s) ds,
t
~
0,
when x(O) = O.
Since the correspondence between u(t) and y(t) has the form of a convolution, it is more convenient to apply the Laplace transform .P to the functions u and y. Denoting these transforms by U(2) and Y(2), we obtain the matrix transfer function Z(2):
L Xl
Z(2) = .P(W(t» =
W(t)e-.l.tI dt.
The connection between the input and output then assumes the form Y(A) = Z(A)U(A). More explicitly, the transfer matrix is given by Z(2) = H(AI - F)-lG. By Cramer's rule, we see that the elements of Z are rational functions of 2, with the degree of the denominator equal to the dimension of ~ (before any possible cancellation of terms in the numerators and denominators takes place). In fact, the denominator of each component in Z(2) equals XF(2), the characteristic polynomial of F. Our basic invariance result is stated next. Theorem 5.6 The matrix impulse response function W(t) (and consequently, its Laplace transform Z(A» is invariant with respect to linear changes of coordinates in the state space X.
5.6
CANONICAL FORMS AND THE BEZOUTIANT MATRIX
101
PROOF Let T E GL(n) represent a change of coordinates in X. Under such a transformation, the system matrices F, G, H transform as
G -+ TG,
Substituting the transformed elements into the definition of Wet), we have Wet) = (HT-l)eTFT-'I(TG).
The theorem follows upon application of the well-known identity e AI = J
2
+ At + A 2 -t + ... 2!
'
convergent for all t. The result for Z(A) follows from the 1-1 correspondence between W(s) and Z(A). EXERCISES
1. Compute the transfer matrix when the system l: = (F, G, H) is: (a) in control canonical form, (b) in the Lur'e-Lefschetz-Letov canonical form. Do the same for the observer canonical forms. 2. What is the significance of a component in the transfer function having a numerator with a factor in common with XF(A)? (Hint: Consider the single input/single output case first.) 3. Is there any system-theoretic interpretation attached to the vanishing of one or more components in Z(A)? 4. Construct a completely controllable and completely observable system l: whose transfer function is
0]
2
A/(A Z(A) =. A/(.P + A) . [ 2
A/V
-
A)
5.6 CANONICAL FORMS AND THE BEZOUTIANT MATRIX
Both the observer and controller canonical form studied thus far came about as the result of a change of coordinate systems in the state space X. A natural question to pose is whether there is some interesting and useful connection between the two forms. More precisely, if T; and r;, represent
5
102
STRUCTURE THEOREMS AND CANONICAL FORMS
the basis changes for the controller and observer canonical forms, respectively, we have and our question becomes: What are the properties of a matrix B such that BF cB- 1 = F o ?
It is clear that such a matrix B exists (it equals 1;, T; 1); however, it is not immediately apparent that B possesses any interesting properties. Our purpose in this section is to show that not only does B have interesting features justifying our attention, but that it is intimately related to the classical Bezoutiant matrix of polynomial algebras. This fact enables us to obtain some useful results concerning transfer functions. Consider two polynomials IX(S) and f3(s) such that IX(S) = IXO + a 1s + a2s2 + 2 f3(s) = bo + b 1s + b 2s +
... + ansn, '" + bnsm
with real coefficients. Without loss of generality, assume that an = 1 (a is monic) and m < n. Then the Bezoutiant matrix of a and 13 is the n x n matrix B[a
13] (a O b 2 ) . (a O b 3 ).
(aOb4).
+ (a1 b 2 ). + (a1 b 3).
(G Ob 3).
. .. + (a 1b 3 ). .. 'J , + (a1 b4). + (a4b J).
(ao b4 ) .
(aOb 5 ) .
where (ajb j). = a.b, - ajb j.
More explicitly, if k = min(i,j) - 1, then the element [B]ij is given by (aO b j +j _ 1).
+ (a 1bi+j-2). + ... + (akbi+j-k-l).'
where we take aj = 0, b p = 0 for j > nand p > m. It is well known that the polynomials a(s) and f3(s) are relatively prime if and only if B[IX, 13] is of full rank and that the rank deficiency of B[a, 13] equals the degree of the greatest common factor between IX and 13. To relate the Bezoutiant to linear systems, we consider a single-input! single-output system 1: = (F, g, h) having transfer function I3(s)!IX(s). The controller canonical form of this system is given by Xc = Fcx c + gcu, Y = hex",
5.6
103
CANONICAL FORMS AND THE BEZOUTIANT MATRIX
where
0 0
1
0
0
1
0 -ao
0 -al
0 -a2
0 0
(Note: here the coefficients a, are labeled to agree with the
Fc =
labeling in the polynomial
rx(s)),
-an-I
0 0 ge =
he = [ho b l
o
· · .
b.; 0··· OJ.
1 The dual of the controller form, the observer canonical form is obtained from ~e = (Fe, ge' he) by means of the relations
It is a simple task to verify that the transfer function for both systems equals fJ(s)/rx(s). In fact, there will be an uncountable number of systems ~ with transfer function fJ(s)/rx(s). Anyone of these systems is referred to as a realization of the transfer function fJ(s)/rx(s). Let us introduce the controllability and observability matrices CC and (f) as CC(F, g) = [gIFgl" ·IFn-lg],
(f)(h, F) = CC'(F', h').
A basic role in relating the Bezoutiant to the observer and controller canonical forms is played by the following result.
Theorem 5.7
The Bezoutiant matrix B satisfies the identity
where
and -al -a2
-a2
-an-I -1
-1
-an-I
-1
T=
PROOF
o
The proof is accomplished by direct computation and verification.
104
5
STRUCTURE THEOREMS AND CANONICAL FORMS
The basic result relating e, (= rtf(Fe , ge)), (20 (=rtf'(Fo', ho')), «, (=rtf(Fo' go)) and (2e (=rtf'(Fo', he'» follows. Theorem 5.8
B(~,
(d)
B(~,
PROOF
/3(Fc ) =
B(~,
/3) satisfies the following
identities
/3) = -rtf; l(2e' /3) = -rtfo(2;;1, /3) = - (2;; l(2e, /3) = -se,«;',
B(~, B(~,
(a) (b) (c)
The Bezoutiant matrix
Part (a) follows immediately from the relations rtf; 1 = T, (2c'
Part (b) follows from (a) by using the symmetry of Bta; /3) since Bi«, /3) =
B'(a, /3) = - (2o'(rtf eT 1. This fact, together with the duality relations (20' = rtf0' rtf0' = (20 establishes (b). The duality relations plus the invertibility of T = rtfe- 1 imply (c), which in
turn implies (d).
Conclusion (c) of the theorem leads to a very simple proof of the fact that any observable realization on; = (F, g, h) can be transformed to the observer canonical form by a change of state variables as xo(t) = Mx(t),
where M is the constant nonsingular matrix M =
(2;; 1(2(F,
h).
Thus, if the controller canonical form is observable, from conclusion (c) we see that B(a, /3), the Bezoutiant, is precisely' the transformation matrix needed to form the observer canonical form. EXERCISE
1. Establish the corresponding results for multi-input/multi-output systems. 5.7 THE FEEDBACK GROUP .AND INVARIANT THEORY
We have seen that the group of linear coordinate transformations in X, the state space, enables us to reduce the apparent complexity of a given system l: significantly by reducing it to a canonical form in which the inherent structure of E is more apparent. The obvious question at this stage is whether a further simplification is possible if we augment our transformation group by allowing not only basis changes in X, but also other coordinate and structural changes in l:. For a variety of reasons, some of which we shall see below, the most interesting new transformations are changes of basis in the input space 0,
5.7
105
THE FEEDBACK GROUP AND INVARIANT THEORY
and application of special inputs of the form u(t) = - Lx(t), where L is an arbitrary n x m constant matrix. Such inputs are termed "feedback" since their effect is to generate the input not as an explicit function of time, but as a function of the state x. Thus our new group of transformations consists of: (I) coordinate changes in X; (II) coordinate changes in il; (IiI) arbitrary constant feedback laws. This set of transformations forms what is generally called the feedback group !F. Given a system 1: = (F, G, H), the action of !F on 1: is (coordinate change in X) (II) (coordinate change in il) (III) (feedback law) (I)
F
-+
TFT- 1 ,
G-+GV- 1 ,
F -+ F, F
-+
H -+ HT-l,
G -+ TG,
H-+H,
G -+G,
F - GL,
H-+H,
ITI"# 0,
IVI"# 0, L arbitrary.
A particular element of !F is determined by specifying the matrices T, V, and L. Clearly, the choice V = I, L = 0, T nonsingular reduces to the case of state coordinate changes considered above. Thus the state space basis changes form a subgroup of !F. We have seen that under coordinate changes in X, the only invariants are the coefficients {IX;} of the characteristic polynomial of F (or equivalently, the characteristic values of F) and the elements of HT- 1 (assuming rank G = m). Since the feedback group allows more flexibility in modifying the structure of 1:, the number of invariants is certainly no greater than under the subgroup of state space basis changes. Our purpose is to determine the precise invariants. We begin by considering the ordered set of vectors gl, ... ,gm;Fg 1,F9 2 , ••• ,Fgm, ... ;Fn-lgI, ... ,Fn-lgm, (5.6) where gi = ith column of G. Under the assumption that the system 1: = (F, G, -) is completely reachable, the list (5.6) contains precisely n linearly independent vectors. Consider the" Young's diagram" I
F
F2
...
Fn -
91
X
X
X
.. ,
X
92
X
k2
93
X
k3
l
kl
106
5 STRUCTURE THEOREMS AND CANONICAL FORMS
where t, is the number of crosses in column (i + 1), and k j is the number of crosses in row j. Here the rule for placing crosses in the diagram is as follows: begin with row 1 and place a cross if element (i, j) is linearly independent of all vectors previously considered. Then go to row 2 and repeat the process, etc. By the complete reachability of 1:, this procedure will result inexactly n crosses being placed in the diagram. The integers {kj } , {l;} must satisfy s-1
Lk; = L t, = n.
t> 1
i."O
The vectors picked by the above procedure, namely, kj - 1 . · { gj' .•• , P gj • ]
E
M},
where M is a subset of {I, 2, ... , m} containing exactly s elements, constitute a basis for X. Let M(i)
= {tEM: k, > i},
i
= 0, 1,2, ... , n -
1.
Then M(i) has 1; elements and M(O) = M. By linear dependence, we have pk'gi
=
L
ki-1
(Xji(kj+ 1l
P k igj
+ L
L
(Xji(d l)pTg j,
iEM,
(i)
r=O jeM(Tl
jeM(k;j
e. = L (Xj;1gj'
i = 1, 2, . :. , m,
i¢M.
(ii)
jeM
We now make the transformation iEM.
(iii)
It then follows from (iii) and the definition of M(i) that gi
= {l; -
L
iEM.
'(Xji(kj+1 l0j, jeM(k;j
Upon substituting (iv) into (i) and (ii), we obtain pk,O;
=
k,-1
L L
(Xji(T+1l
p tOj'
iEM,
r=O jeM(tl
o, = L (Xji10j, jeM
i
= 1,2, ... , m,
i¢M.
(iv)
5.7
107
THE FEEDBACK GROUP AND INVARIANT THEORY
A new basis in X is then defined by eit
-
-
Fki-t{j
ki- l '\'
'\'
L.
i -
L. (lji(t+
1)
Fj-t{j
ieM,
j'
t=t jEM(t)
With respect to this basis, (F, G) take the form i,j, eM,
where (F .J.
~[~
1 0 0 1
-:,J
- (liil
0
(F #)ji
= [
(F #)ji
= [
(F #)ji
e Rkjxk,.
G# =
-
- (ljikj
(ljil
0
0 - (l jil
«G#)ji)'
-(ljikiJ
jeM,
...
oJ
kj > k i ,
i = 1,2, ... , m.
o o (G#L=
o
ieM,
m 1
(G.)"
~
j # i,
ieM,
ieM,
t
= 1,2, ... , k i .
108
5
STRUCTURE THEOREMS AND CANONICAL FORMS
If we renumber the columns of G so that k i ~ k j + l ' then F # and G # take the forms 0 0
x
1
0
x
0 1
0 0
x
0
x
F# =
0 x
x
x
0
0
x
x
0 0
1
x
0 x
) k, 'Ow,
.. , 0 1
X
0 0
... x
x
.,.
I
k,'Ow,
~
k 1 - k2 columns
0
0 1
0
0
x
x
0 x
... 0 .. ....... ... 0 ... x
0
0
0
...
0
...
0
...
0
0 0 1
)
0 x
0
...
0
x
...
x
k 2 rows
~
m-lo columns
Theorem 5.9 Relative to the feedback group fF, the invariants of the system ~ = (F, G, H) consist of the set of integers k l' k 2 ,
••• .k, and the elements hij ofR. Conversely, given any set ofm nonnegative integers k, such that LI= 1 k, = n, and a set of np real numbers hij' a system ~ is determined by transformations from fF, i.e., {kJ and {hij} constitute a complete, independent set of invariants
for~.
5.7
109
THE FEEDBACK GROUP AND INVARIANT THEORY
EXAMPLE
We apply Theorem 5.9 to the system
F=
1 0 2] [001
H = (0
230,
I
0).
First we verify that I: is completely controllable. Computing the controllability matrix «6, we have
I 2 I 4]
o o
2 084
10101 which has rank 3. Following the prescriptions of the theorem, we write the sequence of vectors
{g,~G), F'g,
g,
~ G),
~ G}
r«.
Fg,
~ (D,
Fg,
~ G}
~ (D}
Clearly, the first three form a linearly independent set in R 3 . The Young's diagram is 91
g2
I
x
x
f
X
F2 k l = 2, k 2 = 1. Thus we see that the Kronecker invariants for this case are k l = 2, k 2 = 1. The set M = {I, 2}. We next compute the basis (5.7). We obtain
Ito
5
STRUCTURE THEOREMS AND CANONICAL FORMS
The only missing quantity is the constant dependency relations 2
p2 9 1
-391
This implies that
+ 4F91
IX I J I
= -
= -
I I
I
IXIII'
This is obtained from the
IXljk P k9 j ,
j=1 k=O
=
-{IX I
Jo9 1 + IXJllF91
+ lX J2092}'
4. Thus the new basis is
If we assume that the matrices P, G, H are originally given in terms of the standard basis in R 3, i.e., the basis
and if we express the standard basis in terms of the new basis ell' it is easily verified that the matrix of this transformation is given by
T
e J 2, e2J'
~ [! ! H
Using the basis change T, we finally obtain
kl
f
~
~
TFr'
~ [--:-:--+-: R
t.
= HT-
I
= (2
The feedback law
L=[~
-~
0
=:]
0).
eliminates the second row of F which enables us to see that a complete set of invariants for the system are the integers k I = 2, k 2 = 1, together with the elements of the matrix fJ = (2 0 0). Note that in this example it was not necessary to use any Type (II) transformation because G was of full rank.
III
MISCELLANEOUS EXERCISES
REMARKS (I) The importance of the control invariants k l , ... , k. is that they give the sizes of the smallest cyclic blocks into which F may be decomposed by linear" feedback" transformations. This is the only structural obstruction to arbitrarily altering F by utilization of feedback. (2) The invariants k I' ... , k. are identical to the classical Kronecker indices associated with the matrix "pencil" [zI - FIG]. In fact, Kronecker's definition for the equivalence of two pencils is
P(z) '" Q(z) ¢ > P(z) = AQ(z)C,
for nonsingular constant A and C. It is then easy to see that [zI - FIG]
if and only if C has the form C
A-I
OJ
= [ LA-I B'
= A[zI - FIG]C L arbitrary,
det B ¥- O.
Thus Kronecker's equivalence relation applied to the pencil [zI - FIG] is identical to the equivalence relation induced by the feedback group !F on pairs (F, G). EXERCISES
1. Assume that m divides n and that each k, = nlm, i = i, ... , s. Determine the subgroup of!F which leaves the Kronecker canonical forms for F and G invariant. What happens if all k, are not equal? 2. Complete the proof of Theorem 5.9 by showing that the parameter set S = {k j , !iij} constitutes an independent and complete set of invariants for the system I: = (F, G, H), i.e., for any given set S, there is some system I: whose parameter set equals S (independence) and if two systems I:I and I:2 have the same parameter set S, there exists a transformation !/ E !F such that I:I = !/I: 2 (completeness). MISCELLANEOUS EXERCISES
1. Suppose that the pair (F, G) is not completely controllable and that G ¥- O. Show that there exists a basis change in X such that F and G are transformed into
with the pair (F ll' G I) being completely controllable. Show that the rank of F II equals the rank of rc(F, G).
112
5
STRUCTURE THEOREMS AND CANONICAL FORMS
2. Define the single-input/single-output system ~ = (F, g, h) to be nondegenerate if it is both completely controllable and completely observable. Show that the following statements are equivalent: (a) ~ is nondegenerate. = h(A.I - F)-lg) is irreducible and the (b) The transfer function of~(Z(A) degree of the denominator equals n, the dimension of ~. (c) If any other system t of the same dimension has the same transfer function, then there exists a nonsingular T such that the two systems are related as
9= 3.
Let the system
~
be in Lur'e-Lefschetz-Letov canonical form, i.e.,
F~ [F' Fj =
Tg,
n J"
F2
G=
~2
G q
r'
F22
F,,,J....'
Gj =
nXm
G [G" ] j2
G}S(j) ni x m'
A~ I
1
Fij =
Aj 1
[g'' 1
Aj 1
Prove that vectors
~
g2u G.. = : . u g;ii nij x m
Aj
nij x nij
is controllable if and only if the set of s(i) m-dimensional row
form a linearly independent set, i = 1, 2, ... ,q. (Note that this set is composed of all the first rows of the matrices Gij associated with a given submatrix F j .) 4. Show that a single-input system ~ = (F, g, -) is completely controllable if and only if the transfer matrix Z(A) = (AI - F)-lg is irreducible, i.e., the numerator and denominator have no common factors.
113
MISCELLANEOUS EXERCISES
U sing this result, check the system
F
=
-2J g=G)
[~
-3 '
for complete controllability. 5. (a) Let F, G be fixed and denote the range of G by '§, i.e., '§ = {x E R": x = Gy for some y E R m } . Further, let (FI'§) = '§ + F'§ + '" + F"-l,§ denote the controllable subspace of X for the pair(F, G). Ifdim "§ = m, show that there exist subspaces {J!;} c X such that (i) (ii)
dim("§ n J!;) = 1, i = 1, (Fj"§) = VI
E9 V2 E9
, m;
E9 Vm •
(b) If (F, G) is a controllable pair, what is the relation between the list of integers {dim VI' dim V2 , ••• , dim Vm } and the Kronecker indices k I' k 2 , · · · , km ? 6. In the theory of representations ofthe general linear group GL(R m ) , it can be shown that each set of m nonnegative integers k, ;C 0 satisfying I~ I k, = n corresponds to a certain representation of GL(R m ) . What is the connection between this fact and the control canonical form for a multi-input system? 7. Prove the following structure theorem. Let (F, G)be completely reachable, m = rank G, and k, ;C k 2 ;C ... ;C k m the ordered control invariants of (F, G). Let t/J I, ... , t/J q be arbitrary monic polynomials such that (a) t/Jdt/Ji-I,i=l, ... .a t.s s »: (b) degt/JI;C kl,degt/JI + degt/J2;C k 1 r
+ k2,etc.
Prove that there exists a feedback control law L such that F - GL has the invariant factors t/Jl"'" t/Jq. Conversely, show that the invariant factors of F - GL always satisfy (a) and (b). 8. Let k be an algebraically closed field and assume that the system I: = (F, G, -) and an arbitrary control law L are defined over k. Define a control law J to be purelyfeedforward ifand only ifXF = XF-GJ, while a control law K is purely feedback if and only if
Prove that L may be written as L
= J + K,
where J and K are unique modulo a neutral law M.
114
5
STRUCTURE THEOREMS AND CANONICAL FORMS
(b) Show, by a counterexample, that this result fails if k is not algebraically closed. (Recall: A field k is algebraically closed if any polynomial with coefficientsin k also has all of its roots in k.Thus the field of complex numbers is algebraically closed, but not the field of reals.) NOTES AND REFERENCES
Section 5.1 A systematic exposition of the algebraic point ofview in modern physics is given by Zaitsev, G .. "Algebraic Problems of Mathematical and Theoretical Physics." Nauka, Moscow, 1974.
A similar point of view (in English) is presented by Hermann. R., "Interdisciplinary Mathematics," Vols. I-IX. Math. Sci. Press, Brookline, Massachusetts. 1972-1977.
The interested reader is especially urged to consult Volumes III, VIII, and IX for material specificallydevoted to algebraic and geometric aspects of system theory. For a more classical point of view on system problems, but with deep algebraic overtones, see Popov. V.. "Hyperstability of Control Systems," Springer-Verlag, Berlin and New York, 1973.
The material of this chapter employs only the elementary tools of linear algebra and matrix theory to develop the theory 'of canonical structures for linear systems. For a much deeper and more thorough treatment utilizing the full machinery of abstract algebra, the reader should consult Chapter Nine and the references contained therein. Section 5.3 The Lur'e-Lefschetz-Letov canonical form is extensively used in the book by Popov cited above. For the single-input case, see also Lefschetz, S., " Stability of Nonlinear Control Systems." Academic Press. New York. 1965.
The control canonical form for single-input systems is treated in detail by Kalman, R., Falb, P., and Arbib, M., " Topics in Mathematical System Theory." McGraw-Hili, New York, 1969.
The multi-input case may be found in Kalman, R., Kronecker invariants and feedback, in "Ordinary Differential Equations" (L. Weiss, ed.), pp. 459-471. Academic Press, New York, 1972.
NOTES AND REFERENCES
115
The explicit use of invariant-theoretic arguments to calculate the control invariants under state variable coordinate changes is treated by Popov, V., Invariant description of linear, time-invariant controllable systems, SIAM J. Control 10, 252-264 (1972).
See also the work by Weinert, H., Complete sets of invariants for multivariable linear systems, 7th Princeton Con! Informat. Theory and Systems, Princeton. New Jersey. 1973.
A good treatment of the relationship between canonical forms, invariants and input/output models is given in Guidorzi, R., Complete sets of independent invariants and canonical forms for linear systems identification. Faculty of Engineering Report, U. of Bologna, January 1978.
Section 5.6 The results of this section follow Sidhu, G. S., "A Note on the Bezoutiant Matrix and the Controllability and Observability of Linear Systems" (unpublished manuscript, 1973).
A classical treatment of the Bezoutiant matrix in the context of polynomial algebras is given by Becher, M., "Introduction to Higher Algebra." Dover, New York, 1964.
An efficient constructive algorithm for transforming one observer canonical form to another is detailed in Caroli, M. and Guidorzi, R., Algebraical links' between block-companion structures: A new efficient algorithm, Ricerche di Auto, 4,1-17 (1973).
Section 5.7 A fascinating sociological account of the rise and fall of invariant theory as a fashionable mathematical activity is found in Fisher, C. S., The death of a mathematical theory, Arch. History Exact Sci. 3, 137-159 (1966). Fisher, C. S., The last invariant theorists, Arch. European Sociology 8,216--244 (1967).
The feedback group and its invariant-theoretic implications for system problems is extensively pursued by Wonham, W., and Morse, A., Feedback invariants of linear multivariable systems, Proc. IFAC Symp. Multioariable Systems, Dusseldorf, October 1971. Wang, S., and Davison, E., Canonical forms of linear multivariable systems, SIAM J. Control 14, 236-250 (1976). Hazewinkel, M., and Kalman, R., "Moduli and Canonical Forms for Linear Dynamical Systems," Rep. 7504/M, Erasmus Univ., Rotterdam, April 1974. Hazewinkel M., On the (internal) symmetry groups of linear dynamical systems, Vieweg Tracts in Pure and Applied Physics, 4, 362-404 (1980).
116
5 STRUCTURE THEOREMS AND CANONICAL FORMS
For a general treatment of the overall concept of genericity and its use for linear system theory, see Tchon, K., On generic properties of linear systems: An overview, Kybemetika, 19, 467-474 (1983).
For results concerning the determination of the Kronecker invariants directly from input/output data, rather than from a system given, as in the text, in internal form, the paper by Guidorzi, R., Canonical structure in the identification of multivariable systems. Automatica>J. IFAC II, 361-374 (1975).
is recommended.
CHAPTER
6
Realization Theory
6.1 INTRODUCTION
Of all the problems addressed by the techniques of mathematical system theory, it is only a slight exaggeration to state that the realization problem is the sinequa non of the subject. As discussed in Chapter 1,the realization issue is none other than the problem of constructing a mathematical model of a physical process from experimental data, a problem faced by scientists from the age of antiquity. The objective of the modern system theorist is to remove as much subjectivity as possible from the model-building process. In earlier chapters we saw examples in which experimenters constructed fanciful models of physical phenomena containing components totally unjustified by the data on which the models were built. The guiding principle in realization theory is to apply Occam's razor ruthlessly: When two proposed models both explain the same experimental evidence, the simpler of the two is to be preferred. Our task will be twofold: to devise techniques for constructing some model from the data and then to isolate those properties which make one model "simpler" than another. As might be expected, our previous developments on controllability and observability will play an essential role in carrying out this program. There are several approaches to the realization problem depending on the form in which the data is given. One path is through the system transfer function. We are given the p x m transfer matrix Z(,1,) and the task of finding a minimal system 1: = (F, G, H) such that Z(,1,) = H(U - F)-lG. 117
118
6
REALIZATION THEORY
Since the transfer-function matrix arises as the Laplace transform of the impulse-response matrix, knowledge of Z(A) is equivalent to saying that we know the impulse response, i.e., that our "experiment" on the unknown system consists essentially of applying a delta-function at each of the m input terminals and observing the output at the p output terminals. It is clear that there may be operational difficulties in obtaining Z(A). For the moment, however, we shall ignore these problems and proceed with the mathematics. A later section shall be devoted to overcoming the practical difficulties associated with this technique. In order to avoid unpleasant technical details, we treat mainly constant linear dynamical systems, referring the reader to the references cited at the end of the chapter for details of the time-varying case. 6.2 ALGEBRAIC EQUIVALENCE AND MINIMAL REALIZABILITY
In general, there is.not a unique solution to the realization problem and different realizations of the input/output behavior have quite distinct characteristics. Thus we must focus our attention on "equivalent" realizations. The most important type of equivalence turns out to be algebraic equivalence. Definition 6.1 The constant system I: = (F, G, H) is (strictly) algebraically equivalent to the system i: = (F, G, ll) if and only if there exists a nonsingular matrix T such that
G=
TG,.
H=
HT- 1,
i.e., if I: and i: differ only by a coordinate change in X. We next define realizability of an impulse response matrix W(t, s). Definition 6.2 A representation I: = (F, G, H) is said to be an (nth order) impulse realization of t
~
s,
if F is of size n x n. If such a system representation exists for some finite n, then W(t, s) is said to be a realizable impulse response. A realization I: = (F, G, H) is said to be a minimal impulse-response realization of W if there exists no nth order realization r = (P, G, H) of W with n < n. Transfer-function matrix realizability is defined similarly. The basic question of whether or not some realization of a given impulse response exists is surprisingly easy to answer, as demonstrated by the next theorem.
6.2
119
ALGEBRAIC EQUIVALENCE AND MINIMAL REALIZABILITY
Theorem 6.1 W(t, s) is a realizable impulse-response matrix has the separableform W(t, s) = qJ(t)r/!(s),
if and only if W
t ;::: s.
PROOF (Necessity) Given an n-dimensional system ~ zero-state representation
=
(F, G, H), the
has the impulse-response matrix W = HeF(I-S)G = (HeFI)(e-FsG),
which is of the stated separable form. (Sufficiency) Given the separable form
W(t, s) = qJ(t)r/!(s), we can immediately construct the realization ~ =
(0, r/!(t), qJ(t)).
(6.1)
Theorem 6.1 resolves the basic realizability question in a rather unsatisfactory way. For example, the realization (6.1) is not "practical" in the sense that the matrices qJ and r/! will, in general, be unbounded and, since F == 0, the realization will never be asymptotically stable. Also, note that even if W(t, s) represents a constant system, the realization (6.1) is time-varying. To illustrate the above points, consider the impulse response l), which is time~ = (0, e', evarying, unstable, and has an unbounded coefficient. A much more useful realization of W(t, s) is given by ~ = (-1, 1, 1). EXAMPLE
W(t, s) = e-leS • Realization (6.1) is then
The basic realizability condition for transfer-function matrices is stated next.
Theorem 6.2 Z(A.) is a realizable transfer-function matrix if and only if it is a strictly proper rational matrix, i.e., the numerator of every element of Z(A.) is of lower degree than the denominator. PROOF
(Necessity)
This is clear from the representation
Z(A.) = H(U - F)-lG and Cramer's rule.
120
6
REALIZATION THEORY
To prove sufficiency, consider the case when Z().) is a scalar
(Sufficiency) function
Z().) = hnAn- 1 + hn_ 1An- 2 + ... + h 2A + hI An + anAn - 1 + ... + a2A + a l
(6.2)
It is easy to see that Z(A) is realized by the system .E = (F, g, h), where 0 0
I 0
0 1
0 -al
0 -a2
0 -a3
o o
o o
F=
g
=
o
h = [hI
hs: . hnl
1 (6.3)
In the matrix case, each element of Z(A) has form (6.2) so that Z(A) can be realized by a collection of pm uncoupled representations of form (6.3). Unfortunately, the realization obtained from the foregoing sufficiency proof is, in general, far from minimal. In order to produce minimal realizations, we shall have to delve deeply into the structure of .E and utilize our previous results on controllability and observability. But first it will be necessary to establish some further results on algebraic equivalence. Define the matrix sequences
C6j = [G
I FG I H HF
These matrices are called the j-controllability and j-observability matrices, respectively, of.E = (F, G, H). Let Gl be the smallest positive integer such that rank C6" + I = rank C6". This value of a is called the controllability index of .E. Similarly, the smallest integer f3 such that rank is called the observability index.
(!)P+ I
= rank (!)p
6.2
ALGEBRAIC EQUIVALENCE AND MINIMAL REALIZABILITY
Several useful properties of the sequences rc j and
0(
(!)j
121
are:
(I) 0(,13:::;; (1 = degree of the minimal polynomial of F. (2) rank ~j = rank ~a, i > 0(, rank (!)j = rank (!)p, i > p. (3) If ~ = (F, G, H) and! = (F, G, H) are algebraically equivalent, then = iX, 13 = P, and i
= 1,2, ....
We may now utilize the sequences ~j and (!)j to give an improved version of Theorems 3.10 and 4.3 which we may then combine to give a detailed description of the structure of ~ under algebraic equivalence. Theorem 6.3 If rank ~a to ~ = (F, G, H) where
and
!q
= (F I"
= q :::;; n, then
~
= (F, G, H) is strictly equivalent
GI , HI) is completely controllable.
PROOF Let TI be a matrix whose columns form a basis for the column space of rca and let Tz be any n x (n - q) matrix whose columns along with those of T1 form a basis in R". Then the matrix T- I = [Til Tz ] is nonsingular and defines! = (P, G, H). The first q columns of F are given by
for some K I since FTI is a submatrix of rca + I' Similarly,
since G is a submatrix of ~
a'
Let
r& a be the IX-controllability matrix of ~q
=
(F II' GI' HI)' Then the forms of F and G show that
Hence, rank ~ the proof.
a
= q so that ~q = (F 11' Gl'
-)
is controllable. This completes
122
6
REALIZATION THEORY
The observability version of the above theorem follows. Theorem 6.4 If rank (fJp to ! = (P, G, H) where
F= [~:
F~J,
Also, ~q = (FIt> G1 , PROOF
= q :=:; n, then G=
= (F, G, H) is strictly equivalent
~
[g:J
H = [HI
OJ.
Ht> is completely observable.
Dualize the proof of Theorem 6.3.
Theorems 6.3 and 6.4 together enable us to prove the following canonical decomposition theorem for linear systems. Theorem 6.5 If rank (fJp~f/. = q :=:; n, then system 1: = (P, G, H), where
and ~q
=
o.; G
1,
~
= (F, G, H)
is equivalent to a
HI) is controllable and observable.
PROOF Applying Theorem 6.3 to ~ = (F, G, H) followed by Theorem 6.4 applied to the resulting controllable subsystem shows that there exists a T such that ~ is algebraically equivalent to 1:, where 1: has the indicated structure and ~qis controllable and observable. We must show that ij = q. To demonstrate that ij = q, partition ~f/. and ~p as
These matrices are partitioned conformally with the forms of F, G, H. Then ~~1 and mp1 are the IX-controllability and fJ-observability matrices of ~q = (P l l , G1, HI), respectively, and
=
(fJp~f/.
by Property 3 of ~ j and Thus
(fJ j
mp?Cf/.
=
since lJ P2 = ~
rank mpl~f/.l
f/.3
= O.
= rank (fJp~f/.
By controllability and observability of ~q, rank ~Pl~f/.l Therefore q = ij.
(!jP1?Cf/. l ,
= q.
however, we have = ij.
6.2
123
ALGEBRAIC EQUIVALENCE AND MINIMAL REALIZABILITY
Using the foregoing results, we can now deal with the question of minimal realizations. At first glance, one may ask why it is so important to produce a minimal system realization. After all, should we not be satisfied if a model of the process under study can be produced which explains all the experimental data? The answer, of course, is no! Returning to Occam's razor, we demand the simplest possible explanation. Why do we present minimality of the state space as a plausible definition of what constitutes the "simplest" explanation? The reason is Theorem 6.6 (below) which asserts that minimality is equivalent to a completely controllable and observable model. In other words, if we present a nonminimal realization as our model, then we are including features in the model that are not justified by the experimental evidence: an uncontrollable part cannot be influenced by any external input, while inclusion of an unobservable part means that irreducible internal uncertainties about the system behavior are being built into the model. Both features are clearly undesirable. Define the matrix Ki,{t) as
Kjj{t) = (fJjeFtCC j , i,j = 1,2, .... Using Kjj{t), we establish the connection between controllability, observability, and minimality. Theorem 6.6 (F, G, H).
The following statements are equivalent for constant
~ =
(1) ~n = (F, G, H) is controllable and observable. (2) rank (fJpCC a = n. (3) ~n is minimal. PROOF (2) => (1) Obvious. (1) => (2) Obvious. (3) => (2) Follows from Theorem 6.2 since if rank (f)pCC a < n, we can find a lower-order representation having the same impulse-response matrix. (2) => (3) Suppose that (2) holds and ~n is not minimal. Then there exists ~ll' with ii < n, such that ~ii has the same impulse response matrix as ~n' Since Kij{O) = (fJjCC j , we have'
Kpa(O) = ~p7jJa' But ~p has only ii columns. Thus rank (fJpCC a = rank ~p7jJa dicting (2). Corollary 6.1
~ ii
< n, contra-
Let y and (j be the first integers such that rank (f)y CC/j = rank (fJpCC a = q
~
n.
Then the minimal realizations of~n = (F, G, H) have degree q, controllability index (j, and observability index y.
6
124
REALIZATION THEORY
EXERCISES
1. Prove Properties (1)-(3) of the sequences ~i and (!]j' 2. (a) Show that the decomposition theorem 6.5 can be given in the following form: For any linear system :E = (F, G, H) (including timevarying :E) there exists a coordinate system T(t) in X such that :E is algebraically equivalent to a system t = (ft, c, R), where
(b) What does the above structure say about the internal controllability/ observability structure of E? (Hint: Draw a diagram of the input/output structure of t.) 3. Prove that all time-invariant minimal realizations of a given impulseresponse matrix can be generated from one such realization by means of constant coordinate changes in X. Is this also true for nonminimal realizations? 4. Show that the matrix (!]/l + 1~ ~: (a) is an invariant for the class of all realizations of a particular impulseresponse matrix; (b) is the only invariant necessary to determine if two representations have the same impulse response. . 6.3 CONSTRUCTION OF REALIZATIONS
We have shown that once any realization of an impulse-response matrix is given, the problem of determining all minimal realizations is effectively solved since we need only construct the complete invariant (!] p + 1~ a; and utilize Theorem 6.5, Theorem 6.6, and its corollary to reduce a nonminimal realization to one which is minimal. But, the problem of explicitly constructing some realization remains. In this section, we shall show that the suffices to calculate a minimal realization explicitly. matrix (!]/l+1~a; Our approach to the construction of a realization will be through the Hankel matrix of an infinite sequence of matrices. Definition 6.3 Let J denote a sequence of real p x m matrices J j , i = 0, 1, .... A system :E = (F, G, H) is said to be a realization of J if and only if i = 0, I, ....
(6.4)
6.3
125
CONSTRUCTION OF REALIZATIONS
To see the connection between the sequence of matrices {J;} and the realization problem in both continuous and discrete time for a given transfer function (or equivalently, impulse response), let Z(A.) denote the transfer function of a continuous-time system ~n = (F, G, H). We then have the expansion about A. = 00: 00
Z(A.) =
LJ
i= 1
(6.5)
i - 1 A.".
where J, is given by (6.4). If W(t - s) denotes the impulse response for the same system, then W(t - s)
=
L Ji(t 00
- s)i/i!.
i=O
In a similar manner, if ~n = (F, G, H) characterizes a discrete-time system, its impulse-response matrix Hi is given by while its z-transform transfer function has form (6.5). These remarks show that any procedure suitable for realizing the sequence {J;} will serve equally well,regardless ofthe specificform in which the input/output data is presented. Clearly, the first question is to determine when a given infinite series is realizable. To establish this basic point, we introduce the block Hankel matrices for J as
:Yl'ij
=
r'
J1 J2 ·
~1
Ji-
1
Jj
J)-> J.
J'+~_'
j .
The role of the Hankel matrices in realization theory follows from the fact that (6.6) when J is generated by a system ~ = (F, G, H) having observability matrix and controllability matrix ~j' In terms of the Hankel matrices, we have the next theorem.
(!}i
only
if there
all j = 1,2,... .
(6.7)
Theorem 6.7 (a) An infinite sequence J is realizable exist nonnegative integers P, tX, and n such that
rank :Yl' P« = rank :Yl' P+ 1,«+ j = n (b)
of
J.
for
if and
If J is realizable, then n is the dimension of the minimal realization
6
126
REALIZATION THEORY
(c) If'/ is realizable and {3, ~ are the first integers for which (6.7) is true, then {3 is the observability index and ~ is the controllability index of any minimal realization of ,/. PROOF (a) (Necessity) Necessity follows from (6.6) and the properties of the matrices (!)i and t:{fj' (Sufficiency) We prove sufficiency by constructing an explicit minimal realization of ,/. Let i'ij be the (i,j)th block of the Hankel matrix Jf~.p. Clearly, from the form of Jf~,p, we have
i'i+p,j = i'i,j+m,
(6.8)
i,j = 1,2, ....
Thus (6.7) implies that rank Jfp+i,~+j
= n,
i,j
= 1,2, ....
(6.9)
since the ({3 + i)th block of rows in Jf p + i. j is contained in the ({3 + i-I )st block ofrows in Jf p + i, j + I by (6.8). Let A~ denote the submatrix formed from the first n independent rows of Jfp~ and let A~* be the submatrix of Jfp+I,~ positioned p rows below A~ (i.e., ifthe ith row of A~ is the jth row of Jf p + 1, ~, then the ith row of A~ * is the U + p)th row of Jfp+1,~). The following matrices are then uniquely defined by Jf P+I,~: A
A* Al
A2
the nonsingular n x n matrix formed from the first n independent columns of A~, the n x n matrix occupying the same column positions in A~ * as A does in A~, the p x n matrix occupying the same column positions in Jf h as A does in A~, the n x m matrix occupying the first m columns of A~.
If we define F as F
= A *A - I, then it follows from
(6.9) that
j = 1,2, ... ,
where Aj and A/ are extensions (or restrictions) of Also
(6.10) A~
and
A~*
in
Jfp+I,j'
(6.11)
since the submatrix positioned m columns to the right of a given submatrix in Jfij is the same as the submatrix positioned p rows below it by (6.8). Thus, by (6.10),
6.4
127
MINIMAL REALIZATION ALGORITHM
Next, define G that
=
A 2 • It follows by repeated application of (6.10)and (6.11) j
j ~
= 1,2, ....
Define H = A 1 A - 1, use the fact that Aj spans the row space of Yepj' IX, and employ (6.9) to see that
But, Yeij = [JoIJ 11 .. ·IJj - 1 ] . Hence, we must have J, = HFiG, i = 0, 1, .... Thus we have proved that the triple
defined from the submatrices of Yep + 1./Z' realizes the infinite sequence" if (6.7) holds. Furthermore, it follows immediately that this realization is minimal and has controllability index IX and observability index p.
6.4 MINIMAL REALIZAnON ALGORITHM
Theorem 6.7 provides the following algorithm for constructing a minimal realization of the sequence": Determine integers p, IX such that rank YeP./Z = YeP+ 1./Z = n for all IX. (2) Form the matrix A/Z from the first n independent rows of Yep, /Z' (3) Form the matrix A/Z * from Yep + 1,/Z as the matrix which is positioned p rows below A/Z. (4) Form the four matrices A, A*, A 1 , A 2 as (1)
A
A* A1 A2
nonsingular n x n matrix formed from the first n independent columns of A/Z, the n x n matrix occupying the same column positions in A/Z * as A occupies in A/Z, the p x n matrix occupying the same column positions in YelIZ as A does in A/Z, the n x m matrix occupying the first m columns of A/Z.
(5) Form the minimal realization of" as ~ =
(A*A- 1 , A 2 , A 1A- 1 ) ,
128
6
REALIZAnON THEORY
6.5 EXAMPLES
(1)
Fibonacci Sequence
Let / = (1, 1,2, 3, 5, ...). Applying the above algorithm, we see that condition (1) is first satisfied for 0( = P = 1, giving rise to the one-dimensional realization 1: = (1, 1, 1). It is easy to verify, however, that this system realizes only the first two terms of the sequence /. Thus we must include more data in our search for a realization that "explains" the entire sequence /. We form the additional terms in the Hankel sequence
which shows that condition (1) of the algorithm is also satisfied for C! = P = 2, giving a two-dimensional realization. By virtue of the fact that we "secretly" know that / is generated according to the rule
it is not hard to see that regardless of how many additional terms are taken in the Hankel sequence, the rank condition will always be satisfied with some p, C! combination having n = 2. Thus this outside knowledge of the sequence allows us to conclude that the minimal realization for / is two-dimensional. Of course, on the basis of a finite data .string it will never be possible to guarantee that the infinite sequence / has a finite-dimensional realization without knowing such a realization in advance. For the Fibonacci sequence above, however, it is reasonable to conjecture that n = 2 even without knowledge of the generating formula since it is easily verified that for all 0(, p > 2, the first two columns of Yf ap always generate the remaining columns. Following steps (2)-(4) in the realization algorithm, we form the matrices A z , A z*, A, A*, A l , Az as
=
[~
A=
[~
Az
Al
=
[1
~J. ~J. 1],
A z*
=
A* =
AZ=[~J.
G~J. [~
~J.
6.5
129
EXAMPLES
According to step (5), the minimal realization for / is
F=[~
G=eJ
~J
H=[I
OJ.
This example illustrates the critical point that the success of the foregoing algorithm hinges on the assumption: / has a finite-dimensional realization. This is equivalent to saying that there exists some integer n < co, such that the rank of the sequence of Hankel matrices is bounded above by n, irrespective of the number of elements in .Yl'lJp' Of course, if / were only a sequence of numbers obtained from some experiment, then there is no a priori justification for the finite realization assumption and, in general, the best we can hope for is to realize some finite piece of the data string ,I. A more thorough discussion of this partial-realization problem is given in a later section. (2) The Natural Numbers Let / = (1,2,3,4, ...). Upon forming the relevant Hankel matrices, we see that the rank condition will be satisfied for a. f3 n 2 and, furthermore, all the Hankel matrices .Yl'ij have rank 2 for i, j > 2 (verification ?). Thus / has a two-dimensional realization. Carrying out the remaining steps of the algorithm, we find
===
=G ~) A* =G ~) A =G)'
=G ~). A=G ~). At = 2),
A2 *
A2
(I
A-
l
2
=
(-3 2) 2
-I'
Thus the realization of,l is G=
G)'
H = (l
0).
It is interesting to observe that this is not the only possible realization of ,I. For instance, the transfer function corresponding to / is
130
6
which suggests the alternate realization
~
F=
[0 IJ
-12'
G=
G)'
REALIZA nON THEORY
fJ = (0 I).
What is unique about the realization is its dimension, in this case two. However, the fact that the two realizations are related by the nonsingular transformation
-1/2t l1
o
J '
as
G=
F=
TG,
TFT- 1
suggests that a change of basis in the state space will remove all non uniqueness in the realization. We examine this point later. (3)
Multi-input-Multioutput Realization
In order to dispel any notion that a multi-input/multioutput sequence can be minimally realized by realizing each of its components separately, we consider the sequence
/ =
{(l 1), (1 2), (l 3), (2 4), (l 5), (3 6), ...}
which is made up of the two scalar sequences /
=
{I, 1, 1,2,1,3, ...}
and
/2
=
{I, 2, 3,4,5,6, ...}.
We have already seen that a minimal realization of /2 has dimension 2. It is also easy to see that /1 has a minimal realization of dimension 3; in fact, the transfer function associated with / I is A. 2+2A.+1 Z(A.) = A. 3 + A. 2 - A. - 2 . We can, of course, obtain an upper bound for the dimension of the realization of / by simply adding up the dimensions of each of its component sequences /1 and /2 which shows that a minimal realization of / has dimension no larger than 5. This upper bound may, however, be too large since by accepting nonminimal realizations of some scalar components, we may be able to reduce the dimension of the matrix sequence. Applying the above realization algorithm, it is not too difficult to see that a minimal realization of / has dimension 4. This is confirmed by examination of the irreducible transfer function
A.3+A.2_A. A. ] Z(A.) = [ (A. _ 1)2(A. + 1)2' (A. _ 1)2 .
6.6
131
REALIZATION OF TRANSFER FUNCTIONS EXERCISES
1. Complete the construction of a realization of the sequence in Example (3). 2. Show that the sequence of prime numbers f = {2, 3, 5, 7, 11, ...} has no finite-dimensional realization. 3. In purely algebraic terms, show that a 1:: is a minimal realization of the input/output map f: n ~ r if and only if there exists a finite-dimensional space X and maps g and h such that (i) g is onto, his 1-1, and (iii) the diagram (ii)
n
f
--->
r
x is commutative.
6.6 REALIZATION OF TRANSFER FUNCTIONS
The realization procedure outlined in the previous section was based purely on input/output data, as opposed to complete knowledge of the input/output map f As a result, certain unpleasant difficulties appeared in attempting to decide operationally whether: (i) the data admits a finite-dimensional realization and (ii) if so, what is the dimension of a minimal-realization. In this section we shall explore the question of how much more information on these questions is provided by the knowledge of the map f, rather than having just a finite string of data generated by f As one might expect, since knowing f is equivalent to knowing an infinite string of data, a completely satisfactory solution to the realization problem will be obtained in this case. The more difficult question as to exactly what information a finite data string provides will be treated in Section 6.8. For the present purposes, let us assume that the transfer matrix Z(l) is a strictly proper rational matrix, i.e., the degree of the numerator of each element is strictly less than the degree of the denominator. As pointed out in Theorem 6.2, this is a necessary and sufficient condition for Z(l) to be realizable by some system 1:: = (F, G, H). Such a matrix can always be written in the form &'(l)/X(l), where &'(1) is a polynomial matrix and X(l) a monic polynomial that has no factors in common with &'(1).
6
132
REALIZATION THEORY
For our main result, we need the following easy fact.
Lemma 6.1 If Z(.A.) is a strictly proper rational matrix, then X(.A.) is the minimal polynomial associated with the minimal realization of Z(.A.), i.e., if ~ = (F, G, H) is the minimal realization of Z(.A.), then X(.A.) is the minimal polynomialofF. PROOF
The proof is left as an exercise.
From Lemma 6.1 plus the fact that rank (9 + i = rank (9 and rank ~ a + 1 = rank ~(f for all i, we see that if a is the degree of the minimal polynomial of F, then the Hankel test may be terminated after a steps. This leads to the following basic result concerning realization of transfer matrices. (f
(f
Theorem 6.8 Let Z(.A.) be a strictly properrational matrix and let a = deg X(.A.). Let Jeij be the Hankel matrix associated with the expansion of Z(.A.) as in Eq. (6.5). Then the first integers f3 and IX such that rank Je Pa. = rank Je oo satisfy the realizability condition (6.7). EXERCISES
1. Compute the minimal realizations associated with the transfer matrices (a)
2.
1
Z(.A.) = .A.2 _ 1 .A.3
[2(.A. - 1)
+
.A. - 1
.A. 2 - .A.
(b)
Z(.A.) = [ (.A. _ 1)2(.A. + 1)2
(c)
Z(.A.) =
[
~ + .A.~ ~ o
+ }2 0
.A.
IJ
~
(.A.
~
il
1)2}
Determine a minimal realization for the transfer matrix 1 .A.
+1
_.A. 2 + 1 6.7 UNIQUENESS OF MINIMAL REALIZATIONS
So far we have skirted the question as to whether the realizations produced by Theorems 6.7 or 6.8 are, in some sense, unique. It is clear by Example (2) that strict uniqueness is too much to hope for. It is also highly suggestive
6.8
133
PARTIAL REALIZATIONS
that, as pointed out in Exercise (3) of Section 6.2, the impulse-response matrix (or equivalentally, the transfer matrix) is invariant under coordinate changes in the state space X. The basic result linking these observations is our next theorem. Theorem 6.9 Let,l be a realizable sequence of matrices and let the matrices A, A*, AI' A 2 be as defined in Theorem 6.7. Then any minimal realization of ,I has the form L =
(..4*..4- 1 , ..42 , .41 ..4- 1 ),
where..4 = T A,..4* = T A*,..4 2 = T A 2 with T being an arbitrary nonsinqular n x n matrix, i.e., minimal realizations are unique up to a coordinate change in the state space. PROOF The proof follows directly from Exercise (3) of Section 6.2, plus Theorem 6.7. EXERCISE
1. Use Theorem 6.9 to characterize all possible realizations of the Fibonacci sequence. 6.8 PARTIAL REALIZATIONS
From a practical point of view, the foregoing realization results are somewhat unsatisfactory as they require total knowledge of the input/output map f or, what is equivalent, an infinite string of data for their implementation. In most situations, the data is obtained from measurements made during a finite number of time periods and the objective is to utilize the data to form a model. Dictates of time, money, and manpower usually preclude use of the idealistic realization procedures presented thus far. In this section we shall give a recursive algorithm for solution of the finitedata problem. The procedure to be presented possesses the following important features: (a) Given a finite string of data {J 1, J 2' ... , J N} = ,IN' the algorithm produces a system LN which minimally realizes ,IN' (b) If an additional piece of information J N+ 1 is added to ,IN, then the algorithm produces a new minimal realization LN + 1 for the augmented string ,IN + 1 with the property that LN appears as a subsystem of LN+ 10 i.e., the matrices in LN appear as submatrices of the corresponding ones in L N+ iProperty (b) of the algorithm is quite worthy of note since it enables us to calculate only a few new elements when the sequence ,IN is extended. As a result, the algorithm will be well suited for on-line modeling of a process for
134
6
REALIZATION THEORY
which the sequence J N is generated by means of measurements produced by, for example, the output of a machine, a chemical process, a biological experiment, and so forth. For ease of notation, we consider only the single-input/single-output case, although there is little problem in extending the results to the general situation. We need a few preliminary results. Define the data sequence Y1 = (J l' J 2' ...) and let Yi = uL- 1y 1, where Ul is the left shift operator, i.e.. Ul Y1 = (J 2, J 3, ...). If e denotes the vector (... , 0, 1,0, ...) having a "1" at the zeroth position, then Y1 = f(e) and the set {uLie} spans the input space n, while the set {y;} spans the image off in the output space r. (Here, of course, f is the input/output map to be realized.) We use the numbers in Y1 to form the Hankel matrix
J1 J2
J2 J3
Jm
Jm+ 1
I I
n
n+ 1
Jf=
J m+n-
1
and denote the submatrix consisting of the first m rows and n columns by Jf mn' (N ote: The slight change in indexing Jf from the previous section has been introduced to conform to standard results on this problem.) Since the order of the system equals the number of linearly independent elements in the set {y;}, we have the following useful fact.
Lemma6.2 If dim{Yt>Y2,"'} = n, then the elements Y1,Y2, ... ,Yn are the linearly independent vectors in {y;}. Furthermore, Jf nn is nonsingular and Jf nm has rank n for all m ~ n. PROOF (Ul K
Suppose
zero, i.e.,
°
YK + 1
+ I.f= 1 CiUL- l)Yl
=
YK+2
is linearly dependent on Y 10 ... , YK, i.e., for some {c.}, Then UL of this expression is also
+ CKYK+l + ... + CIY2
= 0,
implying YK+2 is also linearly dependent on Ylo"" YK' Thus the first n must be linearly independent. Consider the last row of Jf n+ l• n• Since Yn+l is linearly dependent on Yl,"" Yn, the rank of Jf n+ l,n is n. By symmetry, the same is true for Jfn,n+ iThus the ranks of Jf nm for m ~ n are no greater than n. But, since they can be no less than n they must all equal n. The realization algorithm is based on a factorization of the type
m
~
n,
rank
Jf nm ~
n - 1,
6.8
135
PARTIAL REALIZATIONS
where P nn is lower triangular with Is on the diagonal, i.e., ... ----- ... -
Ji
: )i+l
.:,: J ..: ·,, .!J . ·, J ·: :
'--ji-~-l-"-i
i+ 2
: :
i+m - I : ,---------, i +m
:
j +m :J -oo ..
1.:
~ r~:. lPnl
Pn2
Pn,n-I
(6.12) The factors are not unique, so by setting certain elements in Qnm equal to zero, we shalI be able to calculate the Pijs recursively. Moreover, an addition of rows and columns to .JItnm will not change the numbers already calculated. The factorization algorithm has the folIowing steps:
l.
Set q Ii = J i for all i. If n = I, we are finished and P11 = (l). Ifn > I, assume we have at the ith step calculated all the PjkS and qjkS, j = 1, ... ,i - 1. Let s(j) be the smallest integer such that qj.o(j) =1= 0, j < n. Such an s(j) exists by virtue of the rank condition on J('nm' Set qk, oW = for k > j. Equation (6.12) then leads to a set of i - I equations, one for each column s(j), j = 1, ... , i-I. Because of the previous conditions, the unknowns Pil" .. , Pi, i - I can be recursively determined one by one from these equations. The submatrix P ii , together with (6.12), determines the remaining elements of the ith row of Q"m which completes the cycle. 2.
°
As an example, consider
J(' 45
= [:
2 We have sCI)
= l.
l ~ ~ ~] . I
323
= 0, i > 1. Then P21,I + I'q21 = I=>P21
Set qjJ
Further, q22 = 0, q23 = I, q24 = -1, q25 i > 2, the first and third columns give
+ P32q23
= J5
l.
= 2, Then, since s(2) = 3, qi3 = 0, 1 => P31 = 1, = 1 => P32 = 0.
P31Qll =
1 'q13
=
136
6
REALIZATION THEORY
Continuing the process, we finally obtain
Jff 45 =
[1
I 1 1 0
J[~
1 -1
2
1
_
[P" P~1
Next we define
*
P
n-l,n-1
P31
-
q11 ]
1
q~1
= [
qn-
-1 2 1 I .
o
Pn,n-1
Hn -
,
0
1]-
P32
Pn2
Gn -
21]
I I 0 1 1 0 0 0
1
= (l
0 .. ·0).
1, 1
The basic theorem for partial realization of a sequence"n can then be stated. Theorem 6.10 Given the data" = (J l' J 2' ...), let Jff n - 1, n - 1 be nonsingular and let m be any integer such that .J'l'nm has rank n - 1. Then the sequence "n+m-1 = (J l' J 2, ... ,1n+m-l) is minimally realized by the system
I: n where H n -
1,
Gn -
1
1
= (F n -
h
Gn -
1,
Hn-
1 ),
are as above and
Fnwhere
P n- 1,n-1 is the P:- 1,n-1 is as above.
1
= P;;':l,n-1 P:-1,n-1'
(n - 1) x (n - 1) principal submatrix of
P nn,
and
PROOF By the factoring algorithm, the last row of Qnm is zero. Hence, by writing the equality between the dashed columns of Jff nm in (6.12), we obtain
for
all i.
Consider the equations x(t
+
1) = F n - 1x(t) + Gn yet) = H n - 1x(t).
1u(t),
(6.13)
With x(O) = 0, u(O) = 1, and u(t) = 0, t > 0, the first equation in (6.13) describes the consecutive states which are just the columns of Qn-l,m in
6.8
137
PARTIAL REALIZATIONS
(6.12), while the second gives y(t) = qlt = J, for t = 1, ... , m. We must now show that (6.13) realizes the remainder of the elements in J,,+m-l' This follows from the special form of F,,_ 1, i.e.,
0 122 1 1
111 121
o o
F"-1 = 1,,-1,1
1"-1,,,-1
Applying (6.13), for t = m + 1, ... , m + n - 1, extends Q"m to (2",,,+m-l' where the last row is extended as a zero row. Multiplying this result by P"", we obtain the extension of £"m to '*'",m+1l-1> but, due to the special form of F,,-I, £".,,+111-1 will have the elements
[J: ... Thus y(t) =
qlt
=
J, for all t
~
n
+m-
1.
Any realization of J"+1I1-1 extends it indefinitely. If such a realization has order k < n - 1, then £"-1."-1 has rank k by Lemma 6.2 which contradicts the assumptions. Hence, (6.13) is minimal. The realization algorithm then takes the following form: (1) Let k be the smallest integer such that J k of; O. Take N = 2k + 1 and form Jf'k+ 1, k+ i- It has rank ~ k. (2) Apply the factoring algorithm and find Pk + 1.k+ 1 and Qk+ l,k+ iIf the last row of Q is nonzero, the rank of Jf'k+ I, k+ 1 is k + 1. Increase N by 2, form £ k + 2. k + 2' and continue the factorization. Repeat this procedure until, say, for N = 2n - 1, the last row of Q"" is zero. By Lemma 6.2 such an n exists if the sequence J admits a finite-order realization. (3) From the formulas for 1 .,, - I' G,,_I, H,,-I, and F,,_I' calculate the partial realization E"-I' (Note that P;;!I"-1 can also be calculated recursively since this matrix is lower triangular) (4) Increase N by 1. Continue the factorization for £".,,-1' If the last row of Q"." + 1 remains zero, increase N by 1, and repeat. If the last row is zero for all m, we have found the total realization. (Of course, this cannot be decided so that the algorithm in practice will never stop. A stopping rule is introduced by setting an upper limit for m.) (5) If for some m (> n) the last element in the last row of Q"m becomes nonzero, then J" +111 _ 1 is not realized by the partial realization E,,_ i- In this case, pick a new point J,,+m and form £,,+ l,m' Continue the factorization,
P:-
138
6
REALIZAnON THEORY
pick a new point, and repeat until either Qn'''' for some smallest n' ::;; m has its last row zero or n' = m and the last row is nonzero. In the first case, go to step (3); in the latter case go to step (2). EXAMPLE Consider the sequence ,I = (I, I, 1,2,3,2,3) coming from the Hankel matrix used as an example of the factorization algorithm. In step (2), n = 2 and in step (3)we obtain I: 1 = «(I), (l), (I». In step (4)an addition of J 4 = 2 makes q23 = 1 :#: O. In step (5) we pick J 5 = 1 and form Jf' 33' Returning to step (2), we take two new points, J 6 = 3, J 7 = 2. This time the last row of Jf'44 is zero and we compute E, following step (3):
1 -I
o
~),
o
0),
-1
which realizes all the numbers given in Jf'45' EXERCISE
1. Extend the foregoing realization algorithm to the case of p outputs and m inputs. 6.9 REDUCED ORDER MODELS AND BALANCED REALIZATIONS
In practice it is often the case that the canonical state-space X associated with a behavior sequence J includes states that are either "hard" to reach or "difficult" to observe. Intuitively, we might expect that such states will tend to playa minor role in the system's behavior and that an approximate model I:*, formed using a state space that excludes these "bad" states, will still capture all of the dominant behavior of the original system. This is the basic idea underlying the notion of reduced-order models. The problem that immediately arises is that a state may be difficult to reach, but easy to observe, or conversely, and so it is not totally straightforward to decide whether to neglect such a state or not. The resolution of this dilemma leads to the concept of a balanced realization. To make the above ideas precise, we first need a measure of the difficulty involved in reaching or observing a given state. Such measures are provided by recalling the reachability and observability Gramians of Chapters 3 and 4. Let I: = (F, G, H) be a reachable and observable system, with F stable, i.e., Re AiF) < 0, i = 1,2, ... , n. The reachability Gramian is defined to be W =
LX) t!tGG'eF't dt.
6.9
REDUCED ORDER MODELS AND BALANCED REALIZATIONS
139
As shown in Exercise 6 of Chapter 3, the quantity (x, W -l~) gives the minimal energy needed to transfer the origin to the state ~; thus, if this number is "large" it means that ~ is difficult to reach relative to a state for which the number is small. In a similar fashion, the observability Gramian
serves to generate the measure (x o, Mx o) giving the observation "energy" in the state X o' Thus, ifthis measure is "large" X o is easy to observe relative to an initial state for which the measure is "small". As we have noted, a state x may have (x, W-lx) large, but (x, Mx) small, or conversely. To deal with these possibilities we introduce the idea of a balanced realization. Definition 6.4 We call a realization ~ = (F, G, H) balanced if W = M, i.e, if the reachability and observability Gramians are equal.
A central result on the existence of balanced realizations is Theorem 6.11 Let ~ = (F, G, H) be any canonical realization of dimension n < 00 with F stable. Then there exists a nonsingular matrix T such that (TFT- \ TG, HT- l) = t is balanced and moreover the Gramians for t satisfy
where
with the Ai being the positive square roots of the characteristic values of the matrix MW formed from the Gramians of any canonical realization. PROOF Let W, M be the Gramians for ~. There exists a Tl E GL(n) such that W = Tl Til' Consider T~ MTl = M i - Clearly, M 1 = Mil> O. Consequently, there exists an orthogonal matrix V such that M 1 = V' A 2 V, with A2 = diag(At, A.~, ... , A;), Ai > O. Take T = A 1/2VT 1 1. Using this T, the new Gramians it: Nt are equal and take the diagonal form asserted in the theorem. Under T, the matrix MW transforms to (T- l)' MWT', which has the same characteristic values as MW In the balanced realization above, the product MW becomes A2 , completing the proof.
Now that we have a procedure for constructing a balanced realization, let us tum to the question of eliminating "difficult" states and generating a
6
140
REALIZAnON THEORY
reduced order model. Let ~ = (F, G, H) be balanced, with W = M = A, and consider the control and observation energies, EC<~)
Eo(x)
== (~, A -1~), == (x, Ax),
(control energy), (observation energy).
If {ej } is the standard basis in W, we define the subspaces
L k = <e1 , e z , , ek), L~ = <ek+ r, en)'
Standard results in linear algebra now assert that Eo(x)
L:dimL~n-k
xe L
.
mf
L:
Eo(x)
max --z = max --z = Ak + l'
inf
IIxll
xeL
x*o
II~II
EcCx).
Ec(~)
Ilxll
II~II
max~
dimL~n-k
xeL'
-1
= mm--Z = Ak + 1 • xeL'
x*o
Thus, by neglecting states in L~ we will be neglecting those states that are simultaneously the hardest to reach and hardest to observe. This suggests that the model reduction which forces X k + 1 = ... = x, = 0 is the natural one to use. Hence, if n-k
k
F= [FF 11 FF12Jnk- 'k Z1
G=
[GG Jnk k ' , 1
z
zz
is balanced with A = diag(A1 , Az,"" An), then we use ~* = (F 11' G 1 , H 1) as a reduced-order model of order k. The value of k for which ~* will be a good approximation to ~ is the first k such that Ak ~ Ak+ 1 ~ O. Under the condition that Ak > Ak+ r- it can be shown that the reduced model ~* will inherit the main properties of ~ (reachable, observable, stable). MISCELLANEOUS EXERCISES
1. (8. L. Ho Algorithm) Assume that a given sequence cI = (11' i z , · ..) has a finite-dimensional realization of dimension n < 00. Show that the following algorithm will realize cI:
(a) Choose N, N' such that rank and define r = min{N, N'}.
JI{'N' N
= n,
141
MISCELLANEOUS EXERCISES
(b) Determine a nonsingular matrix P of size pN' x pN' and a nonsingular matrix M of size mN x mN (m is the number of system inputs, p the number of outputs) such that On.mr-n
]
°
= Epr,nEn,mn
pr- n.mr- n
where
s < q, S = q, s > q.
Here In denotes the 11 x 11 identity matrix, while On1,n> denotes the zero matrix. (c) Form the matrices F, G, Has
F = En.prP[aJt'rrJMEmr,n'
(Ill
x
112)
G = En,prP[Jt'rrJEmr,n'
H = Ep.pr[Jt'rr]MEmr,n' where aJt' denotes the Hankel matrix formed by left-shifting the sequence
f by one position, i.e., the Hankel matrix obtained from the sequence af = {J 2,J 3 ,
. · .}.
2. (a) Let W(t, s) be the impulse-response matrix for a time-varying system. Define ifI(t l ,
... ,
tn) =
T I, .. ·, Tn
[W(t~' W(tn'
Td
.. ;.
" " Tn)].
T 1)
•••
W(tn'
Tn)
Show that if W is realizable, a necessary and sufficient condition for W to be minimally realizable by a completely controllable system of dimension 11 is that for some time sequence t I' ' .. , t n and an arbitrary time TO' there exists a time sequence T l, ••. , Tn' T; > TO, i = 1, ... ,11, such that
(b) Apply this result to the impulse responses W(t, r) =
e/+t,
W(t,T) =
{6: - till -
r ],
Itl~I,
ITI~I,
otherwise,
to decide whether they are minimally realizable.
6
142
REALIZA nON THEORY
3. Prove the following uniqueness result for extending a finite data string: Given a finite sequence of data f N' = (J I' J 2' ... , J N') such that rank .Jft' N' N = rank
.Jft' N' + I . N
(t)
= rank .Jft' N'. N + I
for some N, N' such that N + N' = N*, the extension of the sequence ,IN' to {J)'J2' ... 'JN.'JN.+I •••.• JN.+b ... }, I ~ k ~ 00, for which rank .Jft' m'. m
= rank .Jft' N'. N,
where m' + m = N* + k, is unique. (In other words, an extension of f N' which preserves the rank condition (t) is unique.) 4. Let f N' = (J l ' ...• J N') be a finite data string and let £N' be a partial realization of f N0' Show that the dimension of a minimal partial realization satisfies the following inequality: min dim
£N0
~
N°
L
N'
rank
.Jft'j. N0+)- j -
j=1
L
rank
.Jft'j. N0- i :
j=1
where .Jft'ij is the Hankel matrix associated with f N0' 5. Let f M = {J 1••.• , J M} be a finite data string and define the integer fI(M) = n)(M)
+ ... + nM(M),
where number oflinearly independent rows in the block row [J 1... J MJ. number of linearly independent rows in the block row [J 2 ••• J M] that are also linearly independent of the rows in the block rows [J 1···JM _ 1 ] , .. ·,and number of linearly independent rows in the matrix J M which are also linearly independent ofthe rows ofthe matrices J 1•.•.• J M - iAlso define N'(M) N(M)
the first integer such that every row ofthe block row [JN' +) ••• J MJ is linearly dependent on the rows of the Hankel matrix .Jft' N'. M - N' , the first integer such that every column of the block column
••
is linearly dependent on the columns of the matrix
.Jft'M-N.N'
Prove that (a) ii(M) is the dimension of the minimal realization of the sequence f
M'
143
MISCELLANEOUS EXERCISES
(b) N(M) and N'(M) are (separately) the smallest integers such that the rank condition (R)
rank .1fN'N = rank .1fN' + 1.N = rank .1fN',N+ 1
holds for some extension of f M' (c) N(M) and N'(M) are (separately) the smallest integers such that Eq. (R) holds simultaneously for all minimal extensions of f M; (d) There is a minimal extension of f M of order M'(M) = N'(M) + N(M) for which Eq. (R) is true and whose realization can be computed by the Ho algorithm (but which is, in general, non unique). (e) Every extension that is fixed up to M'(M) is uniquely determined thereafter. 6. Let R(z) be a proper rational matrix with the expansion R(z) = J oz -
1
+ J 1Z - 2 + ....
Define the McMillan degree lJ of R as
... J.] lJ = rank .1f y _
0:
l'
J,:, J 2i
with y being the degree of the least common multiple of the denominator of R(z).
Show that: (a) R(z) has a minimal realization of dimension lJ. (b) If R(z) has the partial fraction expansion y
R(z) = :LZi(Z
i= 1
+ Ai)-1,
then lJ = :L[=1 (rank Zi)' (c) Compute the McMillan degree of z
Z2
R(z) =
[
and find a minimal realization.
+
1
+ 2z +
1
1
z+2
Z2
Z2
z+ 1
+
3~
+2
]
144
6
REALIZATION THEORY
Letf: Q -+ r be a given input/output map. Consider any factorization of f = hg through a space X such that
7.
Q
---.L- r X
If g is onto and h is 1-1, the factorization is said to be canonical with a minimal state space X.
(a) Prove that a canonical factorization always exists. (b) Show that all canonical factorizations are equivalent in the sense that hg = h'g' implies the existence of a unique isomorphism q: X -+ X' such that g' = qg and h = h'q. (c) Translate the above result into the language of matrix theory. 8. What is the connection between the partial realization problem of Section 6.7 and the classical problem of Pade' approximation. (Recall: The Pade' approximation problem is that of finding a scalar rational function whose Taylor series expansion matches that of a given analytic function up to a fixed number of terms.) 9. Let f/ = {JO,J1,J 2, ...} be a finitely realizable behavior sequence. (a) Show that J, = J; if and only if there exists a signature matrix S such that FS = SF',
sa =
H'
(Recall: a signature matrix is a diagonal matrix whose diagonal elements are ±1).
(b) Prove that f/ has a canonical realization L = (F, G, H) with F = P, G = H' if and only if Yt";; = Yt";;, i = 1, 2, 3, .... 10. (a) Prove that the sequence f/ = {J l' J 2' J 3""} admits a finite-dimensional realization of dimension n if and only if there exist scalars IXl' IX2, ••• , IXn such that i = 1, 2, ....
(b)
Show that the set
{IX;}
satisfy
for any F associated with a canonical realization of f/. (c) What is the connection between the IX; and the characteristic and minimal polynomials of F?
145
NOTES AND REFERENCES NOTES AND REFERENCES
Section 6.1 The first statements of the realization problem for linear systems given in transfer matrix form and an algorithm for its solution are found in Kalman, R., Canonical structure of linear dynamical systems, Proc. Nat. Acad. Sci. U.S.A. 48, 596--600 (1962).
Gilbert, E., Controllability and observability in multivariable systems, SIAM J. Control 1, 128-151 (1963).
Kalman, R., Mathematical structure of linear dynamical systems, SIAM J. Control I, 152-192 (1963).
An alternate treatment is by Kalman, R., Irreducible realizations and the degree of a rational matrix, SIAM J. Appl. Math. 13, 520--544 (1965).
The first effective procedure for carrying out the realization procedure in the case of input/output data given in "Markov" form is presented by Ho, B. L., and Kalman, R., Effective construction of linear state-variable models from input/ output functions, Reqelunqstechnik, Prozefi-Datenoerarbeit. 14,545-548 (1966).
Important new work on realization theory in the context of identification of econometric models has been presented in Kalman, R., System-theoretic critique of dynamic models, Int. J. Policy Anal. & Info. Syst., 4, 3-22 (1980).
Kalman, R., Dynamic econometric models: A system-theoretic critique, in "New Quantitative Techniques for Economic Analysis" (Szego, G., ed.) Academic Press, New York, 19-28, 1982.
Mehra, R., Identification in control and economics, in "Current Developments in the Interface: Economics, Econometrics, Mathematics" (Hazewinkel, M., and Rinnooy Kan, A. H. G., eds.) Reidel, Dordrecht, 261-286, 1982. Picci, G., Some connections between the theory of sufficient statistics and the identifiability problem, SIAM J. Appl. Math., 33, 383-398, (1977). Deistler, M., Multivariate time series and linear dynamical systems, Advances in Stat. Anal. & Stat. Computation, vol. 1 (to appear 1986). Deistler, M., General structure and parametrization of ARMA and state-space systems and its relation to statistical problems, in "Handbook of Statistics," vol. 5 (Hannan, E., et al., eds.) Elsevier, 257-277, 1985. .
Sections 6.2-6.4 The realization algorithm, together with the basic concepts of algebraic equivalence, minimality, etc., are discussed in the paper Silverman, L., Realization of linear dynamical systems, IEEE Trans. Automatic Control AC-I6, 554-567 (1971).
The canonical structure theorem given in Exercise 2(a), first appeared in Kalman, R., Canonical structure of linear dynamical systems, Proc. Nat. Acad. Sci. U.S.A. 48, 569--600 (1962).
6
146
Section 6.6
REALIZATION THEORY
A good reference for realization of transfer matrices is
Rubio, J., "The Theory of Linear Systems," Academic Press, New York, 1971.
Additional results are given by Rosenbrock, H. H., Computation of minimal representations of a rational transfer function matrix, Proc. IEEE 115,325-327 (1968). Mayne, D., Computational procedure for the minimal realization of transfer-function matrices. Proc.IEEE 115.1368--1383 (1968). Youla, D., The synthesis of linear dynamical systems from prescribed weighting patterns, SIAM J. Appl. Math, 14, 527-549 (1966). .
Section 6.8 A detailed mathematical (and historical) account of the partial realization problem is given by Kalman, R., On partial realizations of a linear input/output map, in "Ouillemin Anniversary Volume" (N. de Claris and R. Kalman, eds.). Holt, New York, 1968.
Similar work (carried out in collaboration with R. Kalman) is reported by Tether, A., Construction of minimal linear state variable models from finite input/output data. IEEE Trans. Automatic Control AC-15, 427--436 (1970).
and by Willems, J., Minimal Realization in State Space Form from Input/Output Data. Mathematical Inst. Rep., Univ. of Groningen, Groningen, Holland, May 1973.
The rather extensive connections and interrelationships between the partial realization problem. Kronecker indices, canonical forms, continued fractions and much more are developed in detail in Kalman, R., On partial realizations, transfer functions and canonical forms, Acta Poly. Scand., Math. and Compo Ser., No. 31,9-32,1979.
The recursive realization algorithm is due to Rissanen, J., Recursive identification oflinear systems, SIAM J. Control 9, 420--430 (197 I).
Other work along the same lines is reported by Rissanen, J., and Kailath, T., Partial realization of random systems, Automatica-J, IFAC 8, 389-396 (1972).
The actual numerical computation of a canonical realization by the recursive scheme given in the text is an unstable numerical operation. The reasons for the instability, as well as a procedure to "stabilize" the Rissanen's algorithm, are covered in de Jong, L. S., Numerical aspects of recursive realization algorithms, SIAM J. Control & Optim., 16, 646-659 (1978).
See also Pace, I. S., and Barnett, S., Efficient algorithms for linear system calculations II: Minimal realization, Int. J. Sys. Sci. 5,413-424 (1974).
CHAPTER
7
Stability Theory
7.1 INTRODUCTION
Historically, the circle of questions that have now grown into the body of knowledge known in various guises as "system theory," "cybernetics," "system analysis," etc., began with consideration of the equilibria positions of dynamical processes. In rough form, the basic question was whether or not a given equilibrium position was stable under sufficientlysmall perturbations in either the system parameters or initial condition, i.e., if a system were originally in a rest position and some outside influence caused it to depart from this state, would the system return to its original position after a sufficiently large amount of time? Clearly, questions of this type are of extreme importance not only for the type of classical mechanical systems which originally motivated them, but also for numerous economic, social, and biological problems in modern life. The type of stability just mentioned might well be termed "classical" stability since, not only did it originate with so-called classical problems, but it also makes no mention of a controlled system input. Thus classical stability is a property of the internal free dynamics of the process, clearly a limited situation from the viewpoint of modern system theory, although it remains a venerable branch in the theory of ordinary differential equations. In any case, of much greater interest to the system theorist is the idea of a controlled return to equilibrium. As before, the original system is perturbed from its equilibrium state, but now we wish to allow direct interaction with 147
148
7
STABILITY THEORY
the system by means of a control law in order to have the trajectory return to equilibrium. This new concept of controlled input to achieve stability raises several basic questions, among them: (i) Does there exist any control law returning the system to equilibrium? (ii) If there is more than one stabilizing law, can we isolate a unique law by specifyingadditional restrictions such as minimum energy, minimum time, etc.? (iii) What are the possibilities for altering the stability characteristics of the original free system by means of suitably chosen feedback control laws? In connection with controlled inputs, we are also faced with the basic dichotomy of open- versus closed-loop (or feedback) control laws. Our discussion in Chapters 1-6 has been primarily devoted to controls of form u = u(t), the so-called open-loop laws. We will now see, however, that it is particularly advantageous to consider laws of form u = u(y(t), t) in which the controlling action is generated by the current output (or state) of the process. Such laws were originally introduced by Maxwell in connection with regulation of the Watt steam engine and have taken on increased importance with the development of modern techniques for information processing by analog and digital computers. The motivation for the openversus closed-loop terminology is clearly indicated in Figs. 7.1 and 7.2.
"'' --1__-. . J~
r-.---yl I)
,'01 ull)
FIG. 7.1 Open-loop control.
FIG. 7.2 Closed-loop control.
A stability concept differingfrom that presented above is structural stability. Here we are concerned with a family of systems and the type of stability behavior manifested as we pass from one member of this family to another. Roughly speaking, a given system is structurally stable if all ..nearby" members of the given family exhibit the" same" type of qualitative behavior. We shall make these notions precise in a later section. The important point to note now is that structural stability is a property that seems reasonable to demand for all mathematical models pretending to represent reality. This is due to the inherent uncertainties and simplifications present in all mathematical models so, if the model is to depict faithfully what nature presents to us, it is vitally important that the stability characteristics, which are an
7.2
149
SOME EXAMPLES AND BASIC CONCEPTS
inherent part of any viable physical system, be preserved under small perturbations of the model parameters. Naturally many mathematical techniques have evolved in response to the need to characterize the stability of given systems qualitatively and quantitatively. In this chapter we shall present three basic lines of attack on these questions: Routh-Hurwicz methods, Lyapunov techniques, and frequency-domain approaches. The first is purely algebraic in character, based on the satisfaction of certain algebraic relations which imply the asymptotic stability of the process. The remaining two approaches are much more analytic in flavor, their point of view being that the stability of the process may be inferred from the behavior of certain auxiliary functions naturally arising from the physics of the original problem, an energy function for the Lyapunov approach, a transient response function in the frequency method. 7.2 SOME EXAMPLES AND BASIC CONCEPTS
To gain a clearer understanding of the types of stability we shall encounter below, consider the motion of an oscillating system (mechanical, electrical, or ?) described by the linear differential equation
x + C1 X + C z x =
a,
C z =1=
x(O) = x(O) = O.
0,
(7.1 )
Consider the characteristic polynomial of (7.1): X(z)
It has zeros
Ctt> Ctz
=
ZZ
+ ClZ + Cz.
(7.2)
and we can easily verify that the general solution of (7.1) is
where k l and kz are constants of integrations (here we assume that Ctl # Ctz). If one of the numbers Ctl or Ctz has a positive real part, then the trajectory of (7.1) becomes unbounded as L--+ 00, regardless of the value of a. If both real parts are nonpositive, then both x and x can be made arbitrarily small by suitable choice of a. Thus the equilibrium of (7.1) is stable if and only if Re a, S 0, i = 1, 2. In physical terms, the statement" (7.1) is stable" means that a small change in the external driving force a results in only a small change in the displacement x(t). Since Cl and Cz have physical interpretations as damping and spring constants, respectively, we see immediately that a sufficient condition for Re rJ.j S 0 is Cl > 0, Cz > 0, which is the natural operating condition of the process. Later we shall see that necessary and sufficient conditions for the
150
7
STABILITY THEORY
stricter requirement Re (Xi < 0 are (I) C I > 0, (2) C I C2 > 0, which, in this case, is clearly equivalent to the natural conditions just imposed. Thus we see that an undamped spring (c I = 0) cannot possibly be stable with respect to external perturbing disturbances a =F 0, confirming our physical intuition about such processes. Now assume that the oscillating circuit is described by the equation
x+ x
=
x(o) = x(o) = 0.
a sin t,
(7.3)
The explicit solution of this equation is X(f) = ta(sin t - t cos f).
Since for arbitrarily small a =F 0, the system trajectory grows without bound as f ~ 00, this system is unstable with respect to external perturbations of the type a sin t, a =F 0. Equation (7.3) represents an undamped oscillatory circuit tuned to resonance and the instability noted corresponds to the well-known physical fact that an external perturbation whose frequency equals one of the natural vibratory modes of the system will result in unbounded oscillations, i.e., the system resonates with the disturbance. The simple harmonic oscillator also illustrates the concept of structural stability. Assume the equation of motion is X(O)
= a,
(7.4)
x(O) = 0.
Thus there is no forcing term and we investigate the influence of the parameters CI and C2 on the system trajectories. On physical grounds, we consider . only the situation CI ~ 0, C2 > 0. Considering the motion of (7.4) in the (x, x)-plane, we easily see that if CI = 0, the trajectories are concentric circles with the center at the origin and radii a(c2)1/2 (Fig. 7.3). Assume now that we introduce some damping into the system. As mentioned, this means mathematically that CI > in (7.4). If c 1 2 ~ 4c2 , the phase plane portrait of the system shows that the equilibrium point x = x = is a node (Fig. 7.4); while if Cl 2 < 4c2 , it is afocus
°
°
--+-+-1F-+H---+---x
FIG. 7.3 Trajectories of Eqs. (7.4) for c. =
o.
7.2
151
SOME EXAMPLES AND BASIC CONCEPTS
-----i'-------x
FIG. 7.4 A typical system trajectory when c 1 2 ~ 4C2 and the origin is a stable node.
CI
2
FIG. 7.5 A typical system trajectory when « 4C2 and the origin is a stable focus.
(Fig. 7.5). In either case the origin is stable with respect to perturbations in C l and C 2• This behavior is in stark contrast to the undamped case in which the origin is a center and its qualitative character may be changed by an arbitrarily small change in Cl' Thus the processes with Cl =1= 0 are examples of structurally stable systems, in that the qualitative behavior of the equilibrium point (focus, node) is preserved under small changes of the system structure. The foregoing considerations illustrate the main intuitive notions surrounding the stability concepts we are interested in pursuing. To create a mathematical framework within which these ideas may be studied, we must formalize our intuition with precise definitions. Definition 7.1 The equilibrium x = 0 of.the differential equation x = f(x), x(to) = xo, is called stable (in the sense of Lyapunov) if, for each B > 0, there exists a ~ > 0 such that
"x(t)" <
is valid whenever "xoll <
B,
~.
Geometrically, this definition means that the system trajectory always stays within a "tube" of radius B if the initial state belongs to a smaller sphere of radius s (Fig. 7.6). REMARK Iff = [tx, t), i.e., the system dynamics are not autonomous, then the number ~ depends on to, the initial time.
Dejillitioll 7.2 The equilibrium x = 0 of the system ottractor if there exists an " > 0 such that
lim x{t) = 0
whenever
x=
Ilxoll < n.
f(x) is called an
7
152
STABILITY THEORY
x(t)
e
FIG. 7.6 Stability in the Sense of Lyapunov. REMARK The equilibrium x = 0 is assumed here to be isolated, i.e., the neighborhood Ilxoll < h contains no points x', other than x = 0, such that f(x') = 0 for h sufficiently small. EXERCISE
1. Show by counterexample, or other technique, that the equilibrium x = 0 may be stable but not an attractor and conversely (in other words, the two concepts are independent). The type of stability of most interest to us is when the notions" stable" and "attractor " are equivalent. This is given by the next definition. The equilibrium x = 0 of the system i = f(x) is called asymptotically stable (in the sense of Lyapunooi if it is both stable and an attractor. Definition 7.3
Definitions 7.1-7.3 refer to stability of a point which is the classical concept. We defer precise definitions of structural stability to a later section. 7.3 ROUTH-HURWICZ METHODS
We begin by considering the free (uncontrolled) constant, linear dynamical system (7.5) x(O) = xo, i = Fx, where x is an n-vector of states and F an n x n constant matrix. Since we are interested in the stability of the equilibrium state x = 0, we assume Xo ¥ 0 and
7.3
153
ROUTH-HURWICZ METHODS
seek conditions on F that ensure that the trajectories of (7.5) tend to zero as t --+ co, Imitating an argument from the theory of scalar linear differential equations, let us assume that (7.5) has a solution of the form
x(t) = e).lxo'
(7.6)
Substituting (7.6) into (7.5), we find
Ae).lxo
= Fe).lxo
A./xo
or
= Fx.;
Thus system (7.5) will have a nontrivial solution if and only if the parameter A satisfies the characteristic equation (7.7)
det(A./ - F) = 0.
The roots Ai" .. ,An of (7.7) are the characteristic roots of F and, as we see from (7.6), their position in the complex plane determines the stability behavior of (7.5) as t --+ co. It is clear that the condition i
=
1, ... , n,
(7.8)
is necessary and sufficient for stability of the origin, while the stronger condition (7.9) Re(Ai) < 0, i = 1, ... , n, is needed for asymptotic stability for all xo. EXERCISE
1. If T is a nonsingular n x n matrix, show that F and T FT - 1 have the same characteristic roots. The above considerations show that one road to the study of the stability of the origin for (7.5) lies in the direction of an algebraic criterion characterizing the position of the roots of the polynomial equation (7.7). More specifically, we desire a procedure based on the coefficients of (7.7) which ensures satisfaction of (7.9) (or (7.8)). Such a method is the well-known RouthHurwicz criterion.
Theorem 7.1 Let X(z) = aoz n + aiZ n- i + ... + an-iz + an be the characteristic polynomial of F. Then a necessary condition that the roots ofx(z) have negative real parts is ai/aO > 0, a2/aO > 0, ... ,an/ao > 0. Let z 1, •.• , z; be the zeros and, in particular, let z/ be the real roots, complex ones. Then
PROOF
zZ the
XF(Z) = ao
n (z j
z/)
n (z k
zZ),
7
154
STABILITY THEORY
and, combining the complex conjugate factors, we obtain
XF(Z) = ao
n (z -
z/)
n (Z2 -
+ Iz~12).
2 Re(z~)
k
j
If all the numbers z/ and Re(z;) are negative. we can only obtain positive coefficients for the powers of z when we multiply the two products together. EXERCISE
1. Give an example of a polynomial that satisfies Theorem 7.1, but has at least one root with a nonnegative real part. As the foregoing exercise illustrates, Theorem 7.1 is, unfortunately, far from a sufficient condition for stability of the polynomial XF(Z). It does, however, provide a very simple test to discover systems that have no chance of being asymptotically stable. Our next result, given without proof, is the Routh criterion which provides necessary and sufficient conditions for stability.
Form the number array
Theorem 7.2
C22 = a4 - r2a S . C32 = a6 - r2a7' · · · . Cl3
=
C21 -
r3Cn , C23 =
C31 -
C33 =
C41 -
r3 c 32 , r3 C42'···.
i = 1,2, . . .• j = 2,3, ... ,
Ctn =
an'
Define Cm+l,O
=
Cm+I.2
= an,
Cm+l. l C m2
= =
Cm+I.3 C m3
=
= 0
0
if
if
n = 2m n=2m-1.
Then the polynomial XF(Z) has only roots with negative real parts
if Theorem 7.1 is satisfied and CII
> 0,
Cl2
>
O,,,,,Cl n
>
O.
if and only
7.3
155
ROUTH-HURWICZ METHODS
REMARK If one of the numbers Cij = 0, the Routh scheme breaks down, in which case, it is easy to see that then XF(Z) cannot be a stability polynomial.
A close relative of the Routh scheme was discovered by Hurwicz in connection with the study of the stability of a centrifugal steam engine governor. This result involves the array of numbers
A=
a1 ao 0 0 a3 a2 a1 ao as a4 a3 a2 0 0
0 0
0 0
0 0
0 0 0
0 0 0
an-1 0
an-2 an
formed from the coefficients of XF(Z), If 0 < 2i - j ~ n, the general element in the array is aij = a2;- j,otherwiseaij = O. We form the sequence of principal subdeterminants
These quantities, the so-called Hurwicz determinants, give rise to the next theorem. Theorem 7.3
The polynomial XF(Z) has all its roots with negative real parts
if and only if Theorem 7.1 holds and H 1 > 0,
H 2 > 0, ... , H n > O.
PROOF By elementary row operations, we convert the array A to lower triangular form. The numbers appearing on the main diagonal are the quantities C 11 = a 1, C 12' .•. , C 1 n = an, which are precisely the quantities from the Routh scheme of Theorem '7.2. Since elementary row operations leave the principal subdeterminants invariant, we have
and
Thus the Hurwicz determinants are all positive if and only if the same is true of the Routh numbers {Clj}'
156
7
STABILITY THEORY
EXERCISES
1. Check the following polynomials for stability:
xAz) = (b) XF(z) = (a)
(c)
XF(Z)
=
Z6 Z4
Z5
+ 5z 5 + 3z4
+ 2z + 2
-
2z 3 +
Z2
+
7z
+
6;
I;
+ 3z4 + az 3 + tz 2 + 18z + 1.
2. Use the Routh-Hurwicz criteria to state explicitly the necessary and sufficient conditions for the asymptotic stability of second-, third-, and fourth-order systems. 3. (Michailov Criterion) Define the polynomials
V(z)
=
X(z)
+ iY(z).
Show that XF(Z) = aoz n + alZ n - 1 + ... + an is a stability polynomial if and only if the zeros of X and Yare all real and separate each other. From a computational point of view, the Routh scheme is probably preferable to the Hurwicz procedure, although the Hurwicz method is particularly valuable when the influence of changes in the coefficients on the system stability is under investigation.
7.4 LYAPUNOV METHOD
In many situations, particularly when the dimension of the system is high, it is a nontrivial task to determine the coefficients of the characteristic polynomial of F. Unfortunately, it is precisely these quantities that are required in order to apply the simple algebraic tests prescribed by the Routh-Hurwicz theorems. Thus we are motivated to seek alternate procedures that operate with only the given system data, i.e., criteria that can be directly applied to F itself. Such .a procedure is the celebrated "second" method of Lyapunov. The Lyapunov procedure is intuitively based on the simple physical notion that an equilibrium point of a system is asymptotically stable if all trajectories of the process beginning sufficiently near the point move so as to minimize a suitably defined "energy" function with the minimal energy position being at the equilibrium point itself. The trick, of course, is to discover an appropriate energy function that is both complex enough to capture the relevant stability behavior and simple enough to obtain specific mathematical expressions characterizing stability or instability.
7.4
157
LYAPUNOV METHOD
To make the foregoing notions more precise, we consider a process described by the equations
x=
f(x),
x(O) =
Xo,
(7.10)
where the origin is the equilibrium point under consideration, i.e.,- f(O) = O. For simplicity, assumefis a continuous function. Definition 7.4 A function V(x) is called positive definite if V(O) = 0 and if V(x) is positive at every other point in a neighborhood U of the origin. EXERCISES
1. (a) Show that if V(x) is positive definite, then there exist continuous, strictly monotonically increasing functions q>l(r), q>2(r), r > 0, with q>1(0) = q>2(0) = 0 such that
q>l(llxll) :5 V(x)
:5
q>2(llxll).
1
(b) Show that V- (x ) satisfies
q>i 1(llxll) :5 V- 1(x ) :5 q>1 1(llxll). 2.
If u(t), v(t) satisfy the differential equation duldt = f(u)
and the differential inequality dtfdt :5 f(v),
t > 0,
respectively, show that if u(O) = v(O), then v(t) :5 u(t),
t > O.
The above exercises enable us to prove the following basic result on stability of the equilibrium x = 0 of (7.10). Theorem 7.4 If there exists a positive definite function V(x) whose derivative, when evaluated along trajectories of (7.10), is negative definite, then the equilibrium of (7.1 0) is asymptotically stable. PROOF
By assumption, we have
for some continuous, monotonically increasing function q>4. Applying Exercise I(b), we have (7.11)
158
7
STABILITY THEORY
and, hence, for x. a continuous, increasing, positive function. By Exercise 2, we see that
for some function q and a continuous, decreasing function p. Because of (7.11)
Ilxll s
lpi1(q(VO)p(t) ~
lpi 1(q(lpl(ll xo II)))p(t),
which finally yields
Ilxli for some decreasing function as t -+ 00.
0".
~ lp(lI xolI)O"(t)
Thus all trajectories of (7.10) decrease to 0
From Theorem 7.4, we are led to define the notion of a Lyapunov function for (7.10). Definition 7.5
A function Vex) such that
(a) V is positive definite for all x E R" and (b) dV/dt < 0 along trajectories of (7.10) is called a Lyapunov function for (7.10). There are many delicate nuances associated with the use of Lyapunov functions to study stability, particuiarly in the cases when V = 0 along system trajectories. However, we shall not dwell on these situations here, as our concern is with illustrating the basic ideas unencumbered by technical details. We refer to the treatises listed at the end of the chapter for a complete discussion of all issues. The main theorem associated with the use of Lyapunov functions is Theorem 7.4, which gives only sufficient conditions for stability. In addition, it says nothing about how one goes about finding a suitable function or, for that matter, whether or not such a function even exists! However, it can be shown that the existence of an appropriate Lyapunov function is also a necessary condition for stability and, what is equally important for our purposes, in special cases we can develop a systematic procedure for obtaining such functions. Since our interest in this book is with linear systems, we seek to apply the Lyapunov procedure to the linear system
x=
Fx,
x(O)
=
Xo ( =1= 0).
7.4
159
LYAPUNOV METHOD
As a candidate Lyapunov function, we choose Vex) as the quadratic form Vex) = (x, Px),
where P is an, as yet, unknown symmetric matrix. In order that Vex) be a Lyapunov function for the system, we must check condition (b) on the derivative of Vex). We obtain Vex)
= (x, Px) = (x, F'P
+ (x, Px) < 0 + PF)x) < 0,
which implies that the equation F'P
+ PF
(7.12)
= -C
must be solvable for any C > o. Furthermore, condition (a) of Definition 7.5 implies that the solution of (7.12) must be positive-definite, i.e., P > O. Hence, we have Theorem 7.5. Theorem 7.5
The equilibrium of the system
x=
Fx
is asymptotically stable if the linear matrix equation F'P+PF= -C has a positive-definite solution P for any matrix C > O.
EXERCISES
1. Show that the existence of a quadratic Lyapunov function is also a necessary condition for the stability of the origin for the equation x = Fx. 2. Prove the following strengthened version of Theorem 7.5: The equilib-
rium of x = Fx is asymptotically stable if and only if: (1) Eq. (7.11) has a positive semidefinite solution P ~ 0 for any C ~ 0, and (2) no trajectory of x = Fx vanishes identically on {x : Vex) = O} except the trajectory x == O. EXAMPLES
(I)
the solution of F' P
Let F = [- J
+ PF
= - C
_n
C = [c i j
].
It is easily verified that
is given by
for all C>
o.
Thus we verify that the system x = Fx is asymptotically stable.
160
7
STABILITY THEORY
n
(2) Let F = [_? _ C = [Cjj]. Since trace F < 0 and det F > 0, by the Routh-Hurwicz criteria, it is easily checked that F is a stability matrix. We verify this conclusion by application of Theorem 7.5. Substituting F and C into Eq. (7.12), we obtain equations for the components of P yielding
P=
[1<2C ll ~
cn) -
Cl2 1 tC11 1{c11
lC1 1
°
+ C22)
J.
Since this P > if C > 0, we conclude asymptotic stability of the origin for the equation x = Fx. It is interesting and instructive to also consider the stability problem for discrete linear systems, i.e., when x(t
+
I) = Fx(t),
t
=
0, 1, ....
(7.13)
Obvious modifications of Definitions 7.4 and 7.5 lead us to consider the candidate Lyapunov function V(x) = (x, Px),
where instead of
V, we form
the difference
d V = V(x(t
+
1» - V(x(t».
Using (7.13), this yields dV
= (x(t), (F'PF
- F)x(t»
=
-(x(t), Cx(t».
The foregoing considerations lead to the following result. Theorem 7.6 The origin is an asymptotically stable equilibrium position for system (7.13) if and only if there exists a positive-definite matrix P which is the unique solution of the matrix equation F'PF - F = -C
(7.14)
for all C > O.
Thus (7.14) is the Lyapunov matrix equation for discrete linear systems. In terms of the characteristic roots of F, the stability requirement is now that all characteristic roots Ai(F) must satisfy i = 1, ... , n.
(7.15)
7.4
161
LYAPUNOV METHOD
This is easily seen from the fact that (7.13) has the solution t = 1,2, ....
For arbitrary x(O), x(t) -+ 0 if and only if the bound (7.15) is satisfied by the characteristic roots of F.
EXERCISES
1. Show that the solutions of(7.12) and (7.14)are related by the transformations
where B is the Cayley transform of F' such that B
= (1 + F')(I - F')-I.
That is, if P is the solution of (7.12), then using the transformations F -+ B, C -+ 2(1 - FT IC(1 - F)-I, we may convert (7.12) into (7.14), retaining the same solution P. 2. (a) Verify that the solution of the matrix equation
dX/dt = AX
+ XB,
X(O) = C,
is given by X(t) = eAtCe Bt. Use this fact to show that the solution of AX XB = C is given by
+
assuming the integral converges. (b) What conditions must A and B satisfy to guarantee convergence of the integral for X? (c) Specialize the above results to Eq. (7.12). 3. Show that the matrix equation PF+F'P= -C
is equivalent to the vector equation [(F'
® I) + (1 ® F')]a(P) = -a(C),
162
7
STABILITY THEORY
where ® denotes the Kronecker product oftwo matrices and a the "stacking" operator which stacks the columns of an n x m matrix into a nm x 1 vector, i.e., if A = [aij], then
7.5 FREQUENCY-DOMAIN TECHNIQUES
As might be expected, stability theory also possesses a geometric side in addition to its analytic and algebraic sides as typified by the Lyapunov and Routh-Hurwicz procedures, respectively. Although we shall not go into great detail about the frequency-domain approaches in this book, they should be considered as another tool in the systems analyst's arsenal of methods to employ for the study of particular questions. Basically, the frequency methods investigate various stability properties of a system by analyzing the transfer matrix Z(A) as a function of the complex variable A.. Recall that if 1: is given by
x=
Fx
+ Gu,
x(O) = 0,
y = Hx,
(7.16)
then
Thus the characteristic roots of F coincide with the poles of the rational matrix function Z. The frequency-domain methods study the asymptotic behavior of both the open-loop system (7.16) and the closed-loop system obtained from (7.16) by using a feedback-type input u = - Kx for some m x n matrix K. One of the basic geometric results involves determination of conditions under which the closed-loop system will be asymptotically stable if the openloop system is stable. This is the so-called Nyquist theorem which gives a simple geometric criterion for stability of the closed-loop system for the case of a single-input/single-output system. To state this result we must first discuss the notion of a response diagram for a rational function of a complex variable s. Let r(s) be a proper rational function of the complex variable s, i.e., r(s) = p(s)!q(s), where p and q are monic polynomials with deg q > deg p. Then we call the locus of points r(r)
=
{u
+ iv: u =
Re[r(iw)], v
= Im[r(iw)], -
00 ~
w ~ oo}
the response diagram of r. In other words, the response diagram is the image of the line Re s = 0 under the mapping r. The basic result of Nyquist follows
7.5
163
FREQUENCY-DOMAIN TECHNIQUES
upon setting res) = Z(s) in the open-loop case, and res) = Z(s)/O + kZ(s» in the closed-loop case, with k being the output feedback gain, i.e., u(t) = ky(t) = khx(t), where the output matrix h is 1 x n. Nyquist's Theorem The closed-loop system is asymptotically stable if the open-loop system is stable and if its response diagram, traversed in the direction of increasing t» has the" critical" point ( - 1/ k, 0) on its left. Geometrically, the situation is as in Figs. 7.7 and 7.8. The proof of Nyquist's result can be obtained as a fairly straightforward application of the principle of the argument from the theory of functions of a complex variable.
-11k
FIG.7.7
Nyquist diagram: stable case.
FIG.7.8
Nyquist diagram: unstable case.
A far-reaching generalization of Nyquist's theorem, which also gives instability criteria, has been obtained under the name the "circle theorem." The basic technique used for this result is based on a scalar representation of the equations of motion. We consider the system x(t) = Fx(t) y(t) = hx(t),
+ gu(t),
(7.17)
and the related differential equation x(t) = Fx(t) - gf(t)hx(t).
(7.18)
It is easy to verify that if (7.17) is controllable, we can reduce it to control canonical form and the differential equation (7.18) may then be written in the form P(D)x(t)
+
f(t)q(D)x(t)
=
0,
where p and q are relatively prime polynomials of degree n, D the differential operator D = dldt, and f(t) a piecewise-continuous function such that (X < f(t) < {3.
7
164
Letting .@(IX, {3) represent the disk .@(IX,
+
{3) = {u
+~
iv: [u
(~
+ ~)
J
+
2
v <
STABILITY THEORY
~ ~ ~ 1
_
2 1
}.
we can state the theorem. Circle Theorem Let 1: = (F, g, h) be a minimal realization of the transfer matrix Z(A). Assume that v characteristic values of F lie in the half-plane Re A > 0 and that no characteristic values ofF lie on the line Re A = O. Iffis a piecewise-continuous function satisfying IX < f(1) < {3, then (a) All solutions of x = [F - gf(t)h]x are bounded and go to zero as t -+ 00 provided the response diagram ofZ(A) does not intersect the circle .@(IX, /3) and encircles it exactly v times in the counterclockwise sense. (b) At least one solution of x = [F - gf(t)h]x does not remain bounded as t -+ 00 provided the response diagram of Z(A) does not intersect the circle .@(IX, {3) and encircles itfewer than v times in the counterclockwise sense. EXERCISE
1. (a) Use the circle theorem criterion to show that the equation x(t)
is stable if, for some
IX
+
2x(1) + x(t)
+
f(t)x(t) = 0
> 0, we have IX 2
< f(t)
+
1<
(IX
+
2)2.
Hence, conclude that the null solution of
+
x(t)
2x(t)
+
g(t)x(t) = 0
is stable for IX
2
< g(1) <
(IX
+ 2f.
(b) Use the instability part of the circle theorem to show that the null solution of x(t)
+ 2x(t) + g(t)x(t)
= 0
is unstable if!(t) < -1 for all t. 7.6 FEEDBACK CONTROL SYSTEMS AND STABILITY
We return to linear systems of the form x(t)
=
F(x)t
+
Gu(t),
x(O)
= xo '1= 0,
(7.19)
7.6
165
FEEDBACK CONTROL SYSTEMS AND STABILITY
where F and G are constant matrices of sizes n x nand n x m, respectively. Our concern is with the question of how the input function u(t) may influence the stability of the system ~ = (F, G, -). The trivial example A. I
F= [
o
]
0 0
G=
Ai> 0, i i= n, 0 1
shows that the property of controllability will essentially influence our ability to modify the stability characteristics of ~ by means of external inputs u(t). In addition, the representation x(t) = eFtxo
+
{eF(t-SlGU(S) ds
(7.20)
shows that an unstable free system cannot, in general, be stabilized by means of an open-loop input function u. More precisely, we have Theorem 7.7.
Theorem 7.7 Assume the free system x = Fx is asymptotically stable. Then the controlled system (7.19) is also asymptotically stable under open-loop input
if and only if
lim jtGU(S) ds < 1-00
Conversely, PROOF
00.
0
if the free system is unstable, then so is the controlled system.
From (7.20) we have x(t) = ?{x o
+
{e-FSGU(S) dsJ
Thus if F is a stability matrix, x(t) -+ 0 for all Xo if limt~ a: f~ Gu(s) ds < 00. Conversely, if F is unstable, then, since Xo is arbitrary, x(t) is unbounded for any choice of input u, which completes the proof. The foregoing theorem shows that the system structure characterizing the essential stability properties of ~ remains unchanged by application of openloop inputs. As a result, we will focus our attention on closed-loop or "feedback" inputs of the form u(t) = - Kx(t),
where K is an m x n constant feedback matrix. Our goal in this section is to examine to what degree the stability properties of the free system can be modified by application of appropriately chosen feedback laws K.
166
7
STABILITY THEORY
The basic result in the stability theory of linear feedback systems is the so-called pole-shifting theorem.
Pole-Shifting Theorem Let A = P'l' ... , An} be an arbitrary set of symmetric complex numbers. i.e., if A.i EA, then Ai EA. Assume the system E = (F, G, -) is completely controllable. Then there exists a unique feedback matrix K such that the closed-loop system
x=
(F - GK)x
has A as its set of characteristic values. PROOF Without loss of generality, assume that G has full rank. Then from Chapter 5, Eq. (5.7), since (F, G) is completely controllable, there exists a coordinate change T in the state space X such that the system matrix F assumes form (7.21).
ft
= TFT-
0 0 0 x
x
l
0 ... 0 0 1 ... 0 I
0 x
x
0 x
x
... .. .
. ..
1 x
.. . x
x
.. . x
x
0 1 0 ... 0 0 1 ...
0 0
... .. .
1 x
0 x
0 x
0 x
x
x
x
...
x
... x
.. .
x
... x
0 1 0 ... 0 0 0 1 ... 0 0 x
0 0 x x
... 1 ... x (7.21)
In (7.21)the diagonal blocks are of size (k, x k j ) , k, the ith Kronecker index of the pair (F, G), i = 1, ... , s.
7.6
167
FEEDBACK CONTROL SYSTEMS AND STABILITY
Similarly, the new input matrix has form (7.22).
o o 0 1
0 0
G = TG
0 0 (7.22)
=
0
o o o 1
In (7.22) each nontrivial block is of size k, x 1, i = 1, ... , s. Since the characteristic polynomial XF(Z) of F is invariant under such a coordinate transformation, we have xt(z) = XF(Z) = z" +
!XtZn-t
+ ... + IXn •
Assume the desired characteristic polynomial generated by the set A. is Xi-{;i(Z) = zn
+ PtZn-t + ... + Pn.
Then it is evident from the structure of F and Gthat we may choose K in such a way that (i) all elements marked "x" in (7.21) belonging to off-diagonal blocks become zero; (ii) the elements "x" in diagonal blocks assume prescribed values, i.e., the characteristic polynomials of the diagonal blocks are the factors of the polynomial xt-{;i(Z). Hence, suitable choice of K shows that if mj(z) is the characteristic polynomial of the ith block of F - GK, then
= z" + PtZn-t + ... + Pn = nm;(z). s
XF-{;i(z)
i= I
This completes the proof of the theorem.
168
7
STABILITY THEORY
REMARKS (l) From a design standpoint, the importance of the poleshifting theorem is that it enables one to design the internal dynamics more or less without constraint, safe in the assurance that any type of desired stability behavior can be achieved later by means of appropriately chosen linear feedback. Thus this theorem gives the system designer a sound mathematical basis on which to exercise his imagination in the construction of a given system. (2) The reason for the adjective "pole-shifting" in the name of the theorem arises from the representation of l: as a transfer matrix Z(A). In this case, the characteristic values of F are the poles of Z(A) and the theorem states that if (F, G) is controllable, then the poles of Z(A) may be "shifted" to the arbitrary set A by suitable feedback K. EXAMPLE
Let the system l: = (F, G, -) be given by the matrices
-ll
2 3 -1
- 1
F = [ -~
The characteristic polynomial of F is X~z)
= Z3 - 2z2
-
Z + 2 = (z - 1Hz
+ 1Hz - 2).
Thus F has the two unstable roots + 1 and + 2. Assume we desire to apply linear feedback to shift the roots of the closed-loop system to the stable set A = {-1 , -2, -3} ,
i.e., the characteristic polynomial of F '- GK is to be XF-GK(Z) = Z3
+ 6z2 +
l l z + 6.
To apply the pole-shifting theorem, we first check that the pair (F, G) is controllable. The controllability matrix 2 3 -1
has rank 3, so we may apply the theorem. An easy computation shows that the matrix T reducing (F, G) to control canonical form is given by I 0 T = 1 0 [ o I
7.7
169
MODAL CONTROL
Thus we work with F=TFT- 1=
00 0I 0]I,
[-2
I 2
Since the characteristic polynomial is invariant under the transformation K (in the canonical system) such that the last row of F is modified to match the desired characteristic polynomial XF-GK(Z), i.e., we find K such that T, we see that it is necessary to choose a feedback matrix
Xi-GK(Z) =
The correct feedback law
Z3
+ 6z 2 +
llz
+ 6.
K is given by
K = (4
12 8).
Since K = KT- 1 , the desired feedback law in the original coordinate system is - 1 K = (4
12
12
8) [ - ~
12).
7.7 MODAL CONTROL
The results surrounding the pole-shifting theorem were based on the use of the control canonical form. It is natural to inquire as to whether the Lur'eLefschetz-Letov canonical form yields an equally powerful result on the stability question for linear systems. In this section, we show that such a conjecture is indeed valid, leading to the theory of "modal" control. The basic idea in the modal approach to control and stability is to recognize that the motion x(t) of the free dynamical system x(t)
=
Fx(t),
x(O) = c
(7.23)
may be expressed in terms of the characteristic values and associated characteristic vectors of F. Results on stability and controllability are then expressed in terms of the vectors comprising the characteristic vectors of F. Since the Lur'e-Lefschetz-Letov canonical form is based directly on the characteristic values and vectors of F, we immediately see the connection between this canonical form and the problems of modal control theory. Let us now examine a few of the more important results.
170
7
STABILITY THEORY
Assume, for simplicity, that the characteristic values of F are distinct. Then standard results from linear algebra show that there exists a matrix T such that
F = T FT- 1 = diag(A'l> A2"
.. ,All)'
where the set t\ = {Aj} is the collection of characteristic values of F. Further, it is known that the ith column of T, tv; is the characteristic vector associated with the characteristic value Aj, i = 1, ... , n. Application of the transformation T to the system (7.23) yields the resulting "diagonalized" system ~ = F~,
~(O)
= T c = C,
which has the immediate solution i = 1,... , n.
Since x = Tx, we see that the general solution of (7.23) is given as
L cje).it~j, II
x(t) =
i= 1
where ~j is the ith column of the matrix T- 1• We call the vector ~j the mode of the system corresponding to the characteristic value Ai' i = 1, ... , n. Passing to the controlled version of (7.23), we have x(t) = Fx(t)
+ Gu(t),
x(O) = c.
(7.24)
Utilizing the coordinate change T"we obtain the canonical system
+ Gu,
~ = F~
where F = TFT- 1 , G = TG, and solution of the equation
~(O)
c = Tc.
=
+ L {jjjUJ{t),
(7.25)
The ith component of ~ is the
m
xj(t) = Aj~j(t)
c,
i
= 1, ... , n.
j= 1
It is evident that the jth input variable uit) can influence the element the state vector ~(t) if and only if
~j(t)
of
{jij = (TG)jj '1= 0,
= (1(i)' G(i),
i = 1,... , n, j = 1, ... , m,
(7.26)
where 1(i) is the ith row of the matrix T while G(j) denotes the jth column of G. The ith mode is controllable if and only if it is controllable by at least one input.
7.7
171
MODAL CONTROL
These conditions can be illustrated by considering a system for which
F=
[I °
-I
],
G=
[
~ ~].
-4
°
Since 011 :F 0,021 :F 0,031 :F 0, all three system modes are controllable by input Ul(t) but, since 021 = 0, 022 :F 0,032 = 0, only the second mode (the characteristic vector corresponding to A = 0) is controllable by uit). Definitions analogous to those above may also be given for observable modes, as well as for the case when F has multiple characteristic values. We do not go into these matters here as our objective is only to indicate the use of the Lur'e-Lefschetz-Letov canonical form in stability analysis. For the sake of exposition, assume we are given a single-input system (m = I). Suppose the system has been transformed to Lur'e-LefschetzLetov form, i.e., (F, G) is controllable and
We consider-the closed-loop system
(F -
~Gk)x,
(7.27)
where ~ is a scalar to be determined. Equation (7.27)has the effectof changing the system matrix F to the new matrix IF
= F-
~Gk.
(7.28)
If k is now chosen to be equal to e], which is the jth characteristic vector of F' (=F), then the system matrix of the controlled system given by (7.28) will have the form
IF =
F-
~Ge/.
It then follows that
k :F j.
(7.29)
It is also clear that
!Fej = Fej - ~G = Ajej - ~G,
(7.30)
which implies that Aj is not a characteristic value of IF and also that ej is no longer the corresponding characteristic vector if ~ :F 0. The effect of using the input vector e/ is then to change the characteristic value Aj to some new
7
172
STABILITY THEORY
value Pj and the characteristic vector ej to some new vector Wj; leaving the remaining (n - 1) characteristic pairs unchanged. It can be deduced from the foregoing equations that
k
=1= j,
k = 1,2, ... , n,
and
;Y;'ej = (A j -
(7.31)
~)ej'
which indicates that
Pj = Aj -
~.
(7.32)
This equation implies that the" gain " ~ necessary to alter the jth characteristic value to any desired real value Pj is given by the expression ~ =
Aj - Pj'
(7.33)
EXERCISES
1. Extend the foregoing results to cover the case in which it is desired to
move p characteristic values to new locations Ph"" Pp , P > 1. (Hint: Consider the input u(t) = (Lj= 1 ~je/)x(t), where the ~j are to be determined.) 2. Extend the problem to the case of multiple characteristic roots of F and multiple inputs. The preceding development shows that the pole-shifting theorem may also be established using the Lur'e-Lefschetz- Letov canonical form. The essential point is that the pair (F, G) be controllable, since the jth characteristic value can be modified if and only if the jth component of G is nonzero. This observation raises an important question: If it is only desired to stabilize the system L rather than to shift all of its characteristic values, what conditions must be imposed on F and G to ensure that a linear feedback law K can be found that will shift the unstable roots of F into the left-half plane. The answer to this question is given by the next theorem.
Theorem 7.8 The system L = (F, G, -) is stabilizable by linear feedback if and only if the unstable modes of F are controllable, i.e., the unstable modes ~ir' r = 1, ... , p, corresponding to the roots Air' such that Re Air ~ 0, r = 1, ... ,p, must lie in the subspacegeneratedby the vectors {G, FG, . . . ,pn- 1 G}. PROOF We prove the single-input case. Assume, without loss of generality, that the characteristic roots of F are ordered such that Re Al ~ Re A2 ~ ••• ~ ReAp ~ 0 > ReA p + l ~ ... ~ ReA n • Let T be the transformation that diagonalizes F. Then T FT - 1 = diag(Al , ... , An) and the rows of T are the transposes of the characteristic
7.8
173
OBSERVERS
vectors of F. Note that since (F, g) is not assumed to be controllable, the diagonalized system ~ = F~
+ (ju
is not identical with the Lur'e-Lefschetz-Letov canonical form, as 9 will have some entries equal to zero if(F, G) is not controllable. To prove necessity, we have ~i
=
A.i~i
+ Tqu,
i
=
1, ... , n.
In particular, if i :5: p, we see that ~i is controllable by u only if (Tg)i # 0. That is, the ith mode of F is not orthogonal to 9 which, by Corollaries 3 and 4 to Theorem 3.9, implies that ~i lies in the subspace generated by g, Fg, ... , Fn-1g.
Conversely, suppose
~i
lies in the subspace generated by the vectors of
°
g, Fg, ... , Fn-1g. Since ~i is just the ith row of the diagonalizing matrix T, it is clear that (~i' g) # 0, i.e., the element (Tg)i # which implies that ~i is controllable by the single-input terminal g.
EXERCISE
I. Establish Theorem 7.8 for multi-input systems. 7.8 OBSERVERS
We have noted that under the assumption of complete reachability, the poles of a system I: may be placed at arbitrary locations by means of a suitably chosen linear feedback law. There is, however, a basic assumption implicitly made in this result, namely, that the entire state vector x(t) is accessible for measurement at all times. This is clearly an unacceptable assumption in many physical problems and seriously diminishes the practical value of the pole-shifting theorem. In order to overcome the limitation of inaccessible states, the concept of an observer has been introduced. The basic idea is to replace the true, but unmeasurable, state x(t) by an estimated state ~(t), where ~(t) is constructed from the system output y(s), s :5: t, which, by definition, is totally at our disposal. The hope, which we shall justify in a moment, is that use of the estimated state ~(t) will provide the same amount of "information" as the true state x(t) (at least as far as stability properties are concerned). The justification for the above hope requires a detailed analysis using stochastic control theory. However, the following simple result provides a basis for the approach.
174
7
Theorem 7.9
STABILITY THEORY
Consider the system
x=
Fx + Gu, y(t) = Hx, ~ = Fx + L[y(t) - Hx] u(t) = - Kx(t).
+ Gu,
Then the characteristic polynomial ofthis system XT(Z) satisfies the relationship XT(Z)
=
XF-GK(Z)XF-LH(Z),
i.e.;the dynamical behavior ofthe entire system is the direct sum ofthe dynamics of the regulator loop (the matrix F - GK) and the dynamics of the estimation loop (the matrix F - LH). PROOF
X~z)
is the characteristic polynomial of the matrix [:H
Making the change of variable x we obtain the new system
F-~~~GKJ ~
x, X ~ x - X, which leaves XT invariant,
x = (F X=
- GK)x + GKx, (F - LH)x.
Since this is a triangular system, the result follows immediately. The above result suggest the following approach to stabilization of a completely controllable and completely ·constructible system ~: Pick a matrix K yielding a stable control law, i.e., XF-GK is a stability polynomial. Similarly, pick a matrix L yielding a stable state estimator, i.e., XF-LH is a stability polynomial. Define the system to be the system formed by the original dynamics plus the state estimator equations is in Theorem 7.6. Then the overall (closed-loop) system will be stable. In fact, we can actually obtain the same result as in the pole-shifting theorem since, by the controllability/constructibility hypothesis, the matrices K and L can be selected to place the overall system poles at any desired location. These results illustrate the importance of constructibility, as well as controllability, in stability analysis and regulator design. 7.9 STRUcruRAL STABILITY
As noted in the introductory section, an important feature of any dynamical system that purports to represent a real physical process is that small perturbations in the parameters of the model leave the "essential" features of the
7.9
175
STRUCTURAL STABILITY
model unchanged. Within the context of stability, a prime candidate for such an essential feature certainly is the asymptotic stability of the process, i.e., if the model is asymptotically stable, then .,nearby" systems should also possess this property. Since the borderline between asymptotically stable systems and unstable processes are systems whose dynamical matrix F possesses at least two purely imaginary characteristic roots, it is not surprising that such systems will play an essential role in formulating our main results on the structural stability of constant linear systems. First, however, let us consider certain notions of linear system equivalence which will make the job of establishing structural stability conditions particularly simple. Consider the free linear system
x=
Fx.
If h: R" -+ R ft is a one-to-one coordinate transformation (not necessarily linear), then we have the following definitions.
Definition 7.6 Two systems (I) x = Fx, (II) Y = Ay are said to be equivalent if the mapping h takes the vector function x(t) into the vector function y(t) for all t ~ O. Under these conditions, systems (I) and (II) are said to be: (a) linearly equivalent if h: R" -+ R" is a linear automorphism, i.e., h: x-+ T(t)x(t) = y(t), where T(t) are nonsingular matrices, t ~ 0; (b) differentially equivalent if h is a diffeomorphism; (c) topologically equivalent if h is a homeomorphism, i.e., h is 1-1, onto, and continuous in both directions. EXERCISES
1. Prove that linear equivalence implies differentiable equivalence which, in turn, implies topological equivalence. 2. Show that each type of equivalence indeed does define a true equivalence relation, i.e., it is reflexive, symmetric, and transitive.
We give several results about equivalence which, although they are of great importance, are not proved here as they are only stepping stones to our main questions about structural stability. The first is Theorem 7.10.
Theorem 7.10 Let the matrices F and A have simple (distinct) characteristic values. Then systems (I) and (II) are linearly equivalent if and only if the characteristic values of F and A coincide. REMARK
systems F
Simplicity of the characteristic values is essential as the two A = [A indicate. The next theorem shows that
= [A YJ and
n
7
176
STABILITY THEORY
for linear systems there is no need to distinguish between linear and differentiable equivalence. Theorem 7.11 The two systems (I) and (II) are differentiably equivalent ifand only if they are linearly equivalent.
Our final preliminary result forms the cornerstone for the main result on structural stability of linear systems. We consider the two linear systems (I) and (II), both of whose characteristic values have nonzero real parts. Let m_(F) be the number of characteristic roots of F with a negative real part, while m+(F) is the number with a positive real part, m_(F) + m+(F) = n. The central result on topological equivalence is given next. Theorem 7.12 A necessary and sufficient condition that (I) and (II) be topologically equivalent is that and REMARKS (1) This result asserts that stable nodes and foci are equivalent to each other but are not equivalent to a saddle. Thus see Fig. 7.9.
FIG.7.9
Relationship among stable nodes, foci, and saddles.
(2) The number m_ (or, of course, m+) is the unique topological invariant of a linear system. Armed with the preceding results, we return to the question of structural stability. Our goal is to determine conditions on F such that a continuous perturbation of sufficiently small magnitude leaves the qualitative features of the system trajectory invariant. Thus we ask that F and the new matrix F + eP = A have the same phase portraits for e sufficiently small and P an arbitrary, but fixed, perturbation matrix. However, Theorem 7.12 shows that the only system invariant under continuous transformations of R" is the number m _ . Thus F and A can have equivalent phase portraits if and only if m_(F) = m_(A),
which, since the characteristic roots of a matrix are continuous functions of the elements of the matrix, implies the next theorem. Theorem 7.13 The matrix F is structurally stable with respect to continuous deformations if and only ifF has no purely imaginary characteristic roots.
177
MISCELLANEOUS EXERCISES
REMARK It is important to note that the magnitude of the allowable perturbation (the size of B) depends on the root of F nearest the imaginary axis. In particular, this shows the importance of the pole-shifting theorem since, by suitable feedback, we can arrange for all characteristic roots of the closed-loop system to be far away from the imaginary axis. Such a system then exhibits a high degree of structural stability as comparatively large perturbations of the system do not alter the phase portrait.
MISCELLANEOUS EXERCISES
1. If the solutions of x = Fx are bounded as t -+ 00, show that the same is true of the solutions of x = (F + P(t))x provided that SO II pet) 1/ dt < 00. (Such systems are called almost constant iflimr _ 00 pet) = P, a constant matrix.) 2. (Floquet's Theorem) Show that the solution of the matrix equation
dXjdt
= P(t)X,
where pet) is periodic with period
1" and
X(t)
=
X(O) = I, continuous for all t, has the form
Q(t)eB ' ,
where B is a constant matrix and Q(t) has period 1". + alz"- 1 + ... + all be a polynomial with only simple zeros. Show that the zeros of p(z) are all real and negative if and only if
3. (Meerov Criteria) Let p(z) = aoz"
(a) the coefficients aj 2:: 0, i = 0, I, ... , n; (b) the Hurwicz determinants for jJ(z) = p(Z)2 + Zp'(Z2) are all positive. 4. Let fez) = [p(z) + (_I)"P( -z)]j[P(z) - (_I)"P( -z)]. Then p(z) is a stability polynomial if and only if (a) fez) is irreducible, (b) Re fez) > 0 when Re z > 0, (c) Re fez) < 0 when Re z < O. 5. Prove the following generalized stability theorem: The origin is stable for the system x = Fx if andonly ifRe lj(F) S; 0, i = I, ... , n and the elementary divisors ofF corresponding to those rootsfor which Re )"(F) = 0 are linear. 6. Show that the matrix
F = P-1(S - Q), where P, Q are arbitrary positive-definite matrices and S is an arbitrary skew-symmetric matrix, is always a stability matrix. Use this result to show that if F is stable, then F + A will also be a stability matrix if A is of the form A = P-1(So - Qo)' In other words, this gives a
178
7
STABILITY THEORY
sufficient condition for the sum of two stability matrices to again be a stability matrix. 7. Let p(z) = aoz n + alZn-1 + ... + an- 1z + an and let H; denote the nth Hurwicz determinant. If {Sk} are the roots of p(z), prove Orlando's formula
H;
= (_l)n(n+l)/2 a on 2 - n n (Sj
+ sd·
i5k
8. Define the matrix B = I + 2(F - I)-I. Show that F is a stability matrix if and only if B satisfies the condition Bk .... 0, k = 1,2,3, .... 9. Show that Re AiF) < - U, i = 1, ... , n, if and only if for every positive definite matrix C there is a unique positive definite Q such that F'Q +
QF
+ 2uQ = -c.
10. Let (F, G) be controllable and define
Show that if t 1 > 0, then all solutions of the linear system
x=
(F - GG'W- 1(0, t d )x
are bounded on [0, 00]. 11. Consider the linear system
x + P(t.)x = where p(t) = - p(- t) = p(t stable if
+
0,
1) ~ 0. Show that the solution is uniformly
12. (Output Feedback Stability) Let I: = (F, G, H) be a minimal realization of the transfer matrix Z(A.). Assume Re A.j(F) < 0, i = 1, ... , n, and suppose that for all real A. I - Z'( - iA)Z(iA)
Prove that if I - K'(t)K(t) ~ el >
°
x = (F -
~
0.
for all t > 0, then all solutions of
GK(t)H)x(t)
are bounded and approach zero as t .... 00.
179
NOTES AND REFERENCES NOTES AND REFERENCES
Sections 7.1-7.4 General references on stability theory both for classical, as well as controlled dynamical systems include Hahn, W., "Stability of Motion." Springer-Verlag, Berlin and New York, 1967. LaSalle, J., and Lefschetz, S., "Stability by Liapunov's Direct Method with Applications." Academic Press, New York, 1961. Bellman, R., "Stability Theory of Differential Equations." McGraw-Hill, New York, 1953. Barnett, S., and Storey, C., "Matrix Methods in Stability Theory." Nelson, London, 1970. Cesari, L., "Asymptotic Behavior and Stability Problems in Ordinary Differential Equations." Springer-Verlag, Berlin and New York, 1963.
For an extensive survey of the uses of Routh's method for the solution of a wide range of problems in stability and control, see Barnett, S. and Siljak, D., Routh's algorithm: A centennial survey, SIAM Review, 19,472-489 (1977).
For connection with realization theory, see Fuhrmann, P., On realization of linear systems and applications to some questions of stability, Math. Syst. Th., 8,132-141 (1974).
Section 7.5
The material of this section follows
Brockett, R., "Finite-Dimensional Linear Systems," Chap. 4. Wiley. New York, 1970.
For the original development of the circle theorem, see Popov, Y., Hyperstability and optimality of automatic systems with several control functions, Rev. Roumaine Sci. Tech. Sir. Electrotech. Enqerqet, 9, 629-690 (1964).
See also Naumov, B., and Tsypkin, Ya. Z., A frequency criterion for absolute process stability in nonlinear automatic control systems, Automat. Remote Control2S, 765-778 (1964). Aizerman, M., and Gantmacher, F., "Absolute Stability of Regulator Systems." Holden-Day, San Francisco, California, 1964.
An interesting use offrequency-domain ideas for the reduction of dimensionality is given in Lucas, T. and Davidson, A., Frequency-domain reduction of linear systems using schwarz approximation, Int. J. Control, 37,1167-1178 (1983).
Section 7.6 The first proof of the pole-shifting theorem (for more than a single input) was given by Wonham, W. M., On pole assignment in multi-input controllable linear systems, IEEE Trans. Automatic Control AC-12. 660-665 (1967).
A much simpler proof is found in Heymann, M., Comments on pole assignment in multi-input controllable linear systems, IEEE Trans. Automatic Control AC-13, 748-749 (\968).
180
7
STABILITY THEORY
The pole-shifting theorem raises the additional question of how many components of the system state actually need be measured in order to move the poles to desired locations. This problem has been termed the problem of" minimal control fields" and is treated by Casti, J., and Letov, A., Minimal control fields, J. Math. Anal. Appl. 43,15-25 (1973).
for nonlinear systems and in Casti, J., Minimal control fields and pole-shifting by linear feedback, Appl. Math. Comput, 2, 19-28 (1976).
for the linear case. Various refinements, extensions and generalizations of the basic Pole-Shifting Theorem continue to be of interest. For a sample of the recent results, see Armentano, Y., Eigenvalue placement for generalized linear systems, Syst. Controls Lett., 4, 199-202 (1984). Lee, E. B. and Lu, W., Coefficient assignability for linear systems with delays, IEEE Tran. Auto. Cant., AC-29, 1048-1052 (1984). Emre, E., and Khargonekar, P., Pole placement for linear systems over bezout domains, IEEE Tran. Auto. Cont., AC-29, 90-91 (1984).
Section 7.7
A detailed treatment of system stability utilizing the "modal" point of view is provided in the book by
Porter, B., and Crossley, R., "Modal Control: Theory and Applications." Barnes & Noble, New York, 1972.
The paper by Simon, J., and Mitter, S., A theory of modal control, Information and Control 13, 316--353 (1968).
should also be consulted for additional details. Section 7.8
The concept of an observer seems to first have been introduced
in the papers Luenberger, D., Observing the state of a linear system, IEEE Trans. Military Elect. MIL-8, 74-80 (1964). Luenberger, D., Observers for multivariable systems, IEEE Trans. Automatic Control AC-ll, 190--197(1966).
The results by Casti and Letov, cited under Section 7.6, also address the question of stabilization of ~ when not all components of the state may be measured. For a recent book-length treatment of the observer problem, see O'Reilly, J., "Observers for Linear Systems," Academic Press, New York, 1983.
NOTES AND REFERENCES
181
Section 7.9 An excellent introduction to the basic notions of structural stability may be found in the texts by Hirsch, M., and Smale, S., "Differential Equations, Dynamical Systems, and Linear Algebra." Academic Press, New York, 1974. Arnol'd, V., "Ordinary Differential Equations." MIT Press, Cambridge, Massachusetts, 1973.
Our approach in this section follows the latter source. More detailed results, requiring a high degree of mathematical sophistication, are given in the books by Nitecki, Z., "Differentiable Dynamics." MIT Press, Cambridge, Massachusetts, 1971. Peixoto, M. (ed.), "Dynamical Systems." Academic Press, New York, 1973.
The closely related topic ofcatastrophe theory is covered for the beginner by Zeeman, E. C., Catastrophe theory, Sci. Amer. (April 1976). Amson, J. C.; Catastrophe theory: A contribution to the study of urban systems? Environment and Planning-B2, 177-221 (1975).
For a more advanced (and philosophical) treatment, the already classic work is Thorn, R., "Structural Stability and Morphogenesis." Addison-Wesley, Reading, Massachusetts, 1975.
An application of structural stability notions directly to the stability issue for linear systems is given in Bumby, R. and Sontag, E., Stabilization ofpolynomially parametrized familiesoflinear systems: The single-input case, Syst. Controls Lett., 3, 251-254 (1983).
CHAPTER
8
The Linear-Quadratie-Gaussian Problem
8.1 MOTIVATION AND EXAMPLES
At long last we are ready to turn our attention to questions of optimal selection of inputs. To this point, our concerns have been with the selection of inputs either to reach certain positions in state space (controllability) or to make the controlled system stable in some sense. Once these issues have been settled for a given system, or class of systems, and it has been determined that the set of admissible inputs n is large enough to include more than one control which will ensure stability and/or reachability, we are then faced with choosing a rationale for preferring one such input over another. It is at this point that we finally turn to optimal control theory for resolution of the dilemma. Generally speaking, the system input is chosen in order to minimize an integral criteria of system quality. For example, if we desire to transfer the system to a terminal state in minimal time, the criterion J =
f~
ds,
(8.1)
subject to the constraints x(O) = Cl' x(t) = C2 might be used, with Cl and C2 the initial and terminal states, respectively. This is an example of a so-called free time problem since the duration ofthe process is not specified in advance. Another type of problem arises when we have a quality measure g(x(t), u(t), r) specifying the cost at time t of the system being in state x(t) when the control 182
8.1
183
MOTIVATION AND EXAMPLES
u(t) is being applied. If the process duration is specified to be oflength T - to, then an appropriate criterion would be to choose u(t) to minimize J
fT g(x(t), u(t), r) dt.
=
(8.2)
10
To ensure that such a process does not terminate in an unfavorable final state x(T), criterion (8.2) is often augmented by the addition of a terminal cost h(x(T)), measuring the desirability of the final state x(T). A stricter requirement is to demand that the system terminate in a predetermined final state x(T) = d. Such a situation is encountered in various types of navigation processes. Other variations on the basic theme include the imposition of state and/or control constraints of either a local or global nature. Examples of the former are
i
= 1, ... , m,
j
= 1, ... , n,
while the latter are exemplified by forms such as
f
Tll u(S)1I ds
:0:;
M.
10
In this book we only briefly touch on variations of the above sort, since our primary interest is in the determination of the implications of linear dynamics for problems in which the cost is measured in a Euclidean norm, i.e., the costs are quadratic functions of the state and control. Thus we will be investigating the problem of minimizing the quadratic form J(to) =
fT [(x, Q(s)x) + 2(x, S(s)u) + (u, R(s)u)] ds
(8.3)
10
over all admissible input functions u(t), to are connected by the linear system
x == F(t)x + G(t)u,
:0:; t :0:;
T. It is assumed that x and u
x(t o) = c.
(8.4)
The matrices Q and R are assumed to be symmetric with further conditions to be imposed later in order to ensure the existence of an optimal u. The foregoing formulation is the so-called linear-quadratic-Gaussian (LQG) problem which is the focus of our attention throughout this chapter. It arises in a large number of areas of engineering, aerospace, and economics, as well as in situations in which initially nonlinear dynamics (and or nonquadratic costs) are linearized (quadraticized) about a nominal control and corresponding state trajectory.
8
184
THE L1NEAR-QUADRATIC-GAUSSIAN PROBLEM
An important version ofthe LQG problem is the so-called output regulation problem in which the system dynamics are given by Eq. (8.4), while the output is y(t) = H(t)x(t). (8.5) It is desired to minimize (over u) the quadratic functional J(t o) =
iT
[(y, Cy)
+ 2(y, Vu) + (u, Ru)] ds
(8.6)
to
with C > O. Clearly, the output regulation problem is equivalent to problems (8.3)-(8.4) with the identifications
Q -+ H'CH,
S -+ H'V,
R
-+
R.
Thus we shall usually consider criterion (8.3)in this chapter.
Example:
Water Quality Control
A typical sort of problem in which LQG theory arises is in the regulation of water quality in a river. To avoid complications, we assume that the river may be decoupled into k nonoverlapping reaches in such a way that the biochemical oxygen demand (BOD) and dissolved oxygen (DO) do not change with respect to the spatial distance downstream from a given reference point, i.e., the BOD and DO dynamics for a given reach involve only the single independent variable t, the time. We further assume that the reach is defined as being a stretch of the river in which there is at most a single water treatment facility of some kind. With the above assumptions, the lumped BOD-DO dynamics are described by db(t)/dt
= - Kyb(t),
which characterizes the pollution situation in the river. Here b(t), y(t) are k-dimensional vector functions representing the BOD and DO concentrations, respectively, in each reach, while K; is the BOD removal coefficient matrix, K d the deoxygenation coefficient matrix, K. the reaeration coefficient matrix, and ds the saturation level of DO, all assumed constant. The effects of adding effluents to the river have not yet been taken into account. This is accomplished by defining control vectors Ul(t) and U2(t), where Ul(t) is a k-dimensional vector representing the control of effluents by sewage treatment plants, while U2(t) is a k-dimensional vector indicating control by artificial aeration carried out along the reaches. For example, the first control might be the operation rule for a retention reservoir located after the treatment plant, while the second could be the timing schedule for the aeration brushes.
8.2
185
OPEN-LOOP SOLUTIONS
To complete the model, we define state variables and a performance measure. Since there are certain water quality standards to be satisfied during the control periods, we assume that the controlling actions are taken to minimize BOD and DO deviation from these standards. Assume that the standards are given by the constant vectors aD and aD' Then we define the state x(t) =
[X 1(t)J = X2(t)
[b(t) -
aDJ.
y(t) - aD
Thus the complete system dynamics are x(t)
=
Fx(t)
+ Gu(t),
where x(t) is as above,
since the greater the artificial aeration, the less is the oxygen deficit, and conversely. The cost function is to minimize a weighted sum of state deviations from zero, and cost of controls. Hence, we minimize J = f[(X(t), Qx(t))
+
(u(t), Ru(t))]
+
(x(T), Mx(T)),
where Q and R represent appropriate weighting matrices reflecting the relative importance of BOD and DO control in each and the relative costs of sewage treatment and aeration. The terminal cost matrix M accounts for the relative importance ofthe water quality level at the termination ofthe process. 8.2 OPEN-LOOP SOLUTIONS
We have noted earlier the fundamental conceptual (and mathematical) difference between open-IMp and closed-loop (feedback) inputs. Therefore, it should come as no surprise that the basic results associated with the LQG problem also inherit the flavor. We shall begin with a discussion of the openloop situation. It will be seen that the basic results characterizing the optimal input require the solution of a Fredholm integral equation in order to generate the optimal control. Alternatively, utilization of the Pontryagin maximum principle yields the optimal control as a function of the solution of a two-point boundary value problem. In either case, formidable computational problems may arise serving, among other reasons, to motivate a thorough study of the feedback case.
8
186
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
At first, we consider the problem of minimizing
f.T [(x, Q(t)x) + (u, R(t)u)] dt,
J =
(8.7)
10
where
+ G(t)u,
dxjdt = F(t)x
(8.8)
The idea of our approach is to express the cost functional entirely in terms of the control u( .), from which we may employ a standard completion-of-squares argument to find the minimum, as well as conditions on R(t) and Q(t) for which a minimum exists. We write the solution of the differential equation as x(t)
= (t, tok +
f.T (t, s)G(s)u(s) ds,
(8.9)
10
where is the fundamental solution matrix of F. Substituting expression (8.9) for x(·) into the cost functional (8.7) and performing some algebra, we arrive at the expression J
= (c, Me)
- 2
f.T(U' m(s» ds 10
+
f.Tf.T(U. [R(s) b(s 10
10
s')
+ K(s, s')]u) ds ds',
.
where
M m(s)
=
f.T '(t o, s)Q(s)(s, to) ds, '0
= -
f.: I(t -
G'(s)
s)'(t, s)Q(t)(t, tok dt,
10
K(s, s') = G'(s)
f.T I(t -
s)'(t, s)Q(t)(t, s')l(t - s') dt G(s),
10
with 1(·) being the unit step function, i.e.,
I(a) = {I, 0,
a ~ 0, a < o.
It is now easy to see that there will be a unique square-integrable u(·) minimizing J if and only if %(s, s')
=
R(s)b(s - s')
+ K(s, s') >
0,
(8.10)
8.3
187
THE MAXIMUM PRINCIPLE
i.e., res, s') is strictly positive definite on the square to ~ s, s' ~ T. Furthermore, if % > 0, then from standard results in the theory of integral equations we find that the minimizing u( .) will be the unique square-integrable solution of the Fredholm integral equation R(t)u(t)
+
iT
K(t, s)u(s) ds
=
met),
(8.11 )
10
The foregoing derivation has been rather formal, proceeding under the assumption that F, G, Q, and R possess all properties necessary to ensure the square-integrability of K(·, .) plus the positive-definiteness of %(', -], But, what are these conditions? The square integrability of K(·, .) can be ensured by assuming that the matrix functions F(t) and Q(t) are piecewise continuous on the interval to ~ t ~ T. Equation (8.11) shows immediately that the uniqueness of the optimal control will be lost (in general) if R is singular. Thus we provisionaIly assume that R is nonsingular. However, to guarantee that condition (8.10) is satisfied, we should strengthen the condition on R to be R(s) > 0, to ~ s ~ T. This ensures that the first term of (8.10) is positive definite. FinaIly, a sufficient condition for the second term to be positive semidefinite is for Q(s) ~ 0, to ~ s ~ T. Notice, however, that %(', .) may stiIl be positive definite even if Q is negative definite, or even indefinite. In summary, we have established Theorem 8.1. Theorem 8.1 Let the matrices Q(s) ~ 0, R(s) > 0, F(s), and G(s) be piecewise continuous over to ~ s ~ T. Then the functional J has a unique minimum over all square-integrable functions u(s), to ~ s ~ T, and this minimum is given by the solution to the Fredholm integral equation (8.11). EXERCISE
1.
How can the above results be modified to account for (a) a terminal cost of the form (x(T), Q/x(T», (b) a cross-coupling term in J of the form 2(x, Su). 8.3 mE MAXIMUM PRINCIPLE
FoIlowing in the footsteps of Weierstrass, Legendre, and other pioneering 19th century workers in the calculus of variations, an alternate approach to the characterization of the optimal open-loop control law for the LQG problem was developed by Pontryagin, Boltyanskii, Gamkrelidze, and Mischenko in the mid-1950s. This result, termed the "maximum principle," is a substantial generalization of the classical Weierstrass condition from the calculus of variations. In essence, the maximum principle states that the
8 THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
188
optimizing control law must provide the pointwise minimization of (the Pontryagin group worked with the negative of our function J) a certain function, the Hamiltonian of the system, which is determined solely by the given system data F, G, Q, R, S. The basic idea behind the maximum principle is to augment the integral of (8.3) by adding the system dynamics (8.4), multiplied by an unknown vector function p(r), the so-called costate, which is to be determined. Thus we form the Hamiltonian Jf of the system as ,ff(x, u, p, t)
= [(x, Qx) + 2(x, Su) + (u, Ru)] + p'(t)[Fx + Gu], (8.12)
where the prime denotes, as usual, the transpose operation. The content of the maximum principle is given next. Theorem 8.2 (Weak Minimum Principle) Assume Q(s) ~ 0, R(s) > 0, to :::; s :::; T. Let u*(t) be the input which minimizes criterion (8.3), subject to the dynamics (8.4). Then the corresponding optimal trajectories x*(t), p*(t) satisfy the equations
aYf
u", p*, r),
x*(t)
=
p*(t)
= - a: (x*, u*, p", r),
ap (x",
(8.13)
(8.14)
Furthermore, u*(t) is characterized by the condition
a:ft au (* x, u '", p *) , t = 0.
(8.15)
The boundary conditions are x*(ro)
= c,
(8.16)
p*(T) = O.
In more explicit terms, Eqs. (8.13)-(8.14) take on the form x*(t)
= F(t)x* + G(t)u*,
p*(t) = -2Q(t)x*
(8.17)
+ 2S(t)u*
- F'(t)p*,
(8.18)
with the minimizing u being characterized through Eq. (8.15) in terms of p* and x* as u*(t)
= -
R - l(t) [tG'(t)p*
+ S'(t)x*].
(8.19)
Thus we may rewrite Eq. (8.18) as p*(t) = -2(Q
+ SR-1S')x*
- (SR-1G
+ F')p*,
(8.20)
8.3
189
THE MAXIMUM PRINCIPLE
while Eq. (8.17) is
x*(t)
=
(F - GR-IS')x* - tGR-IG'p*.
(8.21)
The boundary conditions are those of Eq. (8.16). Consider the problem of minimizing
EXAMPLE
J = fT[x/(t)
+ u 2(t)] dt
to
with
xl(t) = X2(t), In this problem, we have
F=[~
-~J
G=[~J
S = 0,
The Hamiltonian is Jr(x*, u*, p*, t) = x/ + u 2 Eq. (8.18), we find the costate equations are
PI *(t)
=-
o·1fIi3xl
=-2x
I*(t),
+ PIX2 -
P2 *(t) = - a·1flox2
R = 1.
P2X2 + P2U. For
=
P2*(t) - PI *(t).
The control u* must satisfy
OJr ou = 2u*
+ P2 * =
Since
0,
i.e.,
u*(t) = -tp2*(t).
0 2JrIi3u2 = 2 > 0,
we see that u" = -tp2*(t) does indeed provide the minimizing value of.1f. In the general theory of control processes, the importance of the maximum principle is that it applies even to those cases in which the input space n of admissible controls is constrained. For example, if we had demanded
Iu(t) I ::;
1,
to ::; t
s
T,
in the above example, then, the minimizing control law would still minimize the Hamiltonian .1f pointwise, subject to the constraint. EXERCISES
1. Show that if'[uirlj s; l,t o ::; t s; T,intheaboveexample,theminimizing control law would be u*(t) =
{
- t P2*(t) -1
+1
if IP2 *(t)l s 2, if P2 *(t) > 2, if P2*(t) < 2.
8
190
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
2. Draw a graph of the optimal control law u*(t). 3. Can the optimal control law for the constrained problem be determined by calculating the control law for the unconstrained case and allowing it to saturate whenever the stipulated boundaries are violated? 8.4 SOME COMPUTATIONAL CONSIDERATIONS
In principle, Eqs. (8.20)-(8.21), together with the boundary conditions (8.16), provide sufficient information to compute x* and p* which in turn allow us to calculate the optimal control u* by means of the minimum condition (8.19). However, the fact that (8.20)-(8.21) constitutes a two-point boundary value problem poses a nontrivial set of computational questions. The basic problem is that conditions (8.16) prescribe a value of x* at t = to, while the value for p" is given at t = T. As a result, there is not enough information at any single point to serve as initial conditions for calculating x* and p* in a recursive manner. The linearity of Eqs. (8.20)-(8.21) may be exploited to aid the computational process in the following manner. Let X H(t), PH(t) denote the solution to the homogeneous matrix system XH(t)
=
(F - GR-1S')XH
PH(t) = -2(Q
+
iGR-1G'PH,
-
SR-1S')X H - (SR-1G
+
(8.22)
F')PH ,
(8.23)
with the initial conditions Our objective is to use system (8.22)-(8.23) to determine the unknown value (X = p*(to) which, when used in Eqs. (8.20)-(8.21), will make p*(T) = O. In addition to the homogeneous matrix system (8.22)-(8.23), we must also generate one particular solution to the vector system (8.20)-(8.21). If we let x,; P« denote this solution, then X" = (F - GR-1S')x" - iGR-1G'p",
p" =
-2(Q
+ SR-1S')x"
- (SR-1G
(8.24)
+ F')p",
(8.25)
with x,,(to) = c,
p,,(to) = O.
Using the superposition principle for linear systems, we see that the complete solution of (8.20)-(8.21) is expressed by
+ (X2 xif'(t) + (XIPU'(t) + (X2pif)(t) +
x*(t} = (XtxU)(t) p*(t) =
+ lXlIX!i'(t) + x,,(t), + (XlIPW'(t) + p,,(t),
(8.26)
(8.27)
8.4
SOME COMPUTATIONAL CONSIDERATIONS
191
where the values of (Xl' ••• ' (X" of the vector (X are to be determined and xW, ~) represent the ith column at the matrices XH, PH' respectively, i = I, ... , n. Using Eq. (8.27), we see that it may be written in vector matrix form as
(8.28) where PH(t) is the n x n matrix whose ith column is PU'(t). Since we must have p*(T) = 0, from (8.28) we obtain PH(T)(X
=
-Prr(T),
or (X = Pii l(T)Prr(T).
(8.29)
Assuming that PH(T) is invertible, the value of (X from (8.29) provides the "missing" initial value for p*{to), thereby turning (8.20)-(8.21) into an initial value problem soluble by standard methods. Reexamining the foregoing procedure, we see that the computational requirements are to integrate 2n homogeneous equations and 2n particular solutions from t = to to t = T, then solve the n x n linear algebraic system (8.28). On the surface the above approach would appear to dispose neatly of the problem of determining x* and p*. However, in practice serious difficulties may arise: (i) Ifthe interval length T - to is large, then it may be difficult to compute accurately the homogeneous and particular solutions. In fact, the situation is often far worse than just a case of numerical roundoff or truncation error. This is due to the fact that we may determine the functions X H, PH' x"' Prr by either integrating forward from t = to to t = T as described above, or by integrating backwards beginning at t = T and determining the unknown value x*(T) rather than p*(O). In either case, due to the nature of the system (8.20)-(8.21), one of the equations will be integrated in an unstable direction. This is a consequence of the fact that the equation for p* is "dual" to that for x*. Thus, if the forward direction is stable for x* (i.e., x*(t) is a linear combination of decreasing exponential functions), then it is unstable for p*, and conversely, if the forward direction is stable for p*, it is unstable for x*. In either case, if the interval length is sufficiently large, numerical inaccuracies are guaranteed to appear. (ii) Even if the homogeneous and particular solutions are produced with great accuracy, it may turn out to be difficult to solve the linear algebraic system (8.28), particularly if the dimension n of the system is large. Many times the problem is theoretically solvable in that PH(T) is invertible but, for practical purposes, the problem is out of reach due to ill-conditioning. This phenomenon may produce a value of (X generating a p*(T) far from zero.
192
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
Various procedures and techniques have been proposed to circumvent the above difficulties and are described in detail in the references cited at the end of the chapter for this section. We only mention these points as motivation for development of an alternate conceptual approach to the LQG problem. This approach, utilizing the notion of closed-loop or feedback controls, will be explored in succeeding sections. EXERCISES
1. (a) Find the solution of the system x(t) = -2x(t) - p(t), P
+ 3p(t),
x(O) = 3,
P(l)
= O.
(b) What is the LQG problem associated with this system? 2. Prove that if T is fixed and if the Hamiltonian Yf does not depend explicitly on t, then Yf is constant when evaluated along an extremal trajectory, i.e., .tt(x*(t), u*(t), p*(t»
=
k,
to ~ t ~ T.
3. How would the maximum principle be modified to include linear state and/or control variable constraints of the form Ax u(t)
+ By E
~
U(t),
0,
A and B constant matrices, to s t s T?
(Hint: Transform all inequality constraints into equality constraints; then incorporate the new constraints into the objective function.) 8.S FEEDBACK SOLUTIONS
We now pass from open-loop solutions, the maximum principle, and twopoint boundary value problems to closed-loop solutions, matrix Riccati equations, and initial value problems. The development we present in this section represents a viewpoint toward the LQG problem that was motivated principally by computational considerations and the widespread availability of digital computers capable of numerically integrating large systems of ordinary differential equations of the initial value type, i.e., with all data specified at a single point. Thus the closed-loop or feedback concept departs from the classical variational calculus in that we now desire to characterize the optimal control u*(t) as a function of the system state x. Thus u* = u*(x). In fact, as we shall see u*(t) = - K(t)x(t),
8.5
193
FEEDBACK SOLUTIONS
i.e., u* is a linear function of the state x, where K is a matrix function dependent on the problem data. The importance of the feedback solution is not immediately apparent in the LQG problem as sketched in earlier sections since, in principle, it yields the same result as the open-loop solution obtained from the maximum principle. The gain is primarily operational as the feedback approach generates a characterization of u* that is readily computable and that easily extends to more general situations in which stochastic effects are present. In addition, a feedback point of view makes it particularly easy to study various questions pertaining to the stability of the optimal closed-loop systems, system identification, and sensitivity analysis. We recall the basic problem. It is desired to minimize the functional J =
i~
+ 2(x, Su) + (u, Ru)] ds
[(x, Qx)
(8.30)
over all piecewise-continuous vector functions u, where x and u are related through the linear system dxlds = Fx
+ Gu,
x(t o) = c.
(8.31)
Here, F, G, Q, R, S are continuous matrix functions of appropriate sizes with Q and R being symmetric. The following lemma will be useful.
Lemma 8.1 Let F, G, P be given matrix functions and suppose that Pet) exists on to :$;; t :$;; T. Then if x = F(t)x + G(t)u, we have
i
T [(x,
PGu) + (x, GPu) + (x, [Pet)
10
- (x(t), P(t)x(t)) PROOF
i~
C:
+ F'(t)P(t) + P(t)F(t)], x)] dt
= 0
For any differentiable trajectory x(t), we have
[(x, p(t)x)
+ (x, P(t)x) + (x, P(t)x)] dt
Substitution of F(t)x + G(t)u for
- (x, P(t)x)
C:
=
o.
x completes the proof.
The basic result for closed-loop control in the LQG problem is
Theorem 8.3 Let R(t) > 0 and let the matrix Riccati differential equation - Pet) = Q(t) + PF(t) peT) = 0,
+ F'(t)P -
(PG(t) + S(t))R -l(pG(t)
+ Set))',
(8.32) (8.33)
8
194
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
have a solution on to ~ t ~ T. Then there exists a control u that minimizes the functional J of Eq. (8.30), under constraint (8.31). The minimizing control u*(t) is given by u*(t) = - R -l(tHG'(t)P(t) = - K(t)x(t),
+ S'(t»x(t)
(8.34)
and the minimum value of J is
(8.35)
J*(t o) = (c, P(tok).
x=
PROOF Since which gives
i~
J =
+
[(x, Q(t)x)
i~[(X'
Fx
+ Gu,
we add the identity of Lemma 8.1 to J,
+ 2(x, S(t)u) + (u, R(t)u)]
PGu)
+ (u, G'Px) +
(x, (1'
dt
+ PF + F'P)x)] dt -
(x, Px)
c:.
Combining these two integrals and using Eq. (8.32) for P, we obtain J = fT[(X,(PG
+ S)R- 1(PG + S)'x) + (x,(PG + S)u) + (u,(PG + S)'x)
to
+ (u, Ru)] dt + (c, P(to)c), =
i~I R1/2U
+ R- 1 / 2 (G'p
~
S')x11 2 dt
+ (c, P(to)c).
The last expression clearly indicates the optimal control (8.34) and optimal cost (8.35). EXERCISE
1. Show that addition of the terminal condition (x(T), Mx(T» to the criterion J may be accounted for in Theorem 8.3 by modifying Eq. (33) to read" peT) = M." Computation of the optimal control law using Theorem 8.3 proceeds in a two-sweep fashion: Eq. (8.32) for pet) is integrated from t = T to t = to, either storing the values of the matrices pet) or just determining the value P(t o). In the latter case, Eqs. (8.32) and (8.31) are integrated from t = to to t = T, using the initial values P(t o) and c, while the optimal feedback control law is computed from Eq. (8.34). If the values of pet) have been stored, then only Eq. (8.31) need be integrated in the second sweep.
8.5
195
FEEDBACK SOLUTIONS
If the state space has dimension X = n, the above procedures require the integration of n(n + 1)/2 equations for P (P is symmetric) and n equations for x, a total of (nZ + 3n)/2 equations. In a later section, we shall show how more careful attention to the input and output spaces may often significantly . reduce this number.
Example:
Arms Races
The LQG problem may be employed to analyze various models for the developments of arms races between competing nations. A simple version of such a model is provided by considering two competing nations. Letting N t(t), N 2(t) represent the armaments levels of the two sides at time t. A plausible set of dynamics describing the change of these levels is given by Richardson's equations
dNtldt = kzN z - aNt + g + ut(t), dNz/dt = IN t - bN z
+ h + uz(t),
(8.36) (8.37)
where the constants k, I are defense coefficients, a, b fatigue coefficients, and
g, h grievance coefficients when positive and goodwill coefficients when
negative. The functions Ul(t) and uz(t) represent the armaments policies of each side. Under the assumption that each side wishes simultaneously to maintain a high level of armaments but at minimal expense, the criterion of minimizing
f
T«(I, l N t Z + (l,2Nz2
+ .PIUtZ + Pzu2Z)dt
(8.38)
10
is introduced, where (l,t, (l,z are nonnegative constants expressing the relative importance each side attaches to armaments levels, while Pt, pz are positive constants reflecting the weights each side attaches to defense expenditures. It is fairly easy to see how the foregoing model may be extended to a case of n interacting nations. Unfortunately, Richardson's model says nothing about whether arms races lead to war. A more general theory that would account for relationships between arms buildups and the occurrence of conflicts is needed. However, Richardson's model may be used as an earlywarning device to inform nations when their arms buildup has gone beyond the bounds of a certain threshold which points to danger if the buildup continues. EXERCISES
1. Formulate the arms race problem in LQG terms, i.e., determine the relevant matrices F, G, Q, R, S for problem (8.36)-(8.38).
196
8
THE LlNEAR-QUADRATIC-GAUSSIAN PROBLEM
2. (a) Using the parameter values k = 2, a = 1, g = 0, 1= 2, b = 1, h = 0, N1(0) = N 2(0) = om, OCt = OC2 = 1, /31 = /32 = 2, numerically determine the optimal feedback armament policies for each side (this case corresponds to perfectly symmetric opponents having no grievances toward each other). (b) Consider other cases of nonsymmetric opponents with and without grievances. 3. Solve Exercise 2 for the optimal open-loop policies by employing the maximum principle results of Section 8.4. 4. Investigate model (8.36)-(8.37) (and its higher-dimensional analog) from the point of view of stability, asymptotic and structural. 8.6 GENERALIZED X-V FUNCTIONS
We have seen that the feedback solution of the LQG problem in all essential aspects depends on the solution to the matrix Riccati differential equation -dP/dt = Q P(T) = 0,
+ PF + F'P
- (PG
+ S)R-t(PG + S)',
(8.39) (8.40)
for the problem when no terminal cost is assigned (M = 0). As noted above, a slightly puzzling feature about Eq. (8.30) is that no cognizance is taken of the system input and output spaces insofar as the computational burden is concerned. Single-input/single-output systems represent the same computational problem as full systems (m = p = n). It is reasonable to inquire as to whether this is inherent in the nature of the LQG problem and, if not, in what manner may the input and output spaces be utilized to reduce the computational task posed by Eq. (8.39). In this section we shall show that the LQG problem does indeed possess additional structure that may be exploited to simplify the computation of the optimal feedback control. This reduction will be accomplished by replacing Eq. (8.39) by a set- of coupled nonlinear matrix equations, termed "generalized X - Y" functions, the terminology being borrowed from the theory of radiative transport in the atmosphere. It will be seen that the generalized X - Y system constitutes a set of n(p + m) equations for the determination of the optimal feedback control law, where n, p, m are the dimensions of the system state, output, and input spaces, respectively. Thus, if the number of inputs and outputs is small compared with n, the generalized X- Y system will represent a fewer number of equations than Eq. (8.39). We will present some numerical evidence to indicate the relative magnitude of the computational gain.
8.6
197
GENERALIZED X-Y FUNCTIONS
At first, we assume the system matrices F, G, Q, R, S are constant. Under these assumptions, we state the next theorem. Theorem 8.4
Assume that the following rank conditions are satisfied:
(i) rank(SR - 1 S' - Q) = p ; (ii) rank(GR- IG') = m, and, as a result, that we have the factorization ZZ' = SR - 1S' - Q, where Z is an n x p matrix. Then the solution of the matrix Riccati equation (8.39) has the representation P(t) [GR -1 S - F]
+ [SR - 1 G' -
F']P(t)
= L(t)L'(t)
- N'(t)N(t)
-(SR- 1S' - Q),
(8.41)
where L(t) and N(t) satisfy the equations dLjdt
= - [F' - SR -IG' - N'R -1/2G']L(t),
L(T) = Z,
N(T) = O.
dN jdt = R -1/2G'L(t)L'(t),
Further, the optimal feedback law K(t) through the function N(t) as K(t) = R- 1/2N(t)
(8.42a) (8.42b)
= R-1(G'P(t) + S) is expressed
+ R-1S',
(8.43)
and
(8.44)
dPjdt = L(t)L'(t). PROOF
We may rewrite the Riccati equation (8.39) as dP dt
= (SR-1S _ Q) + P(GR-1S' - F) + (SR- 1G' - F')P
+
PGR-1G'P,
P(T) =0.
Differentiating the above equations with respect to t yields the homogeneous matrix equation p(t) = P[GR-1S' - F + GR-1G'P] + [SR- 1G' - F' + PGR- 1G']P (8.45) with the initial condition P(T)
=
SR- 1S' - Q.
Let U(t) be the solution of the matrix equation dUjdt = [SR- 1G' - F' + PGR- 1G']U,
(8.46)
U(T)
= I.
198
8
THE LlNEAR-QUADRATIC-GAUSSIAN PROBLEM
Then it is easily verified that P(t) = U(t)[SR-1S' - Q]U'(t),
which, by the rank assumption (i), gives P(t) = U(t)ZZ'U'(t),
t :s; T.
Making the definitions L(t) = U(t)Z,
N(t) = R - 1/2G' P(t),
Eqs. (8.42) immediately follow. The algebraic relation (8.41) is a direct consequence of the definitions of Land N, together with the original Riccati equation (8.39). REMARKS (I) Since G is an n x m matrix, the rank condition (ii) is always satisfied if G is of full column rank. Also, if cross-terms are absent from the cost functional (S = 0), then the rank condition (i) is equivalent to restating the problem as one of minimizing the sum of squares of the system output (y, y), with y = Hx, i.e., Q = (iH')(iH), with Hap x n matrix. Thus the L-N system constitutes n(p + m) equations which, if p + m < (n + 1)/2, is a smaller number than that represented by the Riccati equation (8.39). Furthermore, the optimal feedback law is easily realized through N(t) by relation (8.43). (2) Relation (8.44) may be used to compute the matrix P(t) at any point t by means of a quadrature, i.e.,
P(t)
= -
iTL(s)L'(s) ds.
Alternatively, the algebraic expression (8.41) may be inverted for P(t) once L(t) and N(t) have been computed. Example:
Riccati versus Generalized X-Y System
To test the efficiency of the generalized X - Y system, the following numerical experiment was performed. The system matrices S = 0, 0 0
1 0 0 1
0 -an
0 0 -an- 1
R
G = [0, 0 ... 0 1T,
= I, 0 0
F=
Q-al
-[},q.J
8.6
199
GENERALIZED X - Y FUNCTIONS
were chosen, where the numbers ql,"" q" were randomly selected, the constants aI' ... , a" were chosen so that F would have prescribed characteristic roots, and n was chosen to equal 16.Thus in the Riccati system there were 136equations for the components of P, while in the generalized X- Y system we had 16(1 + 1) = 32 equations. Both systems were integrated over the interval to = 0 ::5: t ::5: T = 1, using a Runge-Kutta integration scheme with stepsize L\ = 0.02. The computing times to achieve four significant figure accuracy are shown in Table 8.1. TABLE 8.1 Computing Times for Three Sample Cases Computing time (sec) Generalized X- Y system
Riccati system
-0.5, -0.8, I, I ± i, 2,2 ± i, -3,3 ± i, -4 4 ± 2i,5, -6
0.561
4.150
-0.5, -0.8, I, -1.4, I ± i, 1.5,2 ± i,5, -6, -8, -8, -8, -8,9.5
0.589
4.156
-0.5, -0.8, I, I ± i, 2, 2 ± t. -3, -4, -5, -6. -7,8, 0.1 ± 3.3i
0.580
4.151
Roots of F
The important point to note about Table 8.1 is that although a computational improvement of about 4! was to be expected solely on an equationcounting basis, the actual time ratio was greater than a factor of seven. A count of operations required to integrate the two systems accounts for this apparent discrepancy. (We note, in passing, that the times were not identical for each case with a fixed method due to small time-sharing losses in the CDC Cyber 74 network used for the calculations.)
EXERCISES
1. Extend the generalized X- Y functions to cover the case of terminal costs. 2. Derive the generalized X- Y functions for the case of the discrete-time LQG problem.
200
8
THE LINEAR-QUADRATlC-GAUSSIAN PROBLEM
3. Consider the classical calculus of variations problem min
JT [(x, Ax) + (x, Ex)] dt; 10
where the minimum is over all continuous vector functions x(t), to ~ t ~ T, such that x(t o) = c. What possibilities do the generalized X - Y functions have for this case? The extension of the generalized X - Y functions to the time-dependent case requires some new ideas since the" differentiation trick" employed in the proof of Theorem 8.4 is of no particular avail. To this end, we introduce the (n + m) x (n + m) matrix function M(P) = [F'(t)P(t) + P(t)F(t) + Q(t) G'(t)P(t) + S(t)
P(t)G(t) + S(t)J. R(t)
(8.47)
Our approach will be to factor the matrix M(P) into its symmetric triangular factors, then relate the resulting factors to the generalized X - Y functions given above when the system matrices are constant. In the time-varying case we will not obtain explicit relations for the low-dimensional functions but rather, we will exploit redundant (low rank) structure in the problem to reduce the computational burden presented by the triangular factorization scheme. Upon carrying out the factorization of M(P), we have lp [o
rJ[lp'I" l/J'OJ
l/J
= [F'P
+ PF + Q PG +
'G'P
SJ
+SR'
This implies the relations lplp' + I'f" = F'P + PF + Q,
l/Jr'
= G'P
(8.48)
+ S',
(8.49)
l/Jl/J'=R.
(8.50)
Formulas (8.48)-(8.50) hold' pointwise for any fixed symmetric matrix P(t), to ~ t ~ T. Relations (8.49) and (8.50) immediately give R 1 / 2 (t)V(t )r(t)
= G'P + S',
where V(t) is an arbitrary orthogonal, matrix which, for convenience, we choose equal to the identity. Since R is assumed invertible for all t, Eq. (8.48) shows that lp(t) satisfies lp(t)lp'(t) = Q + F' P + PF - (PG + S)R- l(PG +
Sr.
201
8.6 UENERALIZED X - Y FUNCTIONS
Recalling that the optimal feedback matrix K = R-1(G'P K(t)
+ S'), we see that
= R - 1/2r'(t),
(8.51 ) (8.52)
- p(t) = qJ(t)q/(t).
Formulas (8.51) and (8.52) show the basic connection between the factors qJ and r appearing in the triangular factorization of M, and the basic quantities K and P characterizing the solution to the LQG problem. We emphasize that these results hold pointwise for time-varying F, G, Q, R, and S. To see the relationship between the factors r, tp, and e and the generalized X - Y functions Land N, we compare Eqs. (8.51 )-(8.52) with Eqs. (8.43)(8.44), assuming the system matrices are constant. We finally obtain the following theorem. Theorem 8.5 If F, G, Q, R, S are constant matrices, then the factors rand qJ in the triangular decomposition of M are related to the functions ofLand N of Theorem 8.4 by
(8.53)
tp = iLU',
I"
= T'N + T'R- 1/2S',
(8.54)
where U(t) and T(t) are arbitrary p x p, m x m orthogonal matrices, respectively. PROOF
Since p(t) is symmetric, there exists an orthogonal matrix V(t)
such that
where D is a p x p diagonal matrix with entries ± 1,if rank
P=
p. However,
p(t) = - qJqJ' = LL'.
Thus
qJqJ' = - V'
1/2 0] [~ o
U'U
[~1/2
0
0
0] V, 0
where U(t) is an arbitrary p x p orthogonal matrix. Hence, we see that the identification DI
L = iV' [ 0
2
/
]
'
202
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
obtained by neglecting the last n - p zero columns of
yields the result (8.53). Relation (8.54) is an easy consequence of the definition of N = R 1 / 2G'p and relations (8.49)-(8.50). REMARKS (1) Theorem 8.5 shows that the functions cp and I" arising from the factorization of M are equivalent, modulo the orthogonal group, to the generalized X-Y functions Land N. We need only be careful to note the convention that L is defined by deleting the irrelevant zero columns from the factorization of Q - SR -1 S'. (2) Relation (8.50) shows that the factor v is independent of the matrix P. Thus e can be produced once and for all at the beginning of any computational process, thereby requiring only the triangular decomposition of the first n rows of the matrix M at each stage of the process. Actually, by (8.49) we see that it is possible to obtain I" by solving a triangular system of equations. Thus the only real factorization necessary is that of determining cp which, by Remark (1), requires only algebraic operations on the first p rows and n columns of M when Q - SR- 1S' has rank p. In short, at most we need to triangularly decompose a single n x n matrix at each step of the process. Furthermore, if desired, even the triangularization of cp may be averted by using the relationship
- P=
Q
+ PF + F' P
- I'I"
(8.55)
to update P. The point here, of course, is that if p(t) has low rank, then the partial factorization of M to produce cp may enable us to update P much more efficiently using the relation
- P = cpcp', in place of (8.55). General Computational Procedure Notwithstanding the theoretical interest associated with the triangular decomposition of M, the primary importance of the above results lies in their use for the development of efficient computational approaches to leastsquares problems. We sketch the outline of a computational algorithm suitable for either time-varying or constant systems.
8.6
203
GENERALIZED X- Y FUNCTIONS
The steps of the algorithm are the following: O.
Factor R(t) into its triangular components ljI(t), ljI'(t). Compute F'(r) by solving the triangular system (8.49). 2. Determine K(t) = ljI-l(t)r'(t). 3. Triangularly factor the first n columns of M to determine factorization will terminate after p columns if p(t) has rank pl. 4. Let t -+ t - 6 and determine P(t - 6) using the relation 1.
- p(t)
=
tp
(note the
qJ(t)qJ'(t).
(Here earlier values of P(t), p(t) may be recalled from storage to use in a multistep integration scheme.) 5. Go to Step 0 (or to Step 1 if R(t) is factored in advance for all r), The preceding algorithm makes clear the substantial advantages offered by the generalized X- Y functions Land N for constant systems. In the timevarying case it is necessary to go through the matrix function P in order to obtain the desired quantity r which, in essence, characterizes the optimal control law. However, Theorem 8.5 shows that for time-invariant systems the Land N equations enable us to produce (modulo an inessential orthogonal transformation) the factors qJ and r directly, totally bypassing the matrix P.
EXERCISES
l. In the event that the quadratic costs arise by "quadraticizing" a nonquadratic functional about a nominal state and control trajectory, the matrix R(t) will usually not be positive definite (or even nonsingular). Examine the factorization approach presented above for applicability to this case. 2. Discuss the possibilities of simplifying the matrix M(P) by application of the" feedback" group of transformations introduced in Chapter 5. 3. Form the augmented matrix function
..HlP)
~
M(P} +
[~l
Show that vII(P) is positive semidefinite if and only if P is a solution of the matrix Riccati equation (8.39).
8
204 8.7
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
OPTIMALITY VERSUS STABILITY
As long as the duration of the control process is finite, we do not encounter any theoretical problems regarding stability of the optimal control law. However, when we begin to study processes over the semi-infinite interval (either [ - 00, T] or [0, 00]), the question immediately arises as to whether or not the optimal feedback law u*(t) = - Kx(t) actually generates an asymptotically stable linear system. An associated question is whether or not the optimal feedback "gain" matrix K should be time-varying, as it is for the finite-interval problem, or constant. The answers to these questions are far from evident and it will require a certain amount of analysis to satisfactorily resolve them. By this time the reader should not be too surprised to see the concepts of controllability and constructibility playing a central role in this study. As we are concerned with the semi-infinite interval problem, let us assume that the system matrices F, G, Q, R, S are constant or, equivalently, the process has been in operation for a sufficient time that they have reached their steady state values. Thus we are interested in the problem of minimizing oo (8.56) J = [(x, Qx) + 2(x, Su) + (u, Ru)] dt,
L
over all piecewise-continuous controls u(t), are related through the linear system x(t) = Fx
+ Gu,
00 :5;
x(O)
=
t
:5;
T, where x and u
c.
(8.57)
Proceeding formally, we could consider problem (8.56)-(8.57) to be the limiting case as T -+ + 00 of the finite interval problem given in the last section.f If this is the case, then it is reasonable to suppose that the optimal cost and control are given by J* = (c, P*c),
u*(t) = - R- 1(G'P*
+ S')x(t),
(8.58)
where P* = limT_oo P(t) is the limiting solution of the matrix Riccati equation (8.39). Correct as the above conjecture is, there are several obstacles to be surmounted before we can accept it as fact. Among the many questions that must be answered are: (i) Under what circumstances does Eq. (8.39)have a limiting solution P*? (ii) Is P* unique in any sense? t Here we have made the change of variable
at
+ CXJ rather than -
CXJ.
I -+ T -
I
in order to have the limiting values
8.7
OPTIMALITY VERSUS STABILITY
205
(iii) If p* exists, does Eq. (8.59) minimize the criterion J? (iv) Does substitution of the feedback control law (8.59) into Eq. (8.57) yield an asymptotically stable system? It is our goal in this section to deal with these questions. The first item of business is to establish results concerning when the finiteinterval equation (8.39)has a limiting solution. Since the Riccati equation coefficients are all assumed to be constant, if P(to, Po, T) denotes the solution beginning at time to with the initial matrix Po for a process ending at time T, then by constancy of the coefficients we clearly have P(t o, P, T) = P(O, Po, T - to). Thus it makes no difference whether we consider t -+ - ex) and the half-line - 00 ~ t ~ T or T -+ + ex) and the half-line to ~ t ~ 00. The first arrangement is convenient from the computational point of view, while the latter reflects the time evolution of the system. It is the former viewpoint which we shall use in this section. We begin with weakened versions of controllability and constructibility which will be useful for subsequent discussions. Definition 8.1 The pair (F, G) is said to be stabilizable if a constant matrix L exists such that F - GL is stable, i.e., Re Aj(F - GL) < O. Definition 8.2 If Q = H' H, the pair (H, F) is called detectable if a matrix M exists such that F + MH is stable. EXERCISE
1. (a) Prove that (F, G) is stabilizable if and only if the unstable characteristic values of F are controllable, i.e., the characteristic vectors corresponding to unstable roots of F lie in the controllable subspace of (F, G). (b) Show that (H, F) is detectable if and only if the unstable characteristic values of F are observable, i.e., the characteristic vectors corresponding to unstable roots of F lie in the complement of the unobservable subspace of(H, F).
For ease of writing, we consider the normalized version of Eq. (8.39) in which S = 0, R = I (i.e., we perform a type II transformation in the input space to reduce R to the identity; this is always possible since we have assumed R > 0). The following elementary results about Eq. (8.39)will form the basis for our study of the infinite-interval problem. The first involves global existence of the solution. Theorem 8.6 Let P(t, Po, T) be the solution at time t < T of Eq. (8.39) passing through Po at time t = T. Then, given any to, P(t, Po, T) exists and is unique on to ~ t s T for all Po.
206
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
PROOF In view of standard results on local existence and uniqueness, P(t, Po, T) exists for some t :s; T. Let cI>(t, T) be the transition matrix of F. Then
P(t, Po, T) = cI>(t, T)Po
+
IT
P(s, Po, T)GG'P(s, Po, T)]
10
:s;
+ JT
which is easily verified by differentiation. The a priori bound above gives a Lipschitz constant on any finite interval to :s; t :s; T, no matter how large. Hence, P(t, Po, T) exists and is globally unique for all Po.
Lemma 8.2
For any to :s; t :s; T and any Po P(t, Po, T)
~
~
0, we have
0.
PROOF If P(t, Po, T) satisfies Eq. (8.39), then so does P'(t, Po, T). Thus Pit, Po, T) is a symmetric matrix. Moreover, since
min J(t o) = (c, P(to, Po, T)c)
~
0,
we obtain the final results.
Theorem 8.7 Pit, Po, T) has the following properties (i) P(to, 0, t.):s; P(to, 0, t 2 ) for any to :s;·t 1 s t 2 ; (ii) P(t2 , 0, ri« P(t.,O, T)foranyt1:S; t 2:s; T; (iii) P(t, Pot, T) :s; P(t, Po 2 , T),for any nonnegativePOl PROOF
s
Po2 and t
For (i), we have (c, P(to, 0, t 1 )c)
= min u
s
[(x, Qx)
1
+ (u, u)] dt
10
12
min' u
=
I
II
[(x, Qx)
+ (u, u)] dt
10
(c, P(t o, 0, t 2 )c)
for any c. To see (ii),we use the constancy of the system coefficients to obtain
P(to, 0, T) = P(O, 0, T - to). Thus (ii) follows from (i) upon identifying to with t 1 and t 2 in turn.
s
T.
8.7
207
OPTIMALITY VERSUS STABILITY
Finally, (iii) is obvious since increasing the terminal penalty results in an increased cost functional. At last we are in a position to return to a study of the behavior of pet, Po, T) as t --+ - 00. The principal result is stated next. Theorem 8.8 If the pair (F, G) is stabilizable, then pet, Po, T) is bounded on - 00 :s; t :s; T for any Po and T. PROOF Consider any control aCt) = stable. Then
to :s; t, such that F
L~(t),
(c, P(t o, Po, T)c) = m:n{(X(T), Pox(T))
s
foo [(~,
+ i~[(X'
Qx)
+ GL
is
+ (u, u) dt]}
Q~) + (a, a)] dt
10
= (c, W(to, t)c), since x(T) --+ 0 as T --+ 00 and c = ~(O). Here W(to, r) is a finite matrix for any t :s; T. Hence P(t o , Po, T) is bounded from above for any to, to :s; t :s; 00 and any Po. Our next result deals with the properties of the limiting solution. Theorem 8.9
If(F, G) is stabilizable, then
lim P(t,O, T)
= Pro = lim peT, 0, 0)
(8.59)
T ....oo
, .... - 00
is a finite nonnegative definite matrix. Moreover, P 00 is a solution of the algebraic Riccati equation Q
+ PF + F' P
- PGG'P = O.
(8.60)
PROOF By Theorem 8.7(ii), Ptt, 0, T) is monotone nondecreasing in t in the ordering induced by nonnegative matrices (i.e., if AI' A 2 ~ 0, then Al ~ A 2 <:> Al - A 2 ~ 0). Further, Theorem 8.8 implies that pet, 0, T) is bounded from above and, consequently, there exists a finite matrix P 00 ~ 0 such that P(t,O, T) --+ P 00 as t --+ - 00. Since P 00 is independent of t, it satisfies (8.60).
It is important to note that even if a stabilizing solution Ps of the algebraic Riccati equation exists, it is not necessarily true that P 00 = Ps. Thus the question is asked: To which nonnegative solution of (8.60) does P(t,O, T) converge? The answer is contained in the next few theorems.
8
208
Theorem 8.10
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
If (F, G) is stabilizable, then
Pee = P", where P" is the solution of(8.60) which generates the minimum value ofcriterion (8.56)-(8.58). PROOF Since Po ;;:: 0, we infer from Theorem 8.7(iii) that P(t, Po, T) ;;:: P(t,O, T) for any Po. Thus, as t --. - 00, P(t,O, T) --. P", the smallest nonnegative solution of (8.60).
It will now be necessary to establish that the detectability condition on the pair (H, F) implies that Eq. (8.60) has a unique nonnegative definite solution P" coinciding with Ps. This will require several auxiliary results of some interest in their own right. We introduce the 2n x 2n matrix
-GG'J -F' , and we assume that U has 2n characteristic vectors, i.e., U is diagonalizable (this is only for convenience, and is by no means a necessary condition). Let
i = 1, 2, ... ,2n, and write
where Xi E R", Yi E R", Uj E R", Vi E R". Thus ai is a right characteristic vector of U, while r, is a left characteristic vector. We may always choose the characteristic vectors such that (a;, ri) = 0, (a;, rj) =1= 0,
i =1= j, i = j.
Theorem 8.11 is the first important result.
Theorem 8.11
Each solution P of Eq. (8.60) has the form
P = YX- 1 , where the matrices
correspond to a choice ofcharacteristic values of U such that X - 1 exists. Conversely, all solutions of (8.60) are generated in this way.
8.7
209
OPTIMALITY VERSUS STABILITY
PROOF Let P satisfy (8.9) and let fF = F - GG'P be the closed-loop system matrix. Then we infer from (8.60) that
PfF = -Q - F'P,
and, hence, (8.61) Let
J = X-I fF X = diag(A. I, A. 2 , · •• , A. n ) be the Jordan canonical form of fF and set PX = Y. Then (8.61) yields
V[~J
=
[~}.
(8.62)
Since J is diagonal, the columns of [D constitute the characteristic vectors of V associated with A. I , A. 2 , ••• , A.n and P = Y X-I. The converse is established by reversing the argument. Corollary The matrix fF = F - GG'P has the characteristic values A. i associated with the characteristic vectors Xi> i = 1, ... , n. Theorem 8.12 Let A. i be a characteristic value of V with G:J the corresponding right characteristic vector. Then - A.i is also a characteristic value of V with [~;;J being the corresponding left characteristic vector. PROOF
The prooffollows by direct verification, making use of the identity
V'=G
-IJo '
and the fact that a left characteristic vector of V associated with A. is a transposed right characteristic vector of V' associated with A.. REMARKS (l) Since V is real, its characteristic values occur in quadruples (A., A. *, - Ie, - A. *). (2) There can exist at most one stabilizing solution P; due to the symmetry of the characteristic values of V. If P, = Y X-I is such a solution, then X* y is Hermitian.
The final two results characterize the existence and uniqueness of the stabilizing solution Ps • Theorem 8.13 The stabilizing solution r, of Eq. (8.60) exists if and only (F, G) is stabilizable and Re A. # 0 for all characteristic values of V.
if
210
8
THE L1NEAR-QUADRATIC-GAUSSIAN PROBLEM
PROOF (Necessity) Suppose P; exists. It is associated with n stable roots of V and, hence, no characteristic values have a zero real part due to their symmetrical placement relative to the imaginary axis. Moreover, the matrix L = G'Ps stabilizes F, i.e., F - GG'p. is stable, which means that (F, G) is stabilizable. (Sufficiency) Suppose that the hypothesis is true and that no stabilizing solution exists. Then either (i) we cannot choose n stable characteristic values of V, which is a contradiction, or (ii) we can do so but the matrix X is singular. If (ii) is the case, let z be any nonzero vector in %(X), the null space of X. Since X*y = y*X, we have
0= Y*Xz = X*Yz. By virtue of the definition of V and (8.62),
-H'HX - F'Y = Y f.
FX - GG'y = Xf,
(8.63)
The first equation yields
z*Y*FXz - z*Y*FF'Yz = z*Y* Xfz, and, hence, G' Y z = O. But,
0= FXz - GG'Yz = Xfz, which means that %(X) is a ,I-invariant subspace of R". Thus there exists at least one nonzero vector Z E %(X) such that fZ = p.z,
where p. coincides with one of the stable roots of U. The second equation in (8.63) postmultiplied by Z yields
-F'Yz
= Yfz.
Collecting these results give us (Y2)'F
=
-p.(Yz)',
Re( -p.) > 0,
(Y2)G = O.
Thus (F, G) is not stabilizable, contradicting the original hypothesis. At last we can prove the result linking P, with P" and Pro. Theorem 8.14 The stabilizing solution P s is the only nonnegative solution of (8.60) if and only if(H, F) is detectable.
8.7
211
OPTIMALITY VERSUS STABILITY
PROOF (Sufficiency) Assuming (H, F) detectable, we show that any solution P of (8.60) is stabilizing. Suppose the contrary, i.e., a A. exists such that
ffz = ..1.z,
ReA. ~ 0,
ff = F - GG'P.
Upon rearranging Eq. (8.60), we have
PGG'P + Pff
+ ff'P + H'H =
0,
and, hence, (A. + ..1.*)z*pz = -z*PGG'Pz - z*H'Hz. Since A. + ..1.* = 2 Re A. ~ 0, the left side of this equation is nonnegative, while the right side is nonpositive. Thus both are zero and
G'Pz = 0,
Hz = 0,
which implies
Fz = ..1.z, Hz = 0.
ReA. ~ 0,
Thus (H, F) is not detectable contradicting our hypothesis. Hence, ff is stable. However, there is at most one stabilizing solution, i.e., the solution P = Ps. (Necessity) Suppose an undetectable root . 1. 1 of the pair (H, F) exists. We shall show the existence ofat least two nonnegative solutions ofEq. (8.60). By hypothesis, one of them is the stabilizing solution P s = y X-I. To form another solution PI = YIX!l, we substitute the characteristic vector [0'] of U associated with . 1. 1 for the vector [~:] associated with -..1.1> thus obtaining Also, set
x=
[x21 .. ·Ixn]
,
Now Theorem 8.13 implies.that [Zl*
Hence,
Zl *Y
=
0*][
~~J
= 0,
i = 2,3, ... , n.
0 and
o
X* as X*y
~
O.
8
212
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
To prove that X t is nonsingular, suppose the contrary. Then there exists a vector v :f. 0 such that Zt = Xv and, consequently,
0= ZI*Y = v*X*Y: that is, det X* Y = O. Noting that det X* Y is a principal minor of the nonnegative matrix X* Y, it is easily seen that
[XtIX]*Yv = X*(Yv) = O. Since X is nonsingular and Yv :f. 0, this is a contradiction. TABLE 8.2 Nonnegative Solutions of Eq. (8.60) Solution
Case (F, G) stabilizable, (H, F) detectable
P, exists and is the unique nonnegative definite solution P,
(F, G) stabilizable,
P, exists and there are other nonnegative definite solutions
(H, F) not detectable (F, G) not stabilizable,
=
p.
No nonnegative definite solution exists including P, and p.
(H, Fldetectable (F, G) not stabilizable, (H, F) not detectable
P, does not exist, but other nonnegative definite solutions mayor may not exist
TABLE 8.3 Asymptotic Properties of the Matrix Riccati Equation (8.39)
Case
Tfinite
Po = 0
(F, G) stabilizable, (H, F) detectable
P oc exists,
P",
(F, G) stabilizable, (H, F) not detectable
P a: exists, P", = p.
P ex>
'" P,
= P, = p.
P", exists, = any nonnegative solution depending on
Po (F, G) not stabilizable, (H, F) detectable (F, G) not stabilizable, (H, F) not detectable
P oc, does not exist P", mayor may not exist
There is always a unique solution and it is nonnegative definite
8.7
213
OPTIMALITY VERSUS STABILITY
Thus, PI does exist and PI #- P; because it corresponds to a different n-tuple of characteristic values of U. Since the foregoing results have been long and arduous, we summarize all conclusions in Tables 8.2 and 8.3. Example: Armament Races
( continued)
We return to the elementary buildup model of Section 8.5 and examine the stability ofthe optimal control for the quadratic functional (8.38). We assume that the process has been "building up" over such a time span that it is reasonable to suppose that the two countries involved are attempting to minimize
where the system dynamics are given by Eqs. (8.36)-(8.37). For simplicity, assume the grievance and goodwill coefficients are zero. The appropriate matrics for this process are F= [
- a I
kJ
G
-b'
S = 0,
= I,
R = diag(/3t, /32),
Po =
o.
The minimizing feedback control law is u*(t) = -R-tG'P*N(t),
where P* is a nonnegative definite solution of the equation
Q + PF + F'P - PGR-IG'P
=
O.
Since G = I, there is no question that (F, G) is stabilizable (it is controllable). Thus, referring to Table 8.2, we need only check the detectability of (H, F) to ascertain the closed-loop stability of the control u*. We have
@=[:F]=
((X I )1/2
0
0
((X2)1/2
- a((XI)I/2
k((XI)I/2
1((X2)1/2
- b((X2)1/2
214
8
THE LINEAR-QUADRATlC-GAUSSIAN PROBLEM
If we assume that the choice of a, b, I, and k is such that both characteristic roots of F are unstable, then the detectability condition means that (H, F) must be observable. In this case, the conditions for stability of u* are: either (i) (ii)
(iii)
1X I1X2:1=
IX I
0, or
= 0, I :1= 0, and
1X2 =
0, k :1= O.
Thus we already see the interplay between the weights each country attaches to holding down armaments production and the defense coefficients k and I. It is interesting to observe that the fatigue coefficients a and b playa role only to the extent that they determine the possibility of weaker conditions than (i)-(iii) if F has a stable root. However, satisfaction of (i)-(iii) implies a stable control law irrespective of the fatigue felt by each party. We close this section by again noting that optimality does not imply stability. The optimizing solution P* of Eq. (8.60) minimizes the cost functional whereas the stabilizing solution makes the closed-loop system asymptotically stable. These are quite different properties as the trivial example J
=
f~<XJ
(x/
+ u2 ) dt,
shows. Here an optimal control u* exists that minimizes J, since the unstable component X2 is absent from J. However, the optimal control law u* is not stable since it cannot possibly stabilize X2' The problem here, of course, is that the system is neither stabilizable nor detectable. In cases for which the dynamics and/or cost function are not so trivial, the preceding theory will be required to determine the stability of optimal laws. 8.8 A LOW-DIMENSIONAL ALTERNATIVE TO THE ALGEBRAIC RICCATI EQUATION
The generalized X- Y functions were seen to provide an alternative approach to the determination of optimal feedback controls, one which explicitly took account of redundancies in the system description to reduce the computing burden imposed by the usual matrix Riccati equation. A careful examination of the L-N system (8.41)-(8.42) shows that, unlike the Riccati equation, the infinite interval version of the equations is not well determined by the usual trick of setting L = IiI = O. Such an approach yields only the conclusion L( (0) = 0, a result that follows immediately from the representation P(t) = L(t)L'(t). An equation for N(oo), the quantity which determines the optimal feedback control, must be obtained through other means.
8.8
215
LOW-DIMENSIONAL ALTERNATIVE TO RICCATI EQUATION
We recall the basic equation to be solved is Q
+ PF + F'P
- (PG
+ S)R- 1(PG + s)'
=
o.
Our results hinge on the following basic lemma from matrix theory. Lemma 8.3 Let P, A, Q be any three matrices for which the product PAQ is defined. Then q(PAQ) = (Q' ® P)q(A), where ® denotes the Kronecker product and a the operator that" stacks" the columns of a matrix into a column vector, i.e., if A = [aij], then PROOF The proof follows from direct component-by-component verification of the asserted relation.
Using Lemma 8.3, we may manipulate the basic equation to obtain an equation for N( (0). Theorem 8.15 Assume that the matrix F - GR- 1S' has no purely imaginary characteristic roots and no real characteristic roots symmetric relative to the origin. Then the optimal steady state N( (0) = N satisfies the algebraic equation q(N) = (I ® R - 1/2G') [(F - GR -I S')' ® I
+ I ® (F PROOF
- GR- 1S,)']-l q(N'N - Q
+ SR-IS').
Collecting terms in the basic equation, we see that
(Q - SR- 1S')
+ P(F
- GR-IS')
+ (F'
- SR- 1G')P - PGR- 1G'P = O.
Applying a to both sides of this relation and using the characteristic value hypothesis, we have q(P)
= [(F -
GR-IS')'®I + I®(F - GR- IS')']-l q(N'N - Q + SR- 1S'),
where the definition N = R -1/2G' P 00 has been used. Multiplication of both sides of the above relation' by (I ® R - 1/2G') and employment of Lemma 8.3 and the definition of N completes the proof. REMARKS (l) The optimal steady state gain function K follows immediately from the relation K = R - 1/2 N + R -1 S'. (2) The importance of Theorem 8.15 is that the algebraic relation for N represents only nm equations in the unknown components of N. This is to be compared with the n(n + 1)/2 equations in the usual algebraic Riccati equation (8.60). As before, we expect a substantial computational advantage if the number of system inputs m ~ n.
216
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
8.9 COMPUTATIONAL APPROACHES FOR RICCATI EQUATIONS
Practical implementation of LQG theory requires that we devote substantial attention to the question of efficient computation of the nonnegative definite, stabilizing solution of the algebraic Riccati equation (8.60), assuming the requisite stabilizability/detectability conditions are satisfied. In this section we briefly review various techniques for the solution of Eq. (8.60). It is very difficult to label one method as being superior to another, since a given method may prove better in one application but fail in another. As usual in mathematics, it is best to have as broad an arsenal of weapons as possible to bring to bear on a given problem. Our aim is to survey the most important methods and to indicate their range of applicability, strengths, and weaknesses.
I.
Characteristic Vector Method
Since good procedures now exist for computing the characteristic vectors and values of a matrix, this method has come into increasing prominence in recent years. The essence of the procedure is described in Theorem 8.11 and its corollary. An interesting side aspect of this method is that it is the only procedure available for computing all the solutions of Eq. (8.60), nonnegative or not, by appropriate selection of subsets of the 2n characteristic vectors of U.
II.
Iterative Method
The basis of this method is a variant oftlie Newton approximation method for finding the root of a system of nonlinear equations by successive linear approximations. We assume that Eq. (8.60) possesses a unique stabilizing solution Ps. Define P, as the unique nonnegative solution of the linear algebraic system j = 0, 1, ... ,
where j
= 1,2, ... ,
and where L o is chosen such that the matrix K o = F + GL o is a stability matrix. Then it can be shown that, in the ordering induced by nonnegative matrices, j
= 0,1, ... ,
8.9
217
COMPUTATIONAL APPROACHES FOR RICCATI EQUATIONS
and lim Pj = Ps •
i« »
This method gives monotonic and quadratic convergence to P, and is believed to be one of the best methods for finding Ps •
III.
Sign Function Method
The sign function method may be regarded as a simplification of the characteristic vector method, designed to find the stabilizing solution Ps • We define the matrix sign function Z as sign Z
=
lim Zk+I'
k -+ ex.
where Zo
= Z.
Also define sign + Z
=
1<1
+ sign Z).
Assuming the matrix iF = F - GG'P is stable, we define a matrix V as the solution of the equation
+ ViF' + GG' = O. It is easy to verify that the matrix U = [_~. H -_GF~'J satisfies iFV
U [~ I~ ~V J[~
_~,J[I
=
~~P
~l
Thus
sign+u=[I -V P 1- PV
J[O0
VP = [ -'-(I - PV)P
O][I-VP I -P
- V I - PV
VIJ
J
and the stabilizing matrix P, follows immediately if V is invertible. In summary, the sign function method algorithm is: (i) Find the matrix sign + U. (ii) From sign + U, extract the upper two blocks VP, V. (iii) Invert V to obtain P, from VP. Now we consider a few approaches for the numerical solution of the finite-interval matrix Riccati equation (8.39).
8
218
THE LINEAR-QUADRATlC-GAUSSIAN PROBLEM
IV. Numerical Integration The most straightforward approach to the solution ofEq. (8.39) is by direct numerical integration using, for example, a Runge-Kutta or AdamsMoulton scheme. Clearly, this method can also be used to compute a steady state solution but for this purpose the scheme is usually time-consuming and not sufficiently accurate. V. Transition Matrix Method This procedure is important from the theoretical point of view since it enables us to obtain an explicit representation of the solution of Eq. (8.39). In some exceptional cases the transition matrix solution even admits an explicit closed-form solution. We must first define a modified version of the previous matrix U, now including the cross-term weighting matrix S and dropping the assumption R = I. This new matrix V is given by _
U =
[
F
_(Q + SR-1S/)
Let p{t) = P{t)x{t). Then P{T) = Pox{T).
Let {t, T) be the transition matrix associated with D, i.e., X{t)] = [C1>1l (t, T) [ p{t) C1>21{t, T)
C1>12{t, T)][X{T)] C1>dt, T) p(T) ,
where C1>iit, T) are submatrices obtained by partitioning {t, T). Then it is easy to see that P{t,P o, T) = [C1>21{t, T)
+ C1>dt, T)P O] [C1> l l{t, T) + C1>dt, T)PO]-l.
The transition matrix method has a difficulty in that {t, T) contains stable as well as unstable modes. As time increases, the unstable modes dominate and solution accuracy. suffers. The next method bypasses this difficulty. VI.
Negative Exponential Method
The basic approach of this technique is to recast the equations so that only negative exponentials occur in the computations. To this end, we write
8.10
219
STRUCTURAL STABILITY OF THE OPTIMAL CLOSED-LOOP SYSTEM
for the 2n x 2n matrix that diagonalizes D, i.e.,
V
=
wPLO
O]W- ' 1
-J
and we set
Then .£(T )] = [e-J(T-ll [ p(t) 0
o
e-J(T-ll
][.£(t)] p(T)'
Since p(T) = Z.£(T), where Z
=
-[W22
-
Po W12rl[W21 - Po WI1 ] ,
if we denote we finally obtain P(t, Po, T) = [W21 + W22G(t, T)] [WI1
+
W12 G(t, T)] -I.
The steady state solution Pro is obtained as Pro
=
lim P(t, Po, T)
= W21Wi/o
8.10 STRUCTURAL STABILITY OF THE OPTIMAL CLOSED-LOOP SYSTEM
We noted in the last chapter that only those systems that are structurally stable have a legitimate claim to physical reality. The pole-shifting theorem ensured that any controllable system could be made structurally stable by application of some fixed feedback law. The question that now arises is whether or not an optimal (but not necessarily stabilizing) law P" generated as a solution of the algebraic Riccati equation Q
+ PF + F'P - PGG'P = 0
(8.64)
is structurally stable. Fortunately, the answer to this question is easily obtained by appealing to our earlier results. Let P* be an arbitrary, but fixed, solution of Eq. (8.64). As before, define the closed-loop dynamics associated with P" as !F* = F - GG'P*.
8
220
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
In terms of $'* we make the following definitions. Definition 8.3 With each P*, we associate a triple of integers [n , *, no*, n_ *), the index of P*, defined as
the number of characteristic values of J having positive real part, the number of characteristic values of J having zero real part, n, * the number of characteristic values of J having negative real part,
n, *
no*
where
J
= $'*
® I + I ® $'* (here ® denotes the Kronecker product).
Definition 8.4 The solution P" of Eq. (8.64) is structurally stable if and only if for any sufficiently small, continuous perturbation of F, G, Q, there exists a unique symmetric P, a solution of (8.64), such that index P" = index P.
Another way of stating the structural stability property is that a small, continuous deformation of the coefficients in Eq. (8.64) preserves the local phase portrait about a structurally stable feedback control. The main result is stated next. Theorem 8.16 and only if
A symmetric solution P* of Eq. (8.64) is structurally stable
det($'* ® I
+ I ® $'*) "# 0,
if
(8.65)
and $'* has no real roots symmetrically placed relative to the origin. PROOF Condition (8.65), together with the restriction on the real roots of $'*, is equivalent to saying that $'* has no purely imaginary roots. The theorem then follows from Theorem 7.12. REMARK If (F, G) is stabilizable, then P" will be trivially structurally stable since $'* will have all of its roots in the left half-plane.
8.11 INVERSE PROBLEMS
A problem of some theoretical and practical interest is the following: Given a time-invariant linear feedback control law K and a controllable system 1: = (F, G, .), determine all cost matrices Q, R, S such that the control u(t) = - Kx(t) minimizes the functional
f"
[(x, Qx)
+ 2(x,
Su)
+ (u, Ru)J dt.
At the outset we make no assumptions on the definiteness of Q or R. Such inverse problems are playing an increasingly important role in applications since there are numerous problems in which the dynamics are reasonably well known and control laws (rules of thumb, heuristic reasoning, etc.) have
8.11
221
INVERSE PROBLEMS
evolved over years of operation of the system. What is usually missing in such situations is a well defined and well understood idea of just what is being minimized by such control laws. In this section, we address ourselves to the treatment of such situations. In relation to the inverse problem, we introduce the function of a complex variable s such that
r(s) = R
+ S'(sf -
+ G/(-sf
F)-IG
+ G/(-sf -
F)-IS
- F')-IQ(s1 - F)-IG.
The "frequency" function I', besides being of interest in its own right, will facilitate presentation of the main results. For completeness, we cite without proof the following important results concerning I": (i) If (F, G) is controllable, a necessary and sufficient condition that the algebraic Riccati equation
Q
+ F'P + PF
- (PG
+ S)R- 1(PG + S)' =
0
(8.66)
have a real, symmetric solution P is that r(iw) :2 0
for
all real w.
(ii) There will be one and only one stabilizing P, satisfying the algebraic Riccati equation if and only if there exists an [; > 0 such that for
all real co.
We impose the following conditions on the system matrices F, G, Q, R, S and on the function I": (a) (F, G) is completely controllable. (b) R > O. (c) The regulator problems all reach a zero terminal state, i.e., x(oo) = O. (d) r fulfills condition (ii). (e) If A. is a characteristic root of F, then - A. is not a characteristic root. The relevance of condition (e) is seen from the next lemma. Lemma 8.4
Consider the set of equations
+ F'P = PG + S =
PF
-Q,
(8.67)
H'.
(8.68)
Suppose condition (e) is satisfied. Then i/Q and S are given, Eqs. (8.67)-(8.68) define one and only one matrix H such that F(s)
=
Z(s)
+ Z'( -s),
222
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
where Z(s) PROOF
= I + H(sl -
F)- lG,
J
+
j'
=
R.
Define the symbols <1>+
= (sl
-
rv:',
<1>_'
= (-sl
- £,)-1.
Then Eq. (8.67) may also be written as P+ 1 + <1>'_-1 P = Q,
from which we obtain _'p
+ P$+
= $_'Q$+.
Considering the definition of I', we substitute the last expression for $ _ 'Q$ + and for S we substitute the expression obtained from (8.68), completing the desired result. We are now in a position to solve the following problem. PROBLEM A Given the system (F, G, Qo, R, So) (with x( (0) = 0), find all criteria (Q, S) that lead to the same r (i.e., we keep the dynamics (F, G) fixed). SOLUTION
The solution of Problem A is given by Theorem 8.17.
Theorem 8.17 Let H be obtained from (Qo, So) as in Lemma 8.4. Then all solutions of Problem A are obtained by varying P in the set of real, symmetric, n x n matrices in Eqs. (8.67)-(8.68) and keeping H, F, G, and R fixed. PROOF We note first that lim,s'-->00 r(s) = R, so R is given by the specification of r. Each pair (Q, S) obtained as in the theorem statement is a solution to Problem A because, according to Lemma 8.4, r depends only on H, F, G, R and, by construction, each (Q, S) leads to the same H, the other matrices being held fixed. Each solution to Problem A is obtained in this way. Let (Q*, S*) be such a solution. Then, by virtue of Lemma 8.4, one can construct an H* so that r may be written as
r = R + H*$+ G + G'_'H* = R + H+ G + G'_'H, which gives (H* - H)$+ G = G'$_'(H - H*).
Each pole of the left side is a characteristic value of F, each pole of the right side is a characteristic value of - F, and these two collections of characteristic values are separated by hypothesis (e). The result is that the two
8.11
223
INVERSE PROBLEMS
members are analytic in the whole complex plane (including the point at infinity where they are zero). Thus they are constants equalling zero. But (H* - H)+ G = 0 implies H* = H by the complete controllability of (F, G). Hence, (Q*, S*) belongs to the family generated by the stated procedure. REMARKS (I) Since the system (8.67)-(8.68) has a unique solution H, it is possible to choose Q arbitrarily and then calculate S. (2) The solution to Problem A gives a meaning to every symmetric P and not just to those P 2 O.
EXERCISE
1. Show that all pairs (Q, S), which are solutions of Problem A, give rise to regulator problems having the same optimal solution characterized by the constant gain K = R- 1(G'P + S'). We are now ready to tackle the main problem of this section, namely, the question: Are all regulator problems with given dynamics (F, G) (with x( (0) = 0), and leading to the same optimal gain K, members of the same family, i.e., do they have the same I"? To answer this question, we must solve the next problem. PROBLEM B Given the completely controllable dynamics (F, G) and a gain K such that F - GK is asymptotically stable, find all criteria (Q, R, S) defining a regulator problem (with x( (0) = 0) having u = - Kx as its optimal solution. . SOLUTION
The solution of Problem B is given by Theorem 8.18.
Theorem 8.18 All solutions of Problem B are obtained by applying the solution of Problem A (Theorem 8.17) to the initial regulator (F, G, Q*, R, S*), where R > 0 is arbitrary and Q* = K'RK, S* = K'R. PROOF The triplets (Q, R, S) generated in this way are solutions of Problem B. Once R is fixed, it will suffice, by virtue of Theorem 8.17, to demonstrate
this for the particular pair (Q*, S*) and for this we write the criterion as
50
00
J =
[(x, K'RKx)
+ 2(x, K'Ru) + (u, Ru)] dt,
with R > O. Clearly, u = - Kx is the optimal law since it makes J identically zero and obeys the constraint x( (0) = 0 (F - GK asymptotically stable). Now we show that each triplet (Q, R, S) solving Problem B (having K as optimal gain) is obtained in this way. Indeed, set R = R. An easy calculation
8
224
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
shows that the pair Q* = K'RK, S*' = RK is part of the family generated by Eqs. (8.67)-(8.68) starting with (Q, 05). Conversely, (Q, 05) could have been obtained starting with (Q*, S*). To show that the inverse problem posed as Problem B is actually fairly trivial, we now give a much simpler and more direct way of solving it, with F, G, K being given. (i) Choose R > 0 arbitrarily. (ii) Choose a symmetric matrix (iii) Derive S from the relation
n arbitrarily.
K = R- 1(G'n
+ S').
(iv) Derive Q from the algebraic Riccati equation Q
+ nF + F'Il
- (nG
+ S)R- 1(nG + S)' =
o.
As a result of the foregoing analysis, we see that the answer to the question posed above is negative: those problems having the same optimal gain and the same dynamics do not all lead to the same function r. But, this is true "modulo R," in the sense that if F, G, K, and R are fixed, then r is fixed also. EXAMPLE To illustrate Theorem 8.17 for the solution of Problem A, assume n = 2 and that F, G are in control canonical form, i.e.,
F = Fo = [ 0 -a1
1 ]
-ao'
Further, let R = I, So = 0, and Qo = diag(ql' q2)' Our task is to characterize all matrices (Q, S) that generate the same frequency function r as (Qo, So). According to Theorem 8.17, the first step in such a characterization is to utilize Lemma 8.4 to find the (unique) matrix H corresponding to (Qo, So). Since So = 0, we must solve the equations PF
+ F'P
=
-Q,
PG= H'.
Carrying out the indicated calculations, we find
The conclusion of Theorem 8.17 now states that all solutions of Problem A are obtained by keeping F, G, R, H fixed, and letting P vary over the set of
8.11
225
INVERSE PROBLEMS
real. symmetric matrices. This means that the appropriate pairs (Q. S) are given by the expressions
Q = -(PF + F'P) = [ -(P ll
-
2alP12 aOP 12 - a 1P22 )
-(P ll - aOP 12 - a 1P22)] 2aOP22 - 2P 12 •
where P ll • P 12 • P 22 are arbitrary real numbers. In other words, all quintuples (F o. Go. Q, I. S), with Q and S as above. give rise to the same frequency function r. The solution of Problem B is easily obtained by the foregoing procedure by choosing Qo = K'RK. So = K'R, R > 0, arbitrary. For instance, assume that a control law of the form K = (0, k 2 ), measuring only the state X2. is a stabilizing law for the control canonical system. Then, according to Theorem 8.18,we form all criteria (Q. R, S) that generate this law by letting
Qo = K'RK,
So = K'R
with
R >
°
arbitrary. Choosing, as before, R = I, we have
Qo=[~
k~21
So=K'=[~l
Applying Theorem 8.18, we find the corresponding P, H' as a/ k/ 2ao
P=
°
I·
H' = PG + S = [ 2° k a1k/ a1k2 --+ 2 2ao ao Thus all criteria generating the law K = (0, k 2 ) are characterized by the matrices
°
[
Q _ [
P ll
-
-2a 1P12 aO P 12 - a 1 P 22 12
S
=
[a1k/ -P ] 2ao + k 2 - P 22
with P ll , P 12• P 22 arbitrary.
R
=
I.
226
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM EXERCISES
1. Consider the case when R = I, S = 0, i.e., the criterion function J is parametrized only by the cost matrix Q and assume that (F, G) is controllable. 2. (a) Show that a given gain matrix K is optimal for some Q if and only if (i) Re A(F - GK) < 0, (ii) K = G'L for some real symmetric L. (b) If the conditions of (a) are satisfied, show that one appropriate Q is given by
Q = K' K - LF - F'L. (c) If Gis of full rank m, show that condition (ii) of part (a) may be replaced by (ii') KG is a symmetric matrix. (d) If the restriction Q ~ 0 is imposed, show that K is optimal for some such Q if and only if: (i) A(F - GK) < 0 and (ii) T*(iw) ~ I for all real w where T(s) = I + K(Is - F)-lG. (Hint: Use the fact that T*(iw)T(iw) I ~ 0 implies the existence of an m x n matrix H such that T'( - s)T(s) = I + G'(-Is - F')-lH'H(Is - F)-lG.) 3. (a) Assume that K is optimal for Ql' Prove that K is then optimal for Q2 if and only if there exists a real symmetric matrix Y satisfying
Q1
Q2 = F' Y
-
+ YF
with YG = O. (b) Let rank G = r. Show that the set of real, symmetric matrices Q such that K is optimal for Ql + Q has dimension (n - r)(n - r + 1)/2. 4. Consider the case of a single-input system, i.e., m = 1, R = 1, under the same assumptions as in Problems 1 and 2. (a) Show that there exists a unique equivalent diagonal matrix Q* for all admissible Q if and only if the set of n(n + 1)/2 equations Mg=O,
-qij
= i
L" mikfkj + himkj,
k= 1
= 1,2, ... , n -
1, j
= i + 1, ... , n,
have rank n. (b) If the conditions of (a) are satisfied, show that the elements of Q* are given by q~ = qjj
"
+ 2 L mikhi, k=l
i
= 1,2, ... , n.
(c) Show that the conditions of (a) are satisfied when (F, g) are in control canonical form and that in this case we have the unique Q* given by q~
=
qii - 2qi-l.i+l
+
2qi-2.i+2 - "',
i
= 1,2, ... , n,
8.12
227
LINEAR FILTERING THEORY AND DUALITY
where the alternating sum is continued until all available qs are exhausted. (Remark: This result shows that the equivalent diagonal Q* depends only on those elements of Q which have even index, i.e., those % such that i + j = even.) 8.12 LINEAR FILTERING THEORY AND DUALITY
In Chapter 4 we noted the precise, mathematical duality between the concepts of reachability/observability and controllability/constructibility and saw that results concerning one concept could be interpreted as results about the other simply by reversing the roles of the system inputs and outputs. The question naturally arises as to what type of duality one might expect to find when a quadratic criterion function is superimposed on the system. Obviously, the earlier results remain in force as the criterion has nothing to do with the controllability/observability aspects of the system; however, following the earlier "prescription" of interchanging the inputs and outputs in order to form the dual problem, we find that the deterministic, quadratic cost control problem has for its dual a stochastic least-squares filtering problem involving gaussian statistics. This is the reason for the word "Gaussian" in the nomenclature "linear-quadratic-Gaussian " problem and it will be this remarkable correspondence between two seemingly diverse problem areas that will now occupy our attention. The basic problem is the following. We have a message which is a random process x(t) generated by the model
dxldt
= F(t}x + G(t)u.
The observed signal is
z(t) = y(t)
+ v(t) =
H(t)x(t)
+ v(t),
where the functions u(t), v(t) are independent, white, Gaussian, noise processes with zero means and covariance matrices
8[u(t), u(r)] = Q(t)15(t - r), 8[v(t), v(r)] = R(t)c5(t - r), 8[u(t), v(r)] = 0 for all t, r. Here 15(·) is the Dirac delta function, 8 the mathematical expectation, and Q(t), R(t) symmetric matrices with Q(t) ~ 0, R(t) > 0 for all t. We further assume that is known.
228
8
THE LINEAR-QUADRATlC-GAUSSIAN PROBLEM
The basic problem may now be stated. OPTIMAL FILTERING PROBLEM
to ::::;; s ::::;; t1> find an estimate
~(t1)
Given known values of z(s) in the interval of x(t,) of the form
(where A is an n x p matrix) with the property that 8[X(t1) - ~(t1)'
x(t 1) - ~(t1)J2
-+
min.
An alternate way of viewing the above filtering problem is to regard the basic dynamics of the system to be linear, i.e., x = Fx. The stochastic input term Gu(t) is then added to account for uncertainties in our choice of the original dynamics. The interpretation of the measured output z(t) remains basically as before, with vet) added to account for measurement error. Depending on one's philosophical point of view, this interpretation of the filtering problem may make the linearity assumptions easier to accept. Of course, the mathematics remains insensitive to the philosophy! The main result concerning the above "canonical" filtering problem is stated next. Theorem 8.19 The solution of the optimal filtering problem with to > is given by the relations = F(t)~
d~(t)/dt ~(to)
+ K(t)[z(t)
-
00
(8.69) (8.70)
H(t)~(t)J,
= 0,
where K(t) = P(t)H'(t)R -let) and pet) satisfies the matrix Riccati equation dP/dt = G(t)Q(t)G'(t)
+ F(t)P + PF'(t) - PH'(r)R -1(t)H(t)P,
P(t o) = Po. Furthermore, the optimal error x(t) = x(t) -
(8.71) (8.72)
~(t)
satisfies the equation
dx(t)/dt = F(t)x(r) + ,G(t)u(r) - K(t) [H(t)x(t) x(to) = xo'
+ vet)],
(8.73)
PROOF The above results are by now classical and any number of excellent proofs have been given. We refer the reader to the sources listed in the notes at the end of the Chapter for complete details.
As a consequence of Theorem 8.19, we see that the optimal filter is a feedback system. It is obtained by taking a copy of the model of the message process (omitting the constraint at the input), forming the error signal itt) = z(t) - z(t), and feeding the error back with a gain K(t).
8.12
229
LINEAR FILTERING THEORY AND DUALITY
EXAMPLE A particle leaves the origin at time to = 0 with a fixed but unknown velocity of zero mean and known variance. The position ofthe particle is continually observed in the presence of additive white noise. We are to find the best estimate of position and velocity. Let x1(t) be the position and X2(t) the velocity of the particle. We know that Xl(t O) = 0, but X2(tO) has only a known variance, say X2*, and zero mean. The dynamics of this process are
with the observations being ZI(t)
= xl(t) + v(t),
v(t) being white noise with zero mean and (co)variance r(t)(j(t - r),
We note that in this process there is no uncertainty in the dynamics of the system, only the initial state x(to) is not known exactly. Hence, the system matrices are
[~ ~
F=
J
H = [1 0],
Q(t) = 0,
R(t) = [r(t)],
G(t) = 0,
Po [~x~*J
X2*
=
~
O.
The variance equations for this problem are . P ll
= 2P I2 -
1 2 r(t) PI\>
1'12
= P n - r(t) P llP 12,
Pll(to
)
= 0,
1
. 1 2 P 22 = - r(t) P 12,
The optimal filter gain K(t) is K(t) =
[kll(t)] k 21(t)
= P(t)H'R- 1 = _1 [Pll(t)],
r(t) P tit)
giving the optimal state estimator equations as d~l
-
dt
= ~2
d~2 _ dt-
Pll(t) r(r)
+ - - [z(t)
- ~l(t)],
P tit) [ ( ) ~ ( )] --;:(t)zt-1t,
~1(0)
= 0,
230
8
THE LlNEAR-QUADRATIC-GAUSSIAN PROBLEM
In light of Eq. (8.71) for the covariance of the optimal error, it is now easy to see the duality relations between the above optimal linear filtering problem and the linear quadratic control problem of the earlier sections. If we define a dynamical system which is the dual (or adjoint) of the filtering system, we have the correspondences
t* = -t,
F*(t*) = F'(t),
G*(t*) = H'(t),
H*(t*)
=
G'(t), (8.74)
where the quantities on the left are for the dual system, while those on the right are for the filtering system. The dual system has the dynamics
dx*/dt* = F*(t*)x*
+ G*(t*)u*.
(8.75)
Using the duality relations (8.74) in Eq. (8.71), and comparing the result with the earlier matrix Riccati equation (8.32)for the control problem, we see that the deterministic linear dynamics, quadratic cost control problem in the dual variables (F*, G*, H*) is mathematically equivalent to the optimal linear filtering problem in the variables (F, G, H). That is, the control problem of minimizing
r 1*
[(y*, Q(s*)y* + (u*, R(s*)u*)] ds*
+ (x*(t o*), Po*x*(to*»,
subject to the dynamics (8.75), with y*(t) = H*(t)x*, has the same solution as the optimal linear filtering problem under the duality relations (8.74), and conversely. With the foregoing duality relations well in hand, it is now a straightforward exercise to establish many important results concerning the filtering situation by translating their control-theoretic counterparts using (8.74). For example, the results on the stability, numerical methods, and inverse problems presented in earlier sections may now be transferred to the optimal filtering setting with no difficulty.
EXERCISES
1. Show that if the control problem is modified to require minimization of (y* - Yd*' Q(y* - Yd*» + "', where Yd*(t*) =1= 0, is the desired output, this corresponds to an estimation problem with tfu(t) =1' O. 2. Give an example of a filtering problem that is identical to its dual (i.e., it is self-adjoint). 3. Construct the dual to the particle problem given in the example above. What is the physical interpretation of the corresponding criterion function?
8.13
THE SEPARATION PRINCIPLE AND STOCHASTIC CONTROL THEORY
231
8.13 THE SEPARATION PRINCIPLE AND STOCHASTIC CONTROL THEORY
In the introductory chapter we noted that one of the most striking results in linear systems analysis is the so-called separation principle, which, in effect, allows us to decompose the solution of the LQG problem into two independent parts: an estimation process followed by a deterministic controller. Having the above filtering results at our disposal, we are now in a position to present this fundamental result in mathematical form. However, we shall content ourselves with only a statement of the result rather than a proof, since the proof is not at all easy unless considerable background in probability and statistics is assumed. Let there be given the linear system with additive input noise
x = Fx + Gu + v, where the input noise v is white, Gaussian, and of zero mean with covariance B[v(t)v'(t)] = Q(t)b(t - r),
Q(t) ;?: 0
for
all t.
Further, assume the output of the system is given by y(t) = Hx(t)
+ w,
where w is also white, Gaussian noise of zero mean with covariance S[w(t)w'(r)] = R(t)b(t - r),
R(t) > 0
for all t.
In addition, let the noise processes v and w be illdependent. The initial state x(t o) = Xo is also a normally distributed random variable with mean Xo and covariance Po, and is independent of the processes v and w. It is of utmost
importance to note that we are not talking about the same problem here as in the filtering situation. The foregoing system is assumed to be a deterministic system driven by the noise process v. Thus we are considering a stochastic control process and not a pure filtering problem as before. Due to the noise in the dynamics, as well as the observations, it is not possible to pose an optimization problem requiring minimization of the quadratic form J =
JT [(x, Qx) + (u, Ru)] dt,
Q ;?: 0,
R
> 0,
10
because the performance criterion J is, itself, a random variable depending on v, w, and Xo' To deal with the situation, we replace the deterministic problem of minimizing J by the problem of minimizing its expected value B[J] = J,
232
8
THE LlNEAR-QUADRATIC-GAUSSIAN PROBLEM
where the expectation is taken over xo, v, and w. It is understood that at time t the measurements y(t), to ~ r ~ t, are available, and that the optimal control u*(t) is to be expressed in terms of y(t), to ~ r ~ t (note that u*(t) is not required to be an instantaneous function of y(t». The solution for the above problem is carried out in two steps. 1. Estimation Compute a minimum variance estimate ~(t) of x(t) at time t, using u(t), y(T), to ~ r ~ t. This estimate satisfies the equation d~/dt
= F(t)~
~(to)
= 0,
+ G(t)u + P(t)H'(t)R -l(t)[y(t) -
H(t)~],
where dP/dt = F(t)P P(t o)
=
+ PF'(t)
- PH'(t)R-1(t)H(t)P
+ (2(t)
Po·
Note that the equation for ~ is independent of the cost matrices Q and R. Although the context is quite different, the production of ~(t) by the above prescription is virtually identical with the procedure followed in the Kalman filtering context. 2. Control Compute the optimal control law u*(t) = - K(t)x(t), which would be applied if there were no noise, if x(t) were available, and if J were the performance criterion. Then use the control law u*(t)
= -
K(t)~(t),
where ~(t) is obtained from the equation above. This law will be optimal for the noisy problem. Note that calculation of K(t) is independent of H(t) and the statistics of the noise. Summarizing, we see that the optimal control is obtained by acting as if were the true state of the system and then applying the deterministic theory presented earlier in the chapter. Hence, the name "separation principle," indicating that the two phases, estimation and control, are separate problems which can be tackled independently. Schematically, we have Fig. 8.1. ~(t)
r-------r-I Noisy Linear System t-----,r"'"""'"-
Control Law from Deterministic Problem
FIG. 8.1
~ (I)
The separation principle.
8.14
233
DISCRETE-TIME PROBLEMS
8.14 DISCRETE-TIME PROBLEMS
Many problems of optimal control and filtering involve measurements that are taken at discrete moments in time. For example, economic processes in which annual data is used, sampled-data chemical process control systems in which the output of the system is analyzed only daily, and so on. In these instances, it is more natural to formulate the control/filtering problem dynamics as a finite-difference equation, rather than a differential equation. Thus the dynamics are x(k
+
I)
=
F(k)x(k)
+ G(k)u(k),
with the quadratic costs now being expressed as the finite sum J
=
N-l
L [(x(k), Q(k)x(k») + (u(k), R(k)u(k))] + (x(N), Mx(N).
1=0
In Chapters 3 and 4, we have already observed that there is no fundamental difference between basic systems-theoretic concepts for continuous- or discrete-time problems. The algebraic statement ofthe results is more complicated in discrete time, but there are no foundational issues dependent on the structure of the time set. For this reason we have usually presented only the continuous-time result, as the mathematical formalism is more compact. As illustration of the foregoing remarks, we now give the basic discretetime results for the optimal linear filtering problem. It will be a worthwhile exercise for the reader to translate them (using the duality theorem) to the control-theoretic setting. Let {x(k)} and {z(k)} be n, p-dimensional vector stochastic processes generated by the linear model x(k
+
1) = F(k)x(k) z(k) = H(k)x(k)
+ G(k)u(k), + v(k),
x(O) = xo, k ',2: 0,
where we assume xo, {u(k)}, and {v(k)} have zero mean, are uncorrelated for k > 0, and c!{xox o'} = Po, c!{u(i)u(j)'} = Q(i)<5 ij, c!{v(i)v'(j)} = R(i)<5ij'
R(i) > 0
for
all i.
Here ~ij is the usual Kronecker delta symbol. Our objective is to obtain the best linear estimate (in the least-squares sense) of x(n), given the observations {z(O), z(l), ... , z(n - I)} and the above model. If we denote the best estimate by ~(n), then the following result summarizes the optimal discrete-time Kalman filter.
234
8
Theorem 8.20
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
The optimal estimate ~(k) ~(k
+
1) = F(k)~(k)
~(O)
satisfies the system
+ K(k)[z(k) -
H(k)~(k)],
= 0,
where the optimal gain matrix K(k) is given by K(k) = F(k)P(k)H'(k) [H(k)P(k)H'(k)
+ R(k)] - 1.
The error covariance matrix P(k) = 8 ((x(k) -
~(k))(x(k)
-
~(k))'),
is computed from the discrete-time Riccati equation P(k
+
1) = F(k)P(k)[I - H'(k)(H(k)P(k)H'(k)
+ G(k)Q(k)G'(k),
+ R(k))-1 H(k)P(k)]F'(k)
P(O) = Po.
8.15 GENERALIZED X-Y FUNCTIONS REVISITED
In Section 8.6 we presented an alternate approach to the Riccati equation for the solution of the LQG control problem. The functions replacing the matrix Riccati equation, termed generalized X - Y equations, were seen to have distinct computational advantages whenever the dimensions of the system input and output spaces are substantially less than the dimension of the state space. We now wish to return to this topic and present the corresponding results for discrete-time filtering problems. For simplicity, we treat only the case when the system and covariance matrices are constant, although the more general case may be dealt with in a corresponding manner, as is indicated in the references. Considering the discrete-time filtering process outlined above, we introduce the auxiliary notation
T(k) = FP(k)H',
S(k) = R
+ HP(k)H'.
Then, in this notation, the optimal filter gain matrix K(k) has the form
,K(k) = T(k)S-I(k). Using the above quantities, the following result may be obtained.
Theorem 8.21 relations
The functions S(k), T(k) may be obtained from the recursive
T(k L(k S(k U(k
+ 1) = + 1) = + 1) = + 1) =
T(k) + FL(k)U(k)-1 L(k)'H', [F - T(k + I)S(k + 1)-1 H]L{k), S(k) + HL(k)U(k)-1 L(k)'H', U(k) - L(k)'H'S(k)-IHL(k), k
~
1.
235
MISCELLANEOUS EXERCISES
The initial conditions for Sand T at k = 0 are T(O)
=
FH',
S(O)
=
R
+ HPoH',
while L(O) and V(O) arefound by factoring the matrix D = FPoF'
+ GQG' -
Po - FH'(R
+ HPoHr'HF'
as M+
D = L(O) [ 0
lr
0
M- 1'(0)',
Then L(O) is the initial value for the function L(k), while U(O)
0 = [ M0 + M-
J-'
'
PROOF The proof of this important result may be found in the references cited at the end of the chapter.
From a computational point of view, the importance of Theorem 8.21 is that the sizes of the matrix functions T and S are dependent only on the dimensionality of the measurement process, i.e., T and S are n x p, p x p matrix function, respectively. In addition, we see that the dimensions of L and V are governed by a parameter r = rank D. The matrix L has dimension n x r, while V is of size r x r. Since S = S', V = V', the total number of equations is n(p + r) + t[P(p + 1) + r(r + l)J. Thus, in the event p, r ~ n, substantial computational savings may be anticipated by use of the above discrete-time generalized X - Y functions, as opposed to the Riccati equation, for computation of the optimal filter gain function K(k). MISCELLANEOUS EXERCISES
f:
1. Consider the problem of minimizing J =,
[(x, x)
+ (v, A(t)x)J dt
over all vector functions x(t) differentiable on [0, TJ with x(O) = c. (a) Show that if A(t) is constant and A > 0, then the optimal curve satisfies the Euler equation x(t) - Ax
(b)
= 0,
x(O) = c, x(T) = O.
Introducing the matrix functions sinh X = f{ex - e- X ),
cosh X = f{e x
+ e- X ),
236
8
THE LlNEAR-QUADRATIC-GAUSSIAN PROBLEM
show that the solution to the above equation is x(t)
= (cosh A 1/2T)-I(cosh A 1/2(t -
T»c,
where A 1/2 denotes the positive-definite square root of A. (c) Show that the minimal value of J is given by Jmin(T) = (c, A 1/2(tanh A 1/2T)c). (d) In the case of time-varying A(t), show that the Euler equation remains unchanged and that its solution is given by x(t) = [X 1(t) - X 2(t)X /(T)- 1 X 1'(T)]c,
where XI (t) and X 2(t) are the principal solutions of the matrix equation
X-
A(t)X = 0,
with X 1(0) = I, X1(0) = 0, X 2(0) = 0, X2 (0) = I. (Hint: It must be shown that X /(T) is nonsingular for all T for which J is positive.) (e) Show that Jmin(T) in the time-varying case is given by Jmin(T)
= (c, X 2'(T)-1 X 1'(T)c), = (c, R(T)c),
and that R(T) satisfies a matrix Riccati equation. 2. Consider the problem of minimizing J
=
foT [(x, x) + (u, u)] dt
over all u, where x = Fx + Gu, x(O) = Cl, x(T) = C2' To avoid the problem of determining those u which ensure satisfaction of the condition x(T) = c 2, consider the modified problem of minimizing
r = foT [(x, x) + (u, u)] dt + A(x(T) -
C2, x(T) - C2)
for A ~ O. The only constraint is now x(O) = c i- Study the asymptotic behavior of J~in as A -+ 00 and obtain a sufficient condition that there exist a control u such that x(T) = C2' Compare this result with the controllability results of Chapter 3. 3. (a) Consider the matrix Riccati equation -dP/dt
= Q + PF + F'P
- PGG'P
=
Y(P)
with Q, F, G constant matrices, and let Y + = {Po: Po = Po' and Y(P o) ~ O}, Y - = {Po: Po = Po' and Y(P o) =:;; O}.
(t)
237
MISCELLANEOUS EXERCISES
If P +, P _ denote the supremum and infimum of Y' +, respectively, show that P +(P_) exists if and only if Y' + is nonempty and there exists a matrix L 1 E f/ _(L z E f/ _) such that F' - L 1 GG'( - F' + L z GG') is uniformly asymptotically stable. = {Po: Po - P _ > O}, prove that the matrix Riccati equation (b) If~po (t) has a global solution and as t ~ 00, P(t) ~ P + if Po E ~~_. Conversely, P(t) has a finite escape time if Po $ ~ r-: = {Po : Po - P _ ~ O} and Po - P _ is nonsingular. 4. Consider the matrix Riccati equation dP/dt = Q
+
Show that this equation matrix W such that (a) - W ® R- 1 > 0 (b) A(J¥, F, R, Q) < (b') A(J¥, F, R, Q) ~
PF
+ F'P
- PR- 1 p ,
P(o) = Po.
has a finite escape time if there exists a symmetric
°°
and or and
u(W)'u(Po) > !u(W)'W- 1 ® R(F ® I
+ I ® F)u(W)
+ t[A(J¥, F, R, Q)]l/zu(W)'U(W)/A, where A(J¥, F, R, Q) = -
[U(W)~U(W)}U(W)'(F
® I + I ® F)'W- 1
® R(F ® I + I ® F)u(W) + 4u(W)'u(Q)], and A > 0 is the smallest characteristic value of - W ® R - 1. (Here, u( .) is the column "stacking" operator introduced in Lemma 8.3 and ® is the Kronecker product.) 5. (Generalized Bass-Roth Theorem) Show that every real equilibrium solution of the algebraic Riccati equation PF
+ F'P
+ Q= 0
- PGR- 1G'P
is a solution of [ - P ® I]A(Jr) = 0, where Jr is the Hamiltonian matrix Jr=[F -Q
-GR-F'
1G'J
and A is a real polynomial of degree d possessing roots all of which are characteristic values of Jr. 6. (a) Let (F, G) be stabilizable and let AI.... ,A p be those characteristic values of Jr that are also the undetectable characteristic values of (F, H), where Q = H'H, i.e.,
Hz; = 0,
Re A;
~
0, i
=
l, ... , p.
8
238
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
Further, assume the set A1 , ••• , Ap consists only of cyclic characteristic values, i.e., any two characteristic vectors associated with Ai are linearly dependent. Show that under these hypotheses the algebraic Riccati equation has exactly 2P nonnegative definite solutions. (b) Let CC = {AI>"" An} be the set of all characteristic values of:K and let CCk> k = 1,2, ... , 2n denote the subsets of CC. Write P, for the solution of the algebraic Riccati equation which is generated from the stabilizing solution by replacing all - Ai by Ai E CCk : Prove that any two nonnegative solutions P k , PI satisfy if and only if i.e., the set of all nonnegative definite solutions constitutes a distributive lattice with respect to the partial ordering ~. P* is the smallest (zero element) of the lattice, while P, is the largest (identity element) of the lattice. (Note: The physical interpretation of different nonnegative solutions is that each nonnegative solution is conditionally optimizing, the condition being a certain degree of stability. Specifically, Pk stabilizes the undetectable characteristic values of (R, F) included in CCk and no others. The notion that the more undetectable characteristic values that are stabilized, the higher the cost is made rigorous via the lattice concept.) 7. If (F, G) is stabilizable, but (F, H) is not detectable, show that the solution of the matrix Riccati equation -dP/dt = Q + PF
+ F'P
- PGR- 1G'P
can be made to approach any nonnegative solution of the algebraic Riccati equation by suitable choice of Po. (Thus, in general, P( 00) is not a continuous function of Po.) 8. Show by an example that even a completely controllable and completely observable system may have structurally unstable indefinite equilibria. (Hint: Consider a system with trivial dynamics.) 9. Show that the optimal feedback control law for the problem of minimizing (x, (T), Mx(T))
+
J:
[(x, Qx)
+ (u, u)] dt,
x=
Fx
+ Gu, T <
00
with F, G, Q, M constant, is constant, i.e., u = - Kxit) with K a constant matrix, if M satisfies the algebraic Riccati equation
Q + MF + F'M - M GG'M and is positive definite.
= 0
239
MISCELLANEOUS EXERCISES
10. Using the notation of Lemma 8.3, we define a rectangular matrix H = PH to be positive semidefinite if the symmetric matrix P ~ 0. Let the pair (F, G) be controllable and assume HI and H 2 are solutions of the equations u(Hd = (G' ® 1)(1 ® F' + F' ® I)-lu(H1H 1' - Qd, u(H 2) = (G' ® 1)(1 ® F' + F' ® I)- l u(H 2H 2' - Q2)'
(t)
m
with QI> Q2 ~ 0, (F, Qt/2 ), (F, QY2) completely observable. Show that a necessary and sufficient condition for (t) and W to have the same positivesemidefinite solution is for U(QI - Q2)EKer[(G'®/)(/®F'
+ F'®l)-l].
11. (a) Consider the quadratic performance indices Jj =
iT
[(x, Qjx)
+ (u, Rju)] dr,
i = 1,2.
10
We say that J 1 is equivalent to J 2 if and only if they yield the same feedback control law for the system
x=
Fx
+ Gu.
Show that J 1 is equivalent to J 2 for all T if and only if Ql(t)FkGR"l l = Q2(t)FkGR2 1 for k = 0, 1, ... , n - 1, to ~ t ~ T. (b) To determine all criteria which are equivalent to a given constant pair (Q, R) for a fixed T < 00, show that it is s.ufficient to find all matrix pairs (X, Y) which satisfy the matrix equation X[GIFGI··
./F,,-lGJ
= Q[GR-11F,GR-11·· ·IF,,-lGR-1JY
subject to X ~ 0, Y > 0. 12. In the Kalman filtering problem, assume that the model is completely observable, i.e., for all t there exists some to < t such that the matrix M(to, r) =
I'
<1"(-.; t)H'(T)H(T)<1'(T, r) dt > 0,
10
where <1'(s, t) is the transition matrix of F(t). In statistics, M(to, t) is known as the Fisher information matrix corresponding to the special estimation problem in which (i) u(t) == and (ii) v(t) is Gaussian with unit covariance matrix. Letfbe an arbitrary, but fixed, linear functional and let J.l(t) be an unbiased estimator of (x(t), f).
°
(a) Prove the Cramer-Rao inequality 8[J.l(t) - 8J.l(tW ~ (f, M-1(t o , t)f),
240
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
(b) Show that equality holds for some f if and only if M(t o , r) > 0, i.e., every costate has a minimum variance unbiased estimator if and only if the system model is completely observable. NOTES AND REFERENCES
Section 8.1 General results on control theory, with special emphasis on
the LQG problem, are by Bellman, R., "Introduction to the Mathematical Theory of Control Processes: Linear Equations-Quadratic Criteria." Academic Press, New York, 1967. Anderson, B., and Moore, J., "Linear Optimal Control." Prentice-Hall, Englewood Cliffs, New Jersey, 1971. Brockett, R., "Finite-Dimensional Linear Systems." Wiley, New York, 1970. Kirk, D., "Optimal Control Theory: An Introduction." Prentice-Hall, Englewood Cliffs, New Jersey, 1971.
A comprehensive summary of the state-of-the-art in LQG theory (circa 1970) is found in Special Issue on Linear-Quadratic-Gaussian Problem (Athans, M., ed.) IEEE Tran. Auto. Cont., AC-16, No.6 (1971).
A survey of LQG problems and results is given in Casti, J., The linear-quadratic control problem: Some recent results and outstanding problems, SIAM Review, 22, 459-485 (1980).
Additional details on the water quality example are given by Sz.Nagy, A., On the optimal stochastic control of water resource systems. WP-75-111, International Institute for Applied Systems Analysis, Laxenburg, Austria, September 1975.
Section 8.2 Our development of the open-loop solution is adapted from Kailath, T., Some Chandrasekhar-type algorithms for quadratic regulators, Proc. IEEE Dec. Control Conf., New Orleans, 1972.
Section 8.3 The classic reference for the maximum principle is by Pontryagin, L., Boltyanskii, V., Gamkrelidze, R., and Mischenko, E., "The Mathematical Theory of Optimal Processes,". Wiley (Interscience), New York, 1961.
For connections between the maximum principle and dynamic programming, see Dreyfus, S., "Dynamic Programming and the Calculus of Variations." Academic Press, New York, 1965.
Section 8.4
A more detailed discussion of numerical techniques for both linear and nonlinear boundary value problems is given by
Casti, J., and Kalaba, R., "Imbedding Methods in Applied Mathematics:' Addison-Wesley, Reading, Massachusetts, 1973.
NOTES AND REFERENCES
241
Scott, M., "Invariant Imbedding and Its Application to Ordinary Differential Equation: An Introduction." Addison-Wesley, Reading, Massachusetts, 1973. Keller, H., "Numerical Methods for Two-Point Boundary Value Problems." Ginn (Blaisdell), Boston, Massachusetts, 1968.
Section 8.5 The original derivation of the linear feedback solution using the matrix Riccati equation is Kalman, R., Contributions to the theory of optimal control, Bo//. Soc. Mat. Mexicana 5, 102- II 9 (1960).
Our presentation follows Brockett, R., "Finite-Dimensional Linear Systems." Wiley, New York, 1970.
A treatment of various situations in which the standard LQG assumptions are violated is provided in the volume Jacobson, D., "Extensions of Linear-Quadratic Control, Optimization and Matrix Theory," Academic Press, New York, 1977.
See also Jacobson, D., Martin D., and Pachter M., "Extensions of Linear-Quadratic Control Theory," CSIR Tech. Report TWISK 141, National Res. Institute for Mathematics, CSIR, Pretoria, South Africa, March 1980.
The arms race example is adapted from Saaty, T., "Mathematical Models of Arms Control and Disarmament." Wiley, New York, 1968.
Section 8.6
The development given follows that by
Casti, J., Some recent developments in the theory and computation of linear control problems, in "Calculus of Variations and Control Theory" (David L. Russell, ed.). Academic Press, New York, 1976.
The generalized X- Y functions were first introduced in special form by Casti, J., Kalaba, R., and Murthy, K., A new initial value method for on-line filtering and estimation, IEEE Trans. Inf~rmation Theory IT-18, 515-518 (1972).
The more general case was presented by Kailath, T., Some new algorithms for recursive estimation in constant linear systems, IEEE Trans. Information Theory IT-19, 750-760 (1973).
Casti, J., Matrix Riccati equations, dimensionality reduction, and generalized X-Y functions, Utilitas Math. 6, 95-110 (1974).
For numerical results, see Casti, J., and Kirschner, 0., Numerical experiments in linear control theory using generalized X-Y functions, IEEE Trans. Automatic Control AC-21, October, 1976.
242
8
THE LINEAR-QUADRATIC-GAUSSIAN PROBLEM
The triangular factorization method was presented in Casti, J., Generalized X- Y functions, the linear matrix inequality, and triangular factorization for linear control problems, RM-76-IO, International Institute for Applied Systems Analysis, Laxenburg, Austria, February 1976.
Connections with scattering theory are developed in Kailath, T., and Ljung, L., A scattering theory framework for fast least-squares algorithms, in "Multivariate Analysis-IV" (Krishnaiah, P., ed.) 387-406, 1977.
Group-theoretic ideas unifying many of the X-Y, scattering theory and Riccati approaches to the LQG problem are presented in Orfanidis, S., A group-theoretical approach to optimal estimation and control, J. Math. Anal. Applic., 97, 393-416 (1983).
Section 8.7 The primary results of this section are taken from the survey articles by Kucera, V., A review of the matrix Riccati equation, Kybernetika (Praauev 9,42-61 (1973). Kucera, V., A contribution to matrix quadratic equations, IEEE Trans. Automatic Control AC-I7, 3~347 (1972).
Section 8.8 These results were first given by Casti, J., A new equation for the linear regulator problem, J. Optimization Theory Appl. 17, 169-175 (1975).
For numerical results, see Casti, J., A reduced dimensionality method for the steady-state Kalman filter, in "Stochastic Optimization and Control" (R. Wets, ed.), Vol. 5. pp. 116-123. Math. Prog. Studies, North-Holland Publ., Amsterdam, 1976.
Section 8.9 The characteristic vector method first appeared in Potter, J. E., Matrix quadratic solutions, SIAM J. Appl. Math. 14,496-501 (1966).
The iterative approach is due to Kleinman, D., On an iterative technique for Riccati equation computations, IEEE Trans. Automatic Control AC-13, 114-115 (1968).
A recent account of the many uses of the matrix sign function is Denman, E., and Beavers, A., The matrix sign function and computations in systems, Appl. Math. & Computation 2,63-94 (1976).
Section 8.10 Our discussion follows Bucy, R., Structural stability for the Riccati equation, SIAM J. Control 13, 749-753 (1975).
NOTES AND REFERENCES
243
For a different approach, much more in the spirit of modern geometry, see Schneider, c., Global aspects of the matrix Riccati equation, Math. Systems Theory 7,281-286 (1973).
A decomposition of the standard LQG problem into a direct sum of subproblems using the group of orthogonal symmetries is developed along with several examples in Mozhaev, G., Use of symmetry in linear optimal control problems with a quadratic performance index-I, II, Automation & Remote Control, 6, 22-30 (1975), and 7, 23-31 (in Russian) (1975).
In Chapter 5, the feedback group was used to develop canonical forms and invariants for the system ~ = (F, G, H). By considering the LQG problem £ = (F, G, Q, R, S, Po), (F, G) reachable, Q = Q', R = R', R > 0, Po ~ 0, and the Riccati group of transformations (I): £ -+ l = (TFT- 1, T-1G, T'QT, R, T'S, T'P o T), det T # 0,
= (F, GV, V'RV, SV, Po), det V # 0, (III): £ -+ l = (F + GL, G, Q + SL + L'S' + L'RL, R, S + L'R, Po), L = arbitrary, M=M', (IV): £-+l = (F, G, Q + F'M + MF, R, S + MG, Po - M), (II): £ -+ lz
it is possible to develop canonical structures and invariants for classifying the set of all LQG problems. This idea is developed in Khargonekar, P., "Canonical Forms for Linear-Quadratic Optimal Control Problems," Ph.D. Dissertation, Dept. of Electrical Engineering, U. of Florida, Gainesville, Florida, 1981.
Section 8.11 The first results on the inverse problem of linear control (for single-input systems) are given by Kalman, R., When is a linear control system optimal? J. Basic Eng. Trans. ASME, Ser.D 86D, 51---60 (1964).
More recent results for the general case are presented by Molinari, B., The stable regulatorproblem and its inverse, IEEE Trans. Automatic Control AC-18, 454-459 (1973).
Bernhard, P., and Cohen, G., Study of a frequency function occurring in a problem of optimal control with an application to the reduction of the size of the problem, Rev. RAIRO J-2, 63-85 (1973) (French).
Results relating the LQG problem and algebraic invariant theory are reported in unpublished work by R. Kalman and in Casti, J., Invariant theory, the Riccati group, and linear control problems, RM-OO-OO, International Institute for Applied Systems Analysis, Laxenburg, Austria, 1976.
8
244
THE LINEAR-QUADRATlC-GAUSSIAN PROBLEM
Section 8.12 The pioneering works on linear estimation using a state variable approach are Kalman, R., and Bucy, R., New results in linear prediction and filtering theory, J. Basic Eng. Trans. ASME, Ser. D 83D, 95-100 (1961). Kalman, R., A new approach to linear filtering and prediction problems, J. Basic Eng. Trans. ASME, Ser. D 82D, 35-45 (1960).
More recent results, along with numerous examples, are summarized in the books by Bucy, R., and Joseph, P., "Filtering for Stochastic Processes with Applications to Guidance." Wiley (Interscience), New York, 1968. Astrom, K. J., "Introduction to Stochastic Control Theory." Academic Press, New York, 1970.
A comprehensive survey of the entire field is provided by Kailath, T., A view of three decades of linear filtering theory, IEEE Trans. Information Theory IT-20, 146-181 (\974).
and Willems, J., Recursive filtering, Statistica Neerlandica, 32, 1-39 (1978).
An alternate approach, not requiring an a priori system model, is presented by Kailath, T., and Geesey, R., An innovation approach to least-squares estimation-IV. Recursive estimation given lumped covariance functions, IEEE Trans. Automatic Control AC-16, 720-727 (\971).
For related material, see also Kailath, T., Fredholm resolvents, Wiener-Hopf equations, and Riccati differential equations, IEEE Trans. Information Theory IT-IS, 665-672 (1970).
Section 8.13
A proof of the separation principle may be found in
Bucy, R., and Joseph, P., "Filtering for Stochastic Processes with Applications to Guidance." Wiley (Interscience), New York, 196~.
Another important reference in this area is by Wonham, W. M., On the separation theorem of stochastic control, SIAM J. Control 6, 312-326 (\968).
We have seen that the separation principle is valid for Gaussian statistics and quadratic costs. These conditions are only sufficient for its validity; however, a general set of necessary conditions seems to be unknown, at present.
NOTES AND REFERENCES
245
Section 8.14 The discrete-time recursive filter was first presented in the first paper cited under Section 8.12 above. Additional results and extensions can be found in the books by Bucy, R., and Joseph, P., "Filtering for Stochastic Processes with Applications to Guidance." Wiley (lnterscience), New York, 1968. Astrom, K. J., "Introduction to Stochastic Control Theory." Academic Press, New York, 1970.
Section 8.15 A substantial amount of work has been carried out to extend the discrete-time generalized X- Y functions to nonstationary processes. For some representative samples, see Morf, M., and Kailath, T., Square root algorithms for least-squares estimation, IEEE Trans. Automatic Control AC.20, 487--497 (1975).
This paper also contains extensive results on the triangular factorization approach to the solution of time-varying filtering and estimation problems. Other works dealing with the same circle of questions are Kailath, T., Dickinson, B., Morf, M., and Sidhu, G., Some new algorithms for recursive linear estimation and related problems, Proc. 1973 IEEE Dec. Control Conf., San Diego, December 1973. Lindquist, A., On Fredholm integral equations, Toeplitz equations. and Kalman-Bucy filtering, Internat, J. Appl. Math. Optimization I, 355-373 (1975). Rissanen, J., A fast algorithm for optimum predictors, IEEE Trans. Automatic Control AC-18, 555 (1973). Rissanen, J., Algorithms for triangular decompositions of block Hankel and Toeplitz matrices with application to factorizing positive matrix polynomials, Math. Compo 27, 147-154 (1973).
CHAPTER
9
AGeometric-Algebraic View of Linear Systems
9.1 ALGEBRA, GEOMETRY, AND LINEAR SYSTEMS
The basic results surrounding linear systems-reachability, observability, stability, optimality-all have a resolutely algebraic flavor. In order to determine the basic properties of ~, we must carry out operations involving the ranks of certain matrices, the spaces spanned by a given collection of vectors and so forth. In the preceding chapters, we have stayed within the confines of elementary linear algebra and matrix theory for the presentations of these results. This was primarily for pedagogical purposes; the most compact language for linear systems is that of abstract algebra (module theory) and algebraic geometry (the theory of algebraic varieties). The purpose of this chapter is to show how our earlier results can be unified and streamlined, as well as extended in a number of important directions, by stating them in the "natural" language of systems-algebra and geometry! Before embarking upon an exposition of the algebro-geometric theory of linear systems, it is worthwhile to take a moment to justify the claim that algebra and geometry are the languages of systems. Why should we go to the extra trouble and effort to develop an abstract algebraic theory of linear systems? What advantage does such a theory possess? In short, why bother? The answer to these questions are many-fold: • algebra is the tool for constructing new mathematical objects from old in a natural (i.e, canonical) way; 246
9.2
MATHEMATICAL DESCRIPTION OF A LINEAR SYSTEM
247
• algebra is compact: many of the "gadgets" of analysis (differential equations, Laplace transforms, integral representations) are only artifacts as far as linear system theory is concerned. Algebra provides a means to bypass these artifacts, while at the same time unifying the so-called "state-space" and "frequency-domain" approaches. • algebra is computationally congenial. The tools of analysis by their very nature involve limiting operations of various sorts. Since digital computers can neither exactly represent real numbers nor engage in infinite calculations, the tools and concepts of analysis cannot be directly applied in digital computers. On the other hand, the basic notions of algebra involve finite operations upon sets of elements, emphasizing structure and transformation. Such an orientation is much closer in spirit to the digital computer; • the concepts of modern algebra (homology theory, category theory, algebraic topology, etc.) are, for the most part, of rather recent vintage. Consequently, an algebraic treatment of linear systems enables us at once to make contact with some of the most active mainstreams of modern mathematics. In the sections that follow, we shall assume that the reader is familiar with the elementary concepts of algebra as found in a typical introductory undergraduate course (ring, field, module, homomorphism, etc.). For a brief refresher on these matters, any ofthe introductory texts cited in the references can be recommended. 9.2 MATHEMATICAL DESCRIPTION OF A LINEAR SYSTEM
We adopt the following standard definition of a linear system: Definition 9.1 A discrete-time, constant, linear, m-input, p-output dynamical system ~ over a field k is a composite concept (F, G, H), where
F: X -+X, G: km-+X,
H: X -+k P are abstract k-homomorphisms, with X an abstract vector space over k. The dimension of~ is, by definition, dim X. Naturally, once we fix a basis in X, the k-homomorphisms F, G, H can be identified with their corresponding matrix representations. The dynamical interpretation of ~ is given by the equations x(t
+ 1) = Fx(t) + Gu(t), yet) = Hx(t),
with t E Z, x(·) E X, u(·)
E
k", and y(.) E k".
248
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
The preceding definition describes what is usually termed the "internal" model ofa system E, We now define the corresponding "external", or input/ output, model. Definition 9.2
A linear, zero-state, input/output map over k is a map f: 0
such that
°
~
r
°
0 = {all k-vector sequences w: Z ~ k", such that w(t) = for all t < t - ::; and all t > 0, where t - 1 is some finite integer} (b) r = {all k-vector sequences y: Z ~ kP such that y(t) = for all t ::; O] (c) 0 and rare k-vector spaces with f a k-homomorphism. (d) f is translation invariant in the sense that the following diagram commutes, (a)
°
0-1..- r
anj
jar
0---y- r where (In and (Jr are left-shift operators defined by (In = (0,
, w( -1), w(o); 0, ...) f--+ (0, ... , w(O), 0; 0,
(Jr = (0,
,0; y(l), y(2), ...) f--+ (0, ... ,0; y(2), y(3),
), ).
REMARKS:
(1)
We may interpret the shift operators (In and (Jr as (In = shift left and append a zero (Jr = shift left and discard first symbol
(2)
The sequences [f(e;)]j = jth component of the vector f(e;), with e, = ith unit vector, i.e. [e
a .= { 0,I, k
i
=k
i #- k,
i
= 1, 2, ... , m,
will provide the same information as the impulse-response map of a continuous-time, constant linear system. Knowledge of these sequences suffices to determine the zero-state input/output behavior of a constant linear system. In our standard notation we have e, E 0 corresponding to the sequence t=O t #- 0.
9.3
THE MODULE STRUCTURE OF
n, r AND X
249
The fundamental problem oflinear system theory is to construct (realize) a canonical linear dynamical system !:, whose input/output map II:. agrees with a given input/output map f. Regarding (F, G, H) as matrices, for a moment, Definitions 9.1 and 9.2 immediately imply that E realizes f if and only if
9.3 THE MODULE STRUCTURE OF
n, r
AND X
Our main objective now is to establish the following Fundamental Theorem of Linear System Theory The natural state set X f associated with a discrete-time, linear, constant input/output map f over k admits the structure of a finitely generated module over the ring k[z] (polynomials in the indeterminate z with coefficients in k).
To prove the theorem, we shall introduce a number of definitions and constructions, which will ultimately enable us to verify that the state set X f (as defined below) satisfies the axioms for a finitely generated module over k[z]. It is most convenient to introduce the various canonical constructions in a sequence of steps. STEP 1 Q ~ km[z], regarding km[z] as a k-vector space. The explicit form of the isomorphism is
co ~
L w(t)z-t E km[z]. tEZ
Note that by Definition 9.2(a), the above sum is always finite, and t the convention adopted in Definition 9.2(a).
~
0 by
STEP 2 Q ~ km[z], regarding km[z] as a k[z]-module. In fact, Q is a free k[z]-module with the m generators {el , e 2 , ••• , em}' where
o o o 1
o o
+- i
th
position.
250
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
The above claim is easily validated by defining the action of k[z] on through scalar multiplication as -: k[z]
a
x a --+ a
(n,W)Hn·w,
with
wjEk[z],)= 1,2, ... ,m.
Here nWj is the usual product of one polynomial in k[z] by another. STEP 3 On a the action of the shift operator 0"n is represented by multiplication by z. Thus, dynamical action is transformed into the algebraic operation of multiplication. STEP 4 r is isomorphic to the k-vector subspace of kP[[Z-l]] (formal power series in z - 1) consisting of all formal power series with no constant term. The explicit isomorphism is
y
~
Ly(t)z-tEkP[[Z-l]]. teZ
In general, the sum is infinite and is to be interpreted strictly algebraically with no question of convergence. The isomorphism is completed by noting that y(O) = O. STEP 5 cation as
r
has the structure of a k[z]-module by defining scalar multipli-:k[z] x
r--+r
(n, Y)Hn·y = n(O"r)Y.
This product is equivalent to the rule: multiply y by tt in the usual way and then delete all terms containing non-negative powers of z. We have now seen that a and r admit natural k[z]-module structures. It is now necessary to connect-up these structures with the input/output map f. To this end we have Definition 9.3 Given two inputs w, w' E 0, we say that lent to w', written W == fW' if and only if f(w 0 v)
= f(w' v) 0
for all v EO.
W
is Nerode equiva-
9.3
THE MODULE STRUCTURE OF
n, r AND X
251
Here "0" denotes the operation of concatenation in 0, i.e. o:OxO--+O
(co, v) f--+ O"}}lw v v,
where Iv I = length of v and v is the join operation w v co'
= (0, ... , w( -t), ... , w( -1), co'(-t), ... ,w'( -1);0, ...)
It is easily verified that
== J defines an equivalence relation on O.
Definition 9.4 The set of equivalence classes under == J' denoted X J {(w)J: we O} is the state set of the input/output map f.
=
We now return to the problem of relating the module structure on 0 and r to the map f and its state set X J. Proposition 9.1 The N erode equivalence classes X J offare isomorphic to the k[z] quotient module Olker f. PROOF
By the relation w v= 0
zlvl w
+v
and the k-linearity of f, we have few v) = few' v) 0
0
for all veO,
if and only if f(z'· co) = f(z'· co')
for all r
~
0 in Z.
There is no intrinsic reason for selecting the input space 0 to relate to X J. By duality we could just as easily have chosen the output set r, as is indicated in the problems at the end of the chapter. The preceding development shows that the state set of the input/output map f can be given the structure of a k[z]-module. Let us now consider the corresponding question for the state set Xl; of a dynamical system given in "internal" form. Proposition 9.2 The state set Xl; of the system 1: = (F, G, -) admits a k[z]-module structure. PROOF X = kn is already a k-vector space. To make it into a k[z]-module, we define scalar multiplication as
-: k[z] x k" --+ k" (n, x)f--+n(F)x.
(Here n(F) is just the polynomial n( . ) evaluated at the matrix F).
252
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Let us now restate some basic facts of system theory in the above moduletheoretic language. Proposition 9.3 In the system L F: XI; -,XI; is given by x~z·x.
= (F, G, H) with state module X, the map
The result follows immediately from Proposition 9.2, if X X = XJ' then PROOF
x(l)
XI;' If
= Fx(O) + Gw(O)
+ Gw(O),
= F[~]J assuming x(O) results from the input input z . ~ + w(O). Hence x(l)
=
~.
This implies that x(l) results from the
= [z· ~ + w(O)]J =
z- [~]J
=
z- [~]J
+ [w(O)]J + Gw(O),
which establishes the result. We can now re-state the usual criterion for reachability in more elegant fashion. Proposition 9.4 The system L = (F, G, -) is completely reachable columns ofG generate the k[z]-module XI;'
if the
PROOF
if and only
Assume any x E XI; can be written as m
X
=
L njgj,
j=l
By Proposition 9.3, this is the same as saying m
X
=
L1 nj(F)gj'
. j=
which is equivalent to complete reachability by the usual criterion involving the matrix
It is an easy exercise in application of the basic definitions to show that the external system with state module X J is both completely reachable and completely observable. Let us now show how to obtain a module-theoretic definition of complete observability for the internal system L = (F, -, H).
9.4
SOME SYSTEM-THEORETIC CONSEQUENCES
253
Consider the k-homomorphism H: Xl; --+ Y = k". Let us extend H to a k[z]-homomorphism fl as
u. Xl;
--+
r
x ~ (Hx,
Hiz- x), H(Z2. x), ...).
From the standard definition of an observable state, we see that no non-zero element of the quotient module XJker fl is unobservable. Thus, we have The system ~ = (F, -, H) is completely observable quotient module X Jker fl is isomorphic with Xl;'
Propositon 9.5
only
if the
if and
The above reachability/observability results suggest two important modules: (i) the submodule of Xl; generated by G, i.e. k[z]G; (ii) the quotient module X Jker tt, characterizing the observable states of~.
If we are interested in states which are both reachable and observable, the obvious thing to do is factor the unobservable states out of the submodule of reachable states. This new quotient module xg = k[z]G/ker fl is called the canonical state set for the system ~ = (F, G, H). If we have X~ ~ Xl;' then we say that Xl; is canonical relative to G, H. This terminology now allows us to address the question of modeling input/ output data by an internal system ~. The first main result is Correspondence Theorem of k[z]-homomorphisms f: basis change in Xl;'
There is a bijective correspondence between the set and the set of canonical systems ~, modulo a
n --+ r
In other words, every input/output map f has associated with it an internal model ~, which is unique up to a coordinate change in the state space Xl;' 9.4 SOME SYSTEM-THEORETIC CONSEQUENCES
The above module theory framework has exhibited a number of basic system-theoretic facts in clearer and sharper detail. However, there are a number of less obvious results which can also be obtained deriving mainly from the fact that k[z] is a principal ideal domain. Here we sketch a few of the most interesting developments, leaving others to the Exercises and Problems at the end of the chapter. Let us begin by recalling the notion of a torsion module. Definition 9.5 A module M over a commutative ring R is said to be a torsion module if for every m E M, there exists an r E R such that r . m = O. If this is not the case, then M is called a free module.
9
254
Definition 9.6
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
If L c M, the annihilator A L of L is the set
AL
= {rER: r-l = 0
for alII E L}.
REMARKS:
(l) (2)
A L is an ideal in R. M a torsion module does not imply A L =f O.
The preceding definitions allow us to prove the following important system-theoretic fact:
Theorem 9.1 (Recall: dim PROOF
~ ~
<
isfinite-dimensional 00
means dim
Xr.
if and only if Xr. is a torsion <
00
k[x]-module.
regarded as a k-vector space.)
Let Xr. be finitely generated by the q elements x., ... ,XqEXr.'
Thus, AXE = A X l n A X 2
II'"
n Ax•.
Since k[z] is a principal ideal domain, each A X j is a principal ideal, say yjk[z], YjEk[z]. If Xr. is a torsion module, degYj=nj>O for allj= 1,2, ... ,q. Hence, we can replace the expression q
x=
L »
j=l
by q
x=
L
[ni m od Yj)]' »»
j= 1
which shows that X r. as a k-module is generated by the finite set i.e. ~ is finite-dimensional. On the other hand, assume dim ~ < 00. Let t/JF be the minimal polynomial of the map F: x ~ z . x. Since X r. is finite-dimensional as a k-module, deg t/JF > O. This means that t/JF annihilates every XEXr., so that Xr. is a torsion k[z]-module. Since the above results have established the Fundamental Theorem, namely, that the natural state set of ~ (either X f or X r.) admits the structure of a finitely-generated k[z]-module, we can apply the following central result of algebra to linear dynamical systems.
Invariant Factor Theorem for Modules Every finitely-generated module M with n generators over a principal ideal domain R is isomorphic to
9.4
255
SOME SYSTEM-THEORETIC CONSEQUENCES
where the R/t/JiR are quotient rings of R regarded as modules over R, the t/Ji are uniquely determined by M up to units in R, t/Jilt/Ji-1' and R' denotes the free R-module with s generators, r + s s; n. NOTE M is a torsion module if and only if s = O. Thus, dim ~ < 00 if and only if s = O. We defer a direct system-theoretic translation of the Invariant Factor Theorem to the next section. For the moment, let us push the abstract algebra a bit further and consider the important case when M is cyclic, i.e. generated by a single element. For the special case of interest for linear system theory R = k[z]. Let the state module X be generated by the element g. If t/Jg is the minimal polynomial of g, we can prove the important result
Lemma 9.1
The state module X
~
k[z]/t/Jgk[z].
For ease of notation, we write t/J 9 = t/J. Since X is cyclic, R is commutative, and t/Jg annihilates all XEX. Let us recall that the elements of k[z]/t/Jk[z] are the residue classes of the polynomial n(mod t/J), i.e. the remainder after dividing n by t/J, n Ek[z]. If we write these classes as [n], the notation ii will refer to the standard representative from [n], i.e. a polynomial ofleast degree in [n]. The element ii is uniquely determined by the conditions that ii E [n] and deg ii < deg t/J. It is now easy to establish the result that k[z]/t/Jk[z] is isomorphic to the kvector space {iiEk[z]: deg ii < n = deg t/J}. This leads to the interesting Proposition 9.5
deg t/J.
If XI: is cyclic with minimal polynomial
t/J,
then dim
~
=
Since the Invariant Factor Theorem exhibits the k[z]-module X as the direct sum of cyclic k[z]-modules, we can compute the dim ~ exactly from Proposition 9.6 then
IfX f is a torsion module with invariantfactors
dim
L=
deg
t/J l' t/J2, ... , t/J"
t/Jl + deg t/J2 + ... + deg t/Jr'
The dynamical behavior of a cyclic X f is described by the map: inputs -+ states, which we write as OJ -+
[OJ] =
OJ
mod t/Jf = OJ.
Intuitively, X f is a pattern recognition device in that an input OJ is presented to X f and stored as the remainder after division by t/J r- The stored pattern OJ may have no obvious relation to OJ since [OJ]f may be quite complicated as the following examples due to R. Kalman show. Nonetheless, the operation of the system is quite simple in the algebraic sense.
9
256
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
EXAMPLE 1 If;iz) = Zk. Then [OJJ 1 = (jj = tern "remembers" the last k input values. EXAMPLE
2
If;r(z) = [OJJ
1
=
Z -
(jj
=
1. Thus,
OJ o
+ .., + OJk_IZk-1. Thissys-
= 1 mod If; r- Hence,
Z
+ OJ 1 + .., + OJr ,
OJ o
This system "integrates" the input
r
= deg OJ.
OJ.
EXAMPLE 3 If;,(z) = Zk - ex. Here we have ZklOJ = exlOJ mod If; t : This system is sensitive to inputs of period k in that such patterns are reinforced, whereas non-periodic patterns are averaged out. However, past inputs can be enhanced or deemphasized by selecting ex > 1 or ex < 1.
Another type of problem which can be dealt with algebraically is the following controllability question: EXAMPLE 4 Find an input OJ which transfers a given state x to O. Assume that X is a cyclic k[zJ-module with generator g and annihilator If;. Then we can write
x = e;,.g = ~.g,
ife;,
Thus, the problem is to find an input X·OJ
=
~
such that
+ OJ)'g = 0,
(Zl+d~
where d is an integer such that d satisfy z ' +d~
OJ
= ~modlf;.
deg
OJ.
Since g -# 0, it is clear that
+ OJ = 0 mod If;, d ~
deg
OJ
must
OJ.
Unfortunately, deg OJ enters nonlinearly here in the exponent; however, since the equation must hold only mod If;, we can choose d arbitrarily, subject only to the condition that d ~ n - 1. Then a solution to the problem is OJ
=
_Z1+d~
+ vIf;,
where v E k[zJ is chosen so that- deg OJ = d. Note, however, that such an OJ may not be the OJ of minimal degree transferring x -+ O. If we want the solution of minimal degree which transfer x -+ 0, we must take deg
OJ
--
= P iff deg zP+ 1~ =
p,
p
~
n.
A number of other important system-theoretic facts such as the control canonical form and the pole-shifting theorem shall be deferred to a later section. Let us now examine the classical concept of a transfer function in the light of the foregoing algebraic set-up.
9.5
257
TRANSFER FUNCTIONS
9.5 TRANSFER FUNCTIONS
The machinery introduced in the earlier sections allows us to introduce the classical transfer function as a natural algebraic object, independent of the heuristic notion involving Laplace transforms of inputs and outputs. If we consider an arbitrary k[z]-homomorphism f: n -... r we have that f is "equivalent" to the set {f(eJ: i = 1,2, ... , m, e i = i th unit vector}, since m
few) =
I
wj·f(e)Er.
j=l
By our earlier definition of the k[z]-module r, each fee) is a formal power series in z - 1 with no constant term. Our approach is to represent these formal power series by ratios of polynomials. These ratios are what we shall call "transfer functions." Elements few) will then appear as the product of a polynomial times a ratio of polynomials. The relevant rules of calculation will then constitute a totally algebraic version of the so-called "Heaviside calculus". We begin with a k[z]-torsion-module X f = n/ker f, with I/! = minimal polynomial of X r: Then I/!' f(e j)
= f(I/!' e) = 1/([I/!' ej]) = 17(I/!' [ej]) = O.
(Here· denotes the special module product in T, while no dot will represent the ordinary product of a vector of polynomials by the single polynomial e), Thus, the above calculation implies that the ordinary product I/!f(ej)
= ()jEkP[z],
j=I,2, ... ,m,
i.e. I/! "cancels out" the output fee), leaving the p-vector of ordinary polynomials ()j' Now let us define fee)
= ()il/!·
It is easily checked that the formal division of ()j by I/! into ascending powers of Z-l has the coefficient of ZO as zero. The above construction lead to the basic Representation Theorem. Let f: n -... r be any k[z]-homomorphism with annihilating polynomial w. Then f is uniquely determined by its p x m transfer function matrix W(z), whose columns are the p-oector rational functions f(e j) = ()il/!,j = 1, ... , m, in the indeterminate z. Conversely, any matrix W(z) with proper rational elements induces a unique k[z]-homomorphism fw by the rule fw: n -... I': ejH 8il/!w, where I/!w is the least common denominator of W. The element I/!w is the annihilating polynomial of the k[z]-module X w induced by fw.
258
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
This theorem shows that Wand f are equivalent objects. Thus, to compute f(w) for a given WEn, we can proceed as follows: 1. Compute the ordinary vector-matrix product W(z)w. 2. Expand the results in powers of z -1 and throwaway all terms involving nonnegative powers of z; the result is f(w).
We have already seen the invariant factor theorem for modules in the previous section. The corresponding classical result for polynomial matrices is Invariant Factor Theorem/or Polynomial Matrices with elements in a principal ideal domain R. Then P
Let P be a p x m matrix
= AAB,
where A and B are matrices of sizes p x p, m x m, resp., with elements in Rand det A, det B are units in R. The matrix
is unique (up to units in R) with 2JA.i+ l' T = 1,2, ... , q - 1, q = rank P. The elements Ai are called the invariant factors of P.
As should be expected, there is a direct relation between the elements I/Ji of the module-theoretic invariant factor theorem and the elements Ai of the classical polynomial matrix version. In fact, they are equal as the following theorem shows. Theorem 9.2 The invariant factors ofajinitely-generated torsion k[z]-module X with annihilating poiynomial sl/ and generators {gl' g2"'" gm}, are identical to those of the polynomial matrix I/J W(z) where W(z) is the transfer function matrix associated with X by the rule W(z) = H(zI - F)-IG, where F:X ..... X:x~z·x, m
G: k" ..... X: (oc 1 , ... ,oc~~
IOCkgk,
OC i E k,
k=l
Now let X w be the k[z]-module induced by W(z) as in the Representation Theorem. Our question is: how are the invariant factors {I/JJ associated with X w related to the invariant factors of W(z)? The answer is provided by
9.5
259
TRANSFER FUNCTIONS
Theorem 9.3 (A;, "')
Let {AI' A2 , ••. , Aq } be the invariant factors of ",Wand let 1,2, ... , q. Then the invariant factors of X ware
= 0;, "C =
"'I = '"
"'2 = "'/02 "'3 = "'/03 where r = smallest integer such that'" IAi for i = r
+ 1, ... , q = rank wW.
Viewed another way, we have since the invariant factors of W must be equal to those of X w (by the bijective correspondence W ~ X w). Thus, we see that ljJi = denominator of AJIjJ after cancellation of all common factors. The invariant factor theorems enable us to address the question of when a system ~I can be "simulated" by a system ~2' i.e. when the desired external behavior of ~I can be reproduced by altering the dynamical behavior of ~2 through feedback-free coding of its inputs and outputs involving delay. In other words, we want ~2 to act like a machine with transfer function WI by re-coding its inputs and outputs. To make this problem precise, we have
Definition 9.7 ~11~2 (i.e. ~I can be simulated by ~2) if and only if X1:,IX1:2' i.e. if and only if X 1:, is isomorphic to a submodule of X 1:2 (or isomorphic to a quotient module of X1:,)' The main result governing the issue of whether or not ~11~2 theorem
is the classical
Theorem 9.4 Let R be a principal ideal domain with X and Y R-modules. Then YIX if and only if i
=
1,2, ... ,r(Y).::; r(X).
The above criterion settles the basic simulation question in terms of the invariant factors of the respective state modules X 1:" X k Since X 1: ~ W, it is to be expected that a similar test involving the invariant factors of W1:,(z), WI:,(z) holds. To formalize this notion, we state
Definition 9.8 Let WI and W 2 be transfer functions matrices. Then WI IW2 , i.e. WI divides W; if and only if there exist matrices V, Z over k[z] such that WI = VW2Z.
260
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Now we can state
Theorem9.5
W11W2 ifand only ift/J;(Wi)It/J;(W2 ) , i
= 1, 2, ... ,r.
Putting Theorems 9.4 and 9.5 together, we arrive at the
Prime Decomposition Theorem/or Linear Systems
The followinq conditions
are equivalent:
(1) (2) (3)
Wi l W2 , t/J;(Wi)It/J;(W2 )
for all i,
~11~2.
REMARKS
(1)
The above results enable us to say that the dynamical behavior of ~2 can be arbitrarily altered by feedback-free coding of its inputs and outputs if and only if the invariant factors of the desired external behavior W:E, are divisors of the invariant factors of the external behavior W:E z of the given system ~2. Thus, we may regard the invariant factors as the basic building-blocks oflinear systems in that they cannot be simulated from smaller units by feedback-free coding. We shall make this notion mathematically precise and explicit in the next section. (2) To see how the "coding" operation needed to make ~2 simulate ~i works, consider the following procedure: the original input W 2 is replaced by Wi = B(Z)W2. Similarly, the original output Y2 is replaced by Yi = A(Z)Y2· Now ~2 will act like a system ~i with a transfer function W1 (z) = AW2B. However, this equation will still be satisfied if A and B are replaced by A, jj (i.e. A, B mod t/Jwz). Thus, the coding operations A and B can be carried out physically using only a delay of d = deg t/Jw z units, i.e. it is necessary only to store the last d inputs and outputs.
9.6 REALIZATION OF TRANSFER FUNCTIONS
Given a proper rational matrix W(z) whose entries have least common denominator t/J, the matrix t/J W is clearly a polynomial matrix which, by the invariant factor theorem, can be represented as t/JW= AAB,
where det A, det B are units in k[z], with A a diagonal matrix containing the invariant factors Ai' A2 , ••• , A,. If we replace each polynomial tt in the above
9.6
261
REALIZATION OF TRANSFER FUNCTIONS
representation by its canonical representative it in the class the alternate representation l/JW = PLQ mod
t:
mod
l/J, we have
l/J,
where det P, det Q are units in k[z]/k[z]l/J and Lis a diagonal matrix, unique up to units in k[z]/k[zJl/J. The canonical realization problem for transfer functions is to determine a system I: w = (F, G, H) such that (i) (ii)
W(z) = H(zI - F)-lG dim I: w = dim X w'
In this section, we develop an explicit procedure for computing such a realization from a given W. Let us begin with the following easy result.
Lemma 9.2
Let F be the companion matrix
F=
o o
o
o
o
o o
o
1
1
o
with characteristic polynomial l/JF(Z) = z" + IJ(lZn-1
+ ... + IJ(~.
Then l/Jiz)(zI'- F)-l
= v(z)w'(z) mod l/JF'
where
v(z)
=
UJ
z
w(z) =
r
1
n _
+ 1J(1Zn-2: + ... + IJ(n-l z
.
+ 0(1
1 .
1
The proof is immediate by a direct calculation. The canonical realization of W(z) now proceeds according to the following steps: (1) (2)
(3)
Compute the invariant factors {l/J;} of W(z). Calculate the representation (*) for l/J W(z) and write L = diag(t 1 , t 2 , · · .), Pi = ith column of P, q; = ith row of Q, fli = (t;, l/J). For each invariant factor l/Ji of W, let F, = companion matrix of l/Ji' i.e. l/JFi = l/Ji' Let v;(z), w;(z) be the associated polynomial vectors from Lemma 9.2.
262
(4)
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Solve the pair of equations
Hi», = (t' JI-l;)p; mod
0/;,
i
= 1,2, ... , q,
wiG; = qi mod 0/;. The mod 0/; operation allows us to assume that the elements on the right side of the above equations have deg < deg 0/;; hence, the equations for H; and G; have unique solutions. (5) Write down the realization as F
= diag(F r- F 2 ' " ' ' F q ) , H=[H 1,H2,···,Hq ].
The general structure of a linear system is formalized by the following realization theorem. Theorem 9.6 Every proper rational transfer function matrix W may be realized as the direct sum of the systems
L; = (F;, G;, H;), where F; is a cyclic matrix with characteristic polynomial computed as in the realization algorithm above.
0/; and G; and H; are
FIG. 9.1 General diagram of a linear system.
9.7
263
THE CONSTRUCTION OF CANONICAL REALIZATIONS
r---- ---- ------I
• •
• •
• •
• • 1---.,.--. Yp
I
I JI
I L FIG. 9.2 Conventional realization of W(z).
Schematically, the above realization looks as in Fig. 9.1. We note that the above schematic makes clear the high degree of internal connectivity in a canonical realization of W. Figure 9.1 should be contrasted with Figure 9.2, in which a conventional realization of W is displayed. Such a realization results from an arbitrary choice of connections and is almost never canonical. The solution to the realization problem via the invariant factor theorem provides complete information concerning the structure of a canonical realization. However, the cost of using such a method is high since computation of the invariant factors is both time-consuming and complex. Some alternate approaches are considered in the problems at the end of the chapter. In the next section, we shall consider the question: "if we don't want to know the invariant factors, but only the matrices F, G, H of a canonical realization, can we find a simpler algorithm than that provided by the invariant factor theorem?" 9.7 THE CONSTRUCTION OF CANONICAL REALIZATIONS
Abstractly, we may consider the realization problem as a factorization of the input/output map f: n --+ r through the state space X (see Fig. 9.3).
264
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
n.----L.- r
~}
X
FIG. 9.3
Factorization of the map
f.
Here we define a realization to be canonical if and only if the map g is onto, while h is one-to-one. Realizations always exist since, for example, we could take X = n, g = identity on n, h = f. Of course, this realization is both trivial and useless, as well as non-canonical. The module machinery has already shown how to construct a canonical realization from a transfer matrix, provided we are willing to compute the invariant factors of W. In this section, we pursue the problem of actual construction of a canonical realization from two directions: (1)
(2)
streamlining of the algorithm for construction of ~ from W without first computing invariant factors; direct construction of E from the behavior sequence B = {AI' A z,"'} induced by the map f as f(w)(1)
=
L A_t+lw(t),
t,;;O
where the A k are p x m matrices over the field k. Clearly if
(F, G, H) realizes I, then we have f(w)(1)
=
~
=
L HF-tGw(t),
t,;;O
so a realization of B is equivalent to finding a
~
such that
k = 0, 1,2, .... We have already seen (via the Correspondence Theorem) that any two canonical realizations must be related via a basis change in the state space X. Part of the justification for regarding an observable and reachable realization as being a "natural" requirement is the following easy result. Theorem 9.7 dim ~ is minimal over the class ofall realizations off ifand only is a canonical realization off.
if~
PROOF Let t be any realization and let ~ be a canonical realization. Then f = h a (J and therefore h(X) ::J range f. But X ~ range f, since f may be factored canonically as n -+ range f -+ r, where the second map is the natural injection. So
dim
t == dim X 2 dim range f = dim X == dim
~.
In order for equality to hold in the middle expression, we must have range f, i.e. h is one-to-one. Moreover, range {J ~ X (since otherwise
h(X) ~
9.7
265
THE CONSTRUCTION OF CANONICAL REALIZATIONS
f =I- hog), which implies g = onto. Thus, dim X minimum implies that the factorization is canonical. In order to streamline the realization algorithm given earlier, we begin with a non-canonical, but easily obtained realization of the transfer function matrix W(z). Let us write
lfrW(z) where
= zq-1R o + zq- 2R 1 + '" + R q- 1,
lfr(z) = zq + t/t1zq-l + ... + lfrq
is the monic least common denominator of all elements ~iz) of W(z). The p x m matrices {RJ are, of course, constant. Then it is easily verified that a realization of W(z) is 0 0 0 [ 0o I. 0 i;
F=
-t/t~lm
- t/tq- 11m
H = [Rq- 1 Rq- 2
RoJ,
'"
-:.J
G=
0 1m
where 1m = m x m identity matrix. It is easy to check that the system 1: = (F, G, H) realizes W(z) and is even completely reachable. The problem is that, in general, the above realization is far from completely observable; hence, it is not canonical (unless m = 1 and W(z) is irreducible). To "reduce" the above ~ to a canonical realization, we make the simplifying assumption that the denominators of the elements wij(z) of W(z) have only simple zeros Zl' Z2"'" z.; Define the matrix K, = lim(z - z;)W(z),
i
= 1,2, ... , s,
and let r, = rank K i . By definition of rank, there exist (p x r;) and tr, x m) matrices L, and M i , respectively, such that K;=LiM;.
Now we can state the easy result Theorem 9.8
A canonical realization of W(z) is given by
P=
diag(z l l " , z21" , ... , z s l ,) ,
266
9
PROOF
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
A direct computation shows that the above! = (ft,
that
s
L K;/(z -
H(zI - F)- 1G =
G, H) is such
= W(z),
z)
i+ 1
by the assumption on the zeros of the denominator of wij(z). To prove that ! is canonical, we must show that it is reachable and observable. The reachability matrix is
C
=
M1 M2 .
Z2M2
zi M 1 ziM 2
zsM s
z; u,
Z1
r
Ms
M1
z~
•. •
-1
M1]
zi- 1M 2
•••
,
z:-1M s
where n = r 1 + r2 + ... + rs • Since rank M, = r i and the z, are distinct, rank C = n implying ! is reachable. The observability condition is proved similarly. EXAMPLE
Consider the transfer matrix 1 [ (Z2 + 6) W(z) = t/t(z) (2z 2 - 7z - 2)
t/t(z) =
+ 2z 2 -
Z3
(Z2 (Z2 -
+ Z + 4)
J
5z - 2) ,
2.
Z -
The elements of Wall have the same denominator t/t(z), which has the simple roots Zl = 1, Z2 = -1, Z3 = -2. Computing the matrix K 1 , we have K 1 = lim (z - l)W(z) = [_ z-1
~6
-
~J,
so that rank K 1 = r 1 = 1. Furthermore, K 1 can be factored as
K1=L1M1=[_~JG
1].
Similarly, we find r 2 = 1, r 3 = 1, with K2
~
L2 M
2
=
K 3 = L 3M 3 =
eJ r-r
[2IJh-
10
Hence, a canonical realization of W(z) is
F = diag(l, - 1, - 2),
_
G-
[
-~
~ 10
3'
-2],
2].
1]
-2 , 2
9.7
THE CONSTRUCTION OF CANONICAL REALIZATIONS
267
We have already pointed out that the k[z]-homomorphism f: n ~ r induces a behavior sequence B = {AI' A 2 , A 3 , .. . }. On the other hand, if we are given a transfer matrix W(z) (which, as we know from the last section is equivalent to f), we can expand W(z) about z = 00 to get
+ L 2z- 2 + L 3z- 3 + "', matrices. Since f and W must
W(z) = L 1z- 1
where {L;} are p x m constant same input/output behavior, we must have i
generate the
= 1,2, ....
Thus, any procedure for generating a canonical realization for W(z) must be capable of generating a similar realization for B and conversely. We have already seen how to canonically realize W, so let us turn to the realization ofB. Definition 9.9 A dynamical system 1: = (F, G, H) realizes the infinite matrix sequence B = {AI' A 2 , ••• } if and only if
k = 0, 1,2, ....
Ak+1 = HFkG,
The mathematical "gadget" we need in order to deal with the realization of B is the (infinite) Hankel matrix associated with B, denoted .:!e(B). Explicitly .:!e(B) =
:
r~: ~: ~: :1· A3
..
A4
As
..
..
...
We let .:!ell, v denote the fl x v block submatrix of.:!e appearing in the upper left-hand corner of Jf'. Basically, the utility of the Hankel matrix .:!e in realization theory comes from its role as a matrix representation of the map f: n -+ r, when WEn is regarded as an infinite column vector with elements (Wl(O), ... , Wm(O), w 1 (1), . · .). Realizations of B and the properties of .:!e(B) are related through the following proposition. Theorem 9.9
Let 1: be any realization of B. Then rank .:!ell, v(B)
~
dim 1:
for all u; v ~ 1.
Corollary B has afinite-dimensional realization ifand only ifrank .:!eIl,iB) is constant for u; v sufficiently large. PROOF To avoid an empty (but formally correct) argument, assume dim 1: = n < 00. Define from 1: the block matrices
rev = [G, FG, Oil = [H', H'F',
, Fv-
1GJ,
, H'(F')i"-I].
268
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Then ()~'f6v
= H/ljB).
Since rank 'f6 v and rank ()/l ::; n = dim matrix fact
~,
the theorem reduces to the standard
rank(AB) ::; min{rank A, rank B}.
Now let us turn our attention to proving the "if" part of the Corollary, thereby producing an effective procedure (Ho's Algorithm) for canonically realizing B. For this we require Definition 9.10 The infinite Hankel matrix Yt'(B) has finite length A = (X, A") if and only if one of the following two equivalent conditions holds:
X = min{l: rank £;,v = rank £;+k,v
for all k, v = 1,2,
}<
00,
A" = min{m: rank Yt'/l,m = rank Yt'/l,m+k
for all k, Jl = 1,2,
}<
00.
We call A' the row length of Yt' and A" the column length of Yt'. The last ingredient we need is the notion of a shift of the sequence B. Definition 9.11 The (left) shift operator IJ'B on an infinite sequence B is given by IJ'~:
(Ai' A 2 , · · · )
--+ (Ai +k'
A 2 +k '
" .).
The corresponding shift operator on Hankel matrices is then IJ'~:
Ye(B) H Yt'(IJ'~B),
Since we have seen earlier that the shift operator is the algebraic equivalent of the dynamics F, it should come as no surprise that IJ'B will playa central role in realizing the sequence B. The connection between Ye having finite length and the shift operator IJ'B is that finite length is equivalent to IJ'B having finite-dimensional left and right matrix representations, i.e., there exist finite block matrices Sand Z such that IJ'~Yf'"./,,(B)
= SYe1',I,,(B), = Yt','.I'.(B)Z.
We are finally in a position to state the Ho Realization Algorithm Consider any infinite sequence B offinite length with associated Hankel matrix Yt'. Thefollowing steps will produce a canonical realization of B. (1)
(2)
Determine A', A" Compute n = rank Yt';(,;."
9.7
THE CONSTRUCTION OF CANONICAL REALIZATIONS
(3)
(4)
269
Determine nonsingular matrices P and Q of sizes pA' x pA' and ms" x mA", respectively, such that
(Note: P and Q are determined as part ofthe process ofdetermining the number n.) Write down the realization L = (F, G, H) as F = K nP[()H.1t';.'.).'.JQK n, G = K nP.1t').';.',K m, n, H = K p.1t').',)."QK
where K p , K'" are idempotent editing matrices having the effect: "retain first p rows" and "retain first m columns", respectively.
The most serious apparant drawback to the employment of the Ho algorithm is the need to verify the assumption that B is of finite length. If additional information is given in advance concerning B, then this requirement can sometimes be seen by inspection to hold, e.g. A k = 0 for all k > some fixed N or A k = coefficients of the expansion of a rational matrix function. However, even in the general case the finite length difficulty is more apparent than real as the next result demonstrates.
Rank Condition Theorem Let B be any infinite behavior sequence with corresponding Hankel matrix H. Suppose there exist integers rand s such that rank .1t'r.lB) = rank .1t'r+l,sCB) = rank £'r,s+l(B).
(t)
Then there exists a unique extension E ofB oforder r + s such that A~ ~ rand s. Moreover, applying the Ho algorithm with A' = r and A" = s produces a canonical realization of E.
Ai ~
So, we see that a canonical realization of some extension of B is always possible as soon as the rank condition (t) is satisfied. Furthermore, (t) can be used as a practical criterion for constructing a canonical realization for some B known to be of finite length, but without being given A' and A". If the Ho algorithm is applied without any information concerning the condition (n, then the system L produced will always realize some extension of B, at least of order 1. Details on how to determine the maximal order of this extension, at least for scalar behavior sequences, are given in the chapter references.
270
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Example (The Pelt Numbers):
To illustrate use of Ho's algorithm and the Rank Condition Theorem, let us construct a canonical realization of the sequence of Pell numbers B
= {1, 2, 5,12,29,70, ...}.
The second-order difference equation generating B is A n+ z
= 2A n+ 1 + An'
which suggests that the entire sequence B can be generated by a twodimensional canonical model ~. We now establish this result by He's algorithm. Consider the matrix Jt"11 = [AI] = [1]. It is easy to see that rank Jt"11 = 1 = rank Jt"21 = rank
[~J
= rank Jt"IZ = rank [1 2].
So, the sequence {A d can be canonically realized by a one-dimensional Applying He's procedure immediately yields
P = [1J,
Q = [1J
and
= [2J,
G = [1J,
H = [1J.
F
~l:
~.
The system ~l extends the sequence {l} to [I, 2, 4}, which does not agree with {AI' A z , A 3 } . So, in order to realize {AI' A z , A 3 } we must examine more data. Consider next the Hankel array Jt"z"z =
[~
~J
obtained from the se-
quence {AI' A 2 , A 3 } . We easily see that rank Jt"22 = 2 = rank Jt"Z3 = rank
[~ ~ 1~J
= rank Jt"32 = rank
[~ ~]
.
5 12
G~l
So, we can canonically realize {AI' A 2 , A 3 } with a 2-dimensional realization. Factoring Jt"Z2 by the Ho algorithm, we have PJt"Z2Q =
P
=
G-~J
Q=
[~
-
~J
The remaining steps of the algorithm yield 't'
'F[ -12 -
....2·
-1J
o'
G=[~l
H
= [1 0].
where
9.8
271
PARTIAL REALIZATIONS
After a few calculations, we can verify that ~2 produces the original sequence of Pell numbers B. This can also be seen by examining the ranks of the matrices ~'" r > 2. Of course, without knowledge of the recurrence relation for the {Ai}' we could never empirically check the Rank Condition Theorem for all Ai, since this would entail the examination of an infinite amount of data. This leads to the remaining interesting question: what can be said if the assumptions of the Rank Condition Theorem are not satisfied for a finite amount of data A 1 , A 2 , ••• , AN' and any r, s satisfying r + s = N? This "partial realization" problem is addressed in the following section.
9.8 PARTIAL REALIZATIONS
The basic flaw in the foregoing modeling theory via He's algorithm is the underlying assumption that the behavior sequence B has a finite-dimensional realization. But, suppose we are given an input/output map f and associated sequence B = {A1 , A 2 , ... }, but that we know or assume nothing about the finite-dimensionality of the map f. Suppose we pick numbers A', A" arbitrarily and compute a dynamical system according to Ho's algorithm. Question: What are the properties of such a Question: Is ~ canonical? Question: Does ~ realize a part of B?
~?
The answer to these questions is provided by the following
Partial Realization Theorem Let N, N' be integers such that rank ~N·.N
= rank
~N'+l,N
= rank ~N·.N+l'
and let No = N + N', Then a system ~ given by Ho's algorithm with A = N, At = N' is a canonical partial realization oforder No, i.e. it realizes the sequence B No = {A 1 , A 2 , . · · , A No } ' Furthermore, if the above rank condition does not hold for N, N', then every partial realization off has dimension> rank ~N. N' REMARKS
for partial realizations, "minimal" implies "canonical," but not conversely. In fact, "minimal" realizations are not necessarily unique. (2) In general, there is no way to say whether or not a partial realization of f is useful or not. For example, consider the transfer function matrix (1)
W(z)
=
W1(z) + Z- hW2(Z),
with h much larger than the degree of the common denominator of W1 • The effect of the second term will be absent from JIt'", for all
272
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
l/Jw, ~ h. Thus, we get a minimal realization Ho method and, since
r ~ deg
W(z) - WI:,(z)
=
W(z) - W1(z)
=
~1
of WI by the
z-hWz(z),
it is clear that the minimal realization of the remainder has dimension equal to dim z-hWz(z). The interesting aspects of the partial realization problem arise when the rank condition (*) is not satisfied. Thus, we have the partial sequence B M = {AI' A 2 , ••• , AM},
and our problem is to extend this sequence (hopefully uniquely) to an infinite sequence which can be described via the compact triple ~ = (F, G, H). Let us define n 1 (M ) = the number of linearly independent rows in the block row [A l' ... , AM], nz(M)
= the number of linearly independent rows in the block row [A 2 , ••• , AM]' which are also linearly independent of the rows in the block row [A 1 , ... ,AM - 1 ] ,
nM(M)
= the number of linearly independent rows in
AM which are also linearly independent of the rows of the matrices AI, A 2 ,
···,
Am-I'
=
n 1(M)
Define n(M)
+ n2(M) + ... + nM(M).
It can be shown that the dimension of the minimal realization of any extension of B M is always ~ n(M). Similarly, we can obtain lower bounds for the values of N, N' for which the rank conditions can be satisfied for any extension of B M as follows: N'(M)
= the first integer such that every row of the block row [AN'
+ l ' ... ,
AM] .is linearly dependent on the rows of
YfN',M-N"
N(M)
= the first integer such that every column of the block column
is linearly dependent on the columns of YfM-N,N'
9.9
273
POLE-SHIFTING AND STABILITY
The surprising and happy fact is that all of the above lower bounds can actually be obtained as the following result illustrates. Main Extension Theorem Let B M be a fixed partial sequence with the integers fl(M), N(M), N'(M) as defined above. Then
= dim L M , where L M = (F M' G M , H M) is the minimal realization of any extension of B M . (ii) N(M), N'(M) are the smallest integers such that the rank condition (*) holds for some extension of B M ; (iii) N(M), N'(M) are the smallest integers such that the rank condition (*) holds for all minimal realizations; (iv) there is a minimal extension of order M' = N + N' for which the rank condition (*) holds. The realization of this extension can be carried out (non-uniquely) by the Ho algorithm; (v) every extension of B M which is fixed up to M' is uniquely determined thereafter. (i)
fl(M)
Only recently was the partial realization problem completely and definitively settled for scalar behavior sequences. Since the details would take us too far off the main course of this chapter, we do not present them here and refer only to the chapter notes and bibliography.
9.9 POLE-SHIFTING AND STABILITY
Given the dynamical system
x=
Fx
+ Gu,
x(O)
=
c,
(L)
one of the oldest system-theoretic questions is whether or not L is asymptotically stable, i.e., if c i= 0, does x(t) -+ 0 as t -+ oo? Clearly, if the characteristic values of F lie in the left half-plane, then L is stable with no control needed. Furthermore, if F has some unstable roots, then the classical Pole-Shifting Theorem tells us that if the pair (F, G) is reachable, we may choose a feedback control u(t) = - Kx(t), so that F - GK has prescribed characteristic roots. The module-theoretic machinery developed above allows us to extend the Pole-Shifting Theorem to the case of an arbitrary field k. For a single-input system, the statement of the result is Pole-Shifting Theorem (single-input version)
Let k be an arbitrary field with
X a cyclic k[z]-module with generator g and minimal polynomial X(z) = z" + O(lZn-l + ... + O(n' deg X = n. Then there is a bijection between nth degree
polynomials n(z) = z" + f3 1zn- 1 + '" + f3nEk[Z] and k-homomorphisms {:kn-+kn:X(j)·gf-+{i·g (j= 1, ... ,n), such that tt is the minimal polynomial
9
274
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
for the new module structure induced on X by the map z.: x ~ z- 9 - t(x). (Here the elements xU> are defined by
en = 9 = z·g = X(ll(z)·g en - l
= z·g + IXl·g =
X(2 l (Z) · g
REMARKS
X cyclic with generator 9 means, of course, that the underlying system ~ is completely reachable. (2) The map t defines a control law for ~. The passage from z to z; is the module-theoretic version of the passage from open-loop to closedloop control. (3) Extension of the result to multiple-input systems is possible, at the expense of considerable additional algebraic machinery. The references should be consulted for details. (4) The choice of control t j = Pj - IX j induces the correspondence n <--+ t, which is the stated bijection of the theorem. (1)
9.10 SYSTEMS OVER RINGS
The development given in the preceding sections has assumed that the matrices of the system ~ = (F, G, H) have their coefficients in some field k. As it turns out, most of the important results involving construction of canonical realizations, system equivalence, pole-shifting and the like can be carried through under much more general conditions. In particular, instead of fields a rather complete set of results can be obtained when the coefficients belong to a commutative ring R, where addition and multiplication are defined, but not division. The need for such a generalization arises immediately when we want to consider problems in which the behavior sequence B = {Al , A 2 , ••• } is such that the elements {AJ have only integer entries, i.e., each Ai is an integer matrix. Such a situation naturally comes about if we discretize measurements on the input/output behavior and then agree to express all quantities as integer multiples of a basic "quantum" of measurement. It is then natural to inquire as to whether or not the behavior sequence B has a canonical realization over R = Z, the ring of integers. In this section, we only motivate the need for this more general theory by a few examples, some of which will be used in Chapter 10 on distributed parameter problems. In what follows, we shall make the important restriction that R = integral domain, so that a natural field can be associated with R (the field of quotients Q of elements in R).
9.10
275
SYSTEMS OVER RINGS
One of the most important examples of a system over a ring comes about when we have a process involving time-delays. Consider the system dynamics Xl
=
2x 1(t - 1) + X1(t)
+ Xz
X2 = X1 (t - 1) y(t)
=
x 1(t ) - x 2(t - 1).
If we introduce the delay-operator O"(x)(t)
== x(t - 1),
then the above system can be rewritten as
1] [ x~ 1 ] = [20" + 1 -30"1 s][X + [1]u, 0"
2
0"
X2
or X = F(O")x
+ G(O")u,
y = H(O")x.
So, we see that the delay system can be expressed in the same fashion as for an ordinary system without delays, the only difference now being that the elements of F, G, H belong to the ring of polynomial in 0", where k = reals. For the more general system X y(t)
=
=
r
L F;x(t -
(X;)
;=1
+
s
L Gju(t - P),
j=l
v
L Hkx(t -
Yk),
(X,
k=l
P, Y ~
0,
we can apply the same procedure if all delays (x, P, yare integral multiples of a fixed delay .A.; if not, we need to define a finite set of delay operators 0"l' 0"2' ••• , 0"N and then consider systems whose coefficient matrices F, G, H have entries in k[O" 1,0" 2, ••• ,0"NJ, the set of polynomials in the N indeterminates 0"1' ••• ,0"N- We shall consider such delay, or time-lag, systems in greater detail in the next chapter. Another type of situation in which general rings playa vital role comes about when we use lumped approximations to linear partial differential equations. For example, consider the heat equation ou(x, t) _ 02U(X, t) at ox2
+
f
)
(x, t ,
xER.
276
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
If we discretize the space variable x and introduce an operator (O"X)i == Xi + l' where Xi represents X at the ith integer, then the above system becomes Ii =
(0" -
2 + O"-l)U
+ I(t),
where u and I are infinite column vectors with entries u., Ii' Thus, what we have obtained is a system over the ring R[ 0", 0"-1]. If we write the above system in matrix form Ii
= Fu + I,
then F is Toeplitz. If X E s', the unit circle, instead of R, then F will be a circulant. Another important class of problems for which the ring setting provides the right level of generality is when we deal with a system involving more than one independent variable in its dynamics, the so-called n-d systems. For example, the 2-d input/output relation
y(h, k) =
h
k
L L
Aiju(h - i, k - j),
h, k = 0, 1, 2, ...
i=O j=O
has the internal model
+ 1, k) = F llx 1(h, k) + F 12xz(h, k) + G 1u(h, k), x 2(h, k + 1) = F 21X1(h, k) + F22X2(h, k) + G2u(h, k), y(h, k) = H 1x1(h, k) + H 2x2(h, k), Xl ERn" X2 ERn2. xi(h
The 2-d transfer matrix associated with this system is
W(Zl,Z2) =
I
i,j
Aijzlizz-j,
or
The obvious realization question-is whether every proper rational matrix like W(Zl' Z2) can be canonically realized by an internal model Ej, = (F ij , Gi , H;) of the above type. A simple way to deal with this question is to treat one of the elements z, as a parameter, and then to consider the problem as a realization over a ring. For instance, let us consider z 1 as a parameter and let R Z' be the ring of all proper rational functions in z l' We now consider W(z l' z 2) as a proper rational function in Z2 with coefficients in R Z 1 ' This transfer matrix can be realized over R Z,' giving matrices F(z 1)' G(z 1), H(z 1)' We can now regard each of these proper matrices over z 1 as a new transfer matrix to be realized over whatever
9.10
277
SYSTEMS OVER RINGS
field of coefficients we are using (determined by the coefficients of W(Zt, zz), regarded as a 2-d transfer matrix). This subsequent realization of F(zt), G(Zt), H(z t) produces new triples F(Zt)
--+
(FF' GF, H F),
G(Zt)
--+
(F G' GG' H G),
H(zt)
--+
(F H, G H, HH)'
It can then be shown that a realization of W(Zt, zz) is then given by
o
HF
HG
FF
0
o o o
FG FH
0
0 0 0 0 0
0 0 0 0 0
This procedure has obvious extension to n-d systems involving n dependent variables (Zt, Zz,"" zn)' The main theorem governing realizations over rings is the Ring Realization Theorem
if and
only
if it
An input-output map f is realizable over the ring R is realizable over the quotient field Q of R.
Since only integral domains have quotient fields, the outstanding question is whether or not there exists any reasonably general criteria that can be applied to identify domains whose quotient field is realizable. The ring of integers Z can easily be seen to be such a domain, but are there others? As a result of the work of Sontag, Rouchaleau and others, we know now that every Noetherian domain satisfies the realizability condition. Further results, including papers showing that not every R can be used as a coefficient ring, are given in the chapter references. Before closing this discussion on systems over rings, it must be noted that many of the most important results on systems over fields fail to hold, at least in their strong form, when we pass to rings. For instance, we have seen earlier that all minimal realizations for the same input/output map are necessarily isomorphic, i.e., there exists aTE R" x n such that if L t = (F r- Gt , H t) and LZ = (F z, Gz,H z), then FzT=TF t, TGt=G z, HzT=H t, T-tER n x n.
278
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
This is true if R = field, but false otherwise. A simple counter-example is to let = (a, a, a, ... ),1: 1 = (1, a, 1),1: 2 = (1, 1, a) and let aER, a oF 0, a- 1 ¢R. Then for such a ring R, 1:1 and 1:2 are non-isomorphic minimal realizations of B. In fact, it can be shown that there may exist an uncountable number of nonisomorphic minimal realizations. For details on these and many other related questions, the survey by Sontag cited in the chapter references should be consulted. B
9.11 SOME GEOMETRIC ASPECTS OF LINEAR SYSTEMS
Our approach to the study of linear systems has been resolutely algebraic. There are several justifications for taking such a path: (i) the algebraic machinery enables us to unify the time-domain (state variable) and frequency-domain (transfer function) views of linear systems. (ii) algebra is congenial for computation. Since digital computers cannot, in general, represent real numbers, the methods and spirit of analysis (i.e. limiting operations) cannot be directly carried over to computational algorithms. On the other hand, algebra deals with mathematical abstractions which are closely related to computer organization and programming. (iii) the main emphasis in algebra is upon the construction of new mathematical objects from given objects in a natural (i.e. canonical) way. This is exactly what system theory is all about; namely, the determination of"good" models from given data. Notwithstanding these strong arguments for adoption of an algebraic outlook, there are a number of additional insights to be gained about linear systems by also examining them with geometric tools. As always, no single approach can be expected to be uniformly most powerful and, as we shall see, some ofthe most interesting and powerful methods of modern geometry have direct bearing upon important structural aspects of linear systems. Furthermore, the geometric methods, just as the algebraic, hold promise for providing a suitable foundation upon which to launch a concerted attack upon many nonlinear processes. In the next few sections we shall see that holomorphic vector bundles, Grassmann manifolds and almost-onto maps may all be effectively employed to study the global structure of linear dynamical systems. Let us begin our study by introducing the notion of a vector bundle for the system 1: given by its transfer matrix W(z) = H(zl - F)
9.11
279
SOME GEOMETRIC ASPECTS OF LINEAR SYSTEMS
numbers, obtaining in the usual way the Riemann sphere S2 = ~ U {oo}. Now rewrite the transfer matrix W(z) as zx = Fx
+
(£)
Ga,
.p = Hx where X, a, .p are, of course, the transformed variables. Since we will be interested in the point z = 00, we can substitute z = l/r and let 't" -+ enabling us to rewrite the above relations as
°
x = 't"(Fx
+ Gu),
.p = Hx, or, as 't" -+ 0, X = 0, .p = 0. Define the three sets E = =
E'
{(z, x, a, .9): £ is satisfied for z E~}
«00,0, a, 0) if z
= oo],
= {(z, u, .9): for each z E~, there exists x E X such that £ is satisfied}
a, 0) if z = {(z, x, U): if z E~,
= {( 00,
E" = =
oo}, £ is satisfied}
{Coo, 0, a) if z = oo}.
Since E c S2 X X X U x Y, E' C S2 X U X Y, E" C S2 X X X U, the "bundles" E, E', E" can be given various topological structures as needed. Also, in all three cases we define projections onto the first factor, e.g. n: E -+ S2, (z, x, a, .9) f--+ z. Note that the fibers of these maps n-1(z), n' -l(Z), n"-l(z) are vectors spaces. For instance, n-1(Z) = E(z) = {(x, a, .9): £ is satisfied at z = z}. E(Z) is a subspace of X EEl U EEl Y, which we may identify with X x U x Y. The property that the map rt: E -+ S2, with fibers as vector spaces means that the spaces E, E', E" with maps n, n', tt" are vector bundles. Note also that the z-dependent solutions of £, essentially the transfer matrix W(z), form cross-sections of these bundles. Before proceeding, it is necessary for us to digress for a moment and consider the procedure for attaching coordinates to a subspace of a vector space V. The set of all m-dimensional subspaces of V is called the Grassman variety of V and is denoted GmeV), and it plays an important role in the global algebraic theory of linear systems. Let wo , w1, ... , Wm be a basis for an (m + Ij-dimensional subspace Wm + 1 EGm+leV). Suppose we have dim V= n + 1 and assume i = 0,1, ... ,m
280
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
relative to some fixed basis in V. Set 1t io• i " ...• im
= det
[xXj.~:
xi?' ... x!~)
'0
xXj.~:],
'm
where 0 ~ i j ~ n. Not all a's are zero since the vectors W o, ... , W m are linearly independent. Also, interchanging any two indices changes the sign of 1t io ... im and if two indices are equal z., ... im = O. Thus, these quantities are uniquely determined if we know those with io < i 1 < ... < im • It can be shown that the numbers 1t io ... im can be used as coordinates of a point in a projective space PN(k) (k = any field) of dimension N = (::.~ D- 1. Furthermore, this point only depends on the subspace Wm + iThe definition of 1t io ... i; gives us a well-defined 1-1 map '1': (m
+ Ij-dimensional subspaces of
Wm + 1
The quantities
are called the Plucker coordinates of Wm + 1-
Is 'I' onto?
Question
Answer.
1t io ... im
V~PN(k).
It is not!
The term '¥(Gm + l(V» is contained in a proper algebraic subset of PN(k). To describe this subset, introduce the new variables Zio ... im ' where 0 ~ ij ~ n. We impose the following conditions on the z's. (i) (ii)
Zio
im
zio
im
= 0 if two indices are the same; reverse sign if two indices are interchanged; m
(iii)
z·10··· lm . z·.10···)m .
= "L..
k=O
z·· lk1l
.
.
.
..
.
... lm Jo ···}k- tIO}k+ 1"')m'
The quantities Zio ... im with io < i 1 < ... < t; are homogeneous coordinates of a point in P N(k) (if all these quantities are not zero). Conditions (i)-(iii) constitute a set of homogeneous. polynomial equations determining an algebraic set A m + 1 in PN(k). This algebraic set is called the Grassmann variety associated with the (m + 1)-dimensional subspaces of V. Am + 1 consists of the points (Zio ... iJ, i o < i 1 < ... < im , satisfying conditions (i)-(iii). In general A m + 1 is a proper algebraic subset of PN(k). For example, in the case m = 1, n = 3, conditions (i)-(iii) reduce to the single equation We can regard this as a quadric in P 5(k) since N = 5 and it is clearly a proper subset of P 5(k).
9.11
SOME GEOMETRIC ASPECTS OF LINEAR SYSTEMS
281
It is a nontrivial step to prove that the map
is onto. This fact, however, establishes a 1-1 onto correspondence between the points of cr: leV) and A m + 1> allowing us to conclude that A m + 1 parametrizes o-: leV). The set of all subspaces of V is denoted as G(V) =
Un + l
m=1
A
m"
We now utilize the notion of a Grassmann space in order to define a canonical or "standard" vector bundle E. Let V be a vector space and let G(V) denote its associated Grassmann space. A point of E is a pair (y, v), y E G(V), v E y. The projection it: E.... G(V) is given by it(y, v) = y. Thus, the fiber of E above the point y in the base is just y itself regarded as a vector space. The last elementary concept we need is the notion of a pull-back of the canonical bundle via a map ¢. Let ¢: X .... G(V) be a map of any space X into G(V). We can define a vector bundle if with base X by letting a point of if be a pair (x, v), x E X, V E ¢(x). We map if into E as (x,
v)~(¢(x),
v).
Diagramatically, we have
and the above definitions show that our diagram of maps is commutative. The vector bundle E is said to be the pull-back of the canonical bundle via the map ¢. The point of introducing ¢ is to have a way of replacing the vector bundle if by the map ¢ from the base space of if to a "standard" space G(V). It turns out that the topological and" algebraic properties of such maps ¢ are of considerable importance in the study of invariants of vector bundles. Now let us show how the above machinery may be employed to deal with some standard linear system questions. Consider the system £ above and let V = X x U, W = X. Define the map y: S2 .... G(V)
by the rule y(z)
=
{(x, u): (F - zl)x
+ Gu = OJ.
282
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Then we have that the vector bundle E" introduced above is a pull-back of the canonical bundle E via the map y. The following result can now be established. Theorem 9.10 If the system £ is reachable then dim y(z) = dim U for all z E S2, i.e. the bundle E" is non-singular.
This result is important since it establishes a direct connection between a basic system property-reachability and a bundle property-nonsingularity (i.e., all fibers of E" have the same dimension). We shall use this same idea now to show that reach ability and observability together (i.e., £ canonical) imply another interesting bundle-theoretic property. Consider the two bundles E' and E" with the fibers denoted E'(z) and E"(z), i.e. E'(z)
=
{(u, y): there exists an x such that £ is satisfied for each Z
E"(z) = {(x, u): zx = Fx
(Here we omit the Now let
A
E ~},
+ Gu}.
symbol for ease of notation.)
h: E" --+ E' be the linear bundle map given by h(x, u)
= (u, Hx),
(x, u) E E"(z).
We now have the Bundle Isomorphism Theorem If £ is canonical, i.e. reachable and observable, then h: E" --+ E' is a bundle isomorphism. PROOF By construction, the map h is onto E'. We must only show that it is one-to-one on E". First of all, since £ is reachable, E" is non-singular. Let (x, u) E E"(z) be such that h(x, u) = 0,
i.e. u = Hx = 0.
Then zx
= Fx, implying x is a characteristic vector of F. Also, HFx =zHx = 0, HF 2x=0, ....
Observability now implies that x = 0, showing that h is 1-1, hence, an isomorphism.
9.12
FEEDBACK, THE MCMILLAN DEGREE, AND KRONECKER INDICES
283
Another way of looking at the above result is to introduce the pencil of matrices A(z) = [F - zI, G]. Clearly, E"(z) = ker A(z). The bundle E" is called the kernel bundle of A(z) on 8 2 . The theorem then says that the map h from the kernel bundle of the internal description of the system is isomorphic to the bundle E' generated from the external description if an only if £ is canonical. 9.12 FEEDBACK, THE McMILLAN DEGREE, AND KRONECKER INDICES
So far, we have spoken about a single system 1: characterized by its transfer matrix W(z). Now let us consider the question of when two systems 1: = (F, G, H), 1:' = (F', G', H') are equivalent (i.e. transformable one to the other) under the following group of transformations: (I)
state coordinate changes T:(F, G, H) ....... (TFT- 1 , TG, HT- 1 ) ,
(II)
input coordinate changes V:(F, G, H) ....... (F, GV-t, H),
(III)
output coordinate changes U: (F, G, H) ....... (F, G, UH),
(IV)
state feedback L:(F, G, H} ....... (F - GL, G, H).
A little bit of algebra quickly shows that under the above feedback group, the respective transfer matrices of the systems 1:, 1:' are related as W'(z) = VW(z)U(I - LVW(Z)U)-l.
The important point to note is that the transformation W --+ W' is a fractional linear transformation, suggesting that the natural setting in which to study feedback is a Grassmann manifold, since the Grassmann manifolds are the homogeneous spaces of the group of fractional linear transformations. For each W(z) we shall construct a map
cPw: 8 2 --+ Gm(~m+p), where Gm(~m+p) = Grassmann manifold of m-dimensional subspaces of + p)-dimensional complex space. We shall see that 1: and 1:' are then feedback equivalent if and only if cPw and cPw' are transformable into each other under the group GL(p + m,~) acting on Gm(~m+p). Such an approach also enables us to define an algebraic invariant of the map cPw' which turns (m
284
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
out to be the classical McMillan degree of W(z), enabling us to characterize systems in the same equivalence class. In another direction, since we have already seen that Gm(~m+p) has a canonical vector bundle structure, we can use ¢w to construct a new bundle, the pull-back, on S2, We shall also see that feedback equivalent systems determine isomorphic vector bundles. Furthermore, since a complete classification of holomorphic vector bundles on S2 is known (via the work of Grothendeick), we obtain a complete set of feedback invariants which correspond to the well-known Kronecker invariants of 1:. Let us now proceed to construct ¢w. For each z E~, define ¢w(z)
= {(W(z)u,
u): u E ~m}.
¢w is defined everywhere except at the poles of W(z). Using the (not entirely trivial) fact that there exist polynomial matrices N(z), D(z) such that W(z) = N(z)D-1(z), we can extend ¢w(z) to all ~ by defining ¢w(z)
=
{(N(z)u, D(z)u): u E ~m}.
By setting
¢w(OO) = {(a, U):UE~m}, it can be shown that ¢w is defined everywhere on S2 and takes values in (since ¢w(z) is a subspace of ~m x ~p). At this point, it is instructive to examine the effect of a feedback law L on the transfer matrix W(z) and the consequent action through ¢w(z) in Gm(~m+ P). A small calculation shows that a feedback law L transforms Gm(~m+p)
W(z) --+ W(z)[I
+ LW(z)r 1.
Now consider the matrix
acting on ¢w(z). We have A(¢w(z))
=
{(W(z)u, (LW(z)
+ I)u): u E ~m}.
Putting these computations together, we see that A(¢w(z))
=
¢w'(z),
with W' = W(I + LW)-l, whenever (I + LW)-l exists. Thus, feedback is determined by a linear action on a Grassmann manifold. It can be shown that the other coordinate changes in the feedback group can also be realized by corresponding linear actions, as well.
9.13
SOME ADDITIONAL IDEAS FROM ALGEBRAIC GEOMETRY
285
To define geometrically the McMillan degree of W(z), let r = (p;m) and let = the projective space for which ftr is the space of homogeneous coordinates. The transfer matrix W(z) determines a rational map S2--+ P r - 1 (ft) and the McMillan degree fJ of W(z) is equal to the number of intersections of 4J(P 1( ft » with a certain hyperplane n in P r - 1 (ft). Without going into more detail, it is difficult to describe n, but the important point is that the algebraic quantity fJ(W), usually defined as fJ(W) = deg[det D(z)], can be defined in purely geometric terms, as well. As is well-known, <5(W) = dim E. Relative to the problem offeedback equivalence, we can state that ~ and E' are feedback equivalent if and only if <5(W) = <5(W'). Now let us return for a moment to vector bundles. We have already seen that the vector bundles associated with a transfer matrix W(z) and its canonical model ~ are isomorphic. Now what about the case when we have two input systems ~ = (F, G, -), ~' = (F', G', -) (here we neglect the output) which are feedback equivalent, i.e. Pr-1(ft)
F'
= T(F + GL)T- 1 ,
G' = TGV.
What is the connection between feedback equivalence and the properties of the vector bundles corresponding to ~ and ~'? This question is answered by Theorem 9.11 The input systems ~ and ~' are feedback equivalent if and only corresponding bundles are isomorphic.
if their
We have assigned a holomorphic vector bundle E' with base S2 with each transfer matrix W(z) as E' = {(z, u, y): y = W(z)u,
This is just the pullback of 4Jw(z). By general results, every such bundle is isomorphic to a direct sum of line bundles and the isomorphism classes are in one-to-one correspondence with sets of positive integers k 1 ~ k 2 ~ ••. ~ k.; where k, = degree of the ith line bundle in the decomposition. It can be proved that the McMillan degree of the ith line bundle equals k;, which in turn equals the classical rth Kronecker index of the matrix pencil [F - zl, G]. Thus, the set {k;} forms a complete system of arithmetic invariants for the feedback group, the same objects that we encountered earlier in Chapter Five. 9.13 SOME ADDITIONAL IDEAS FROM ALGEBRAIC GEOMETRY
According to one school of thought, the most efficient way to study constant linear systems is through the transfer matrix
286
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
a rational matrix function in the complex variable A. In quite another direction, the Thorn theory of catastrophes suggests that the canonical structure of a wide class of nonlinear systems is represented by a process whose dynamics x =f(x, u) are such that the functionf(·, -) is a polynomial oflow degree in the variable x and linear in u. In pursuing either of these lines of thought, we are led to consideration of the properties of polynomial rational maps between vector spaces or manifolds. The study of rational maps is the cornerstone of modern algebraic geometry, the tools of which have only recently been applied for system-theoretic purposes. In this section we outline the main concepts from algebraic geometry needed for our subsequent development. Let en denote the set of complex n vectors. We say a subset W of en is algebraic if there exists a set of polynomials P I(X), , Pn(x) such that W = {xEen:Plx) = 0, i = 1,2,
, n},
i.e., W is the solution set of polynomial equations Pi(x) = 0, i = 1,2, ... , n. A subset of C is said to be Zariski closed if it is a finite union of algebraic subsets. The complement of a Zariski closed subset is said to be Zariski open. The Zariski open subsets define a topology for en which, however, is non-Hausdorff so that some of our intuition about "nearness" gained from the usual topology for real and complex numbers must be given up. Basically, we can think of Zariski open subsets as being big, while a Zariski closed subset is small in C. A subset of C is said to be almost all of en if it contains a nonempty Zariski open subset. In classical algebraic geometry, a property depending upon points of C is called generic if the set of points where the property holds is almost all of en. For instance, the property that the determinant of an n x n complex matrix A is nonzero is a generic property. A map
4J:X _ en, where X is an arbitrary set, is called almost onto if 4J(X) is almost all of en. A mapping .
t.cr-;«: x J---+ p(x)/q(x), where p(x) and g(x) are polynomials functions is called a rational function on en. If we remove from en all those points at which q(x) = 0, we can regard f as a mapping ofZariski open subsets of en _ C. We let RF(C) denote the set of all rational functions on en, with PF( C) being the set of all polynomial functions on en.
9.13
287
SOME ADDITIONAL IDEAS FROM ALGEBRAIC GEOMETRY
We define a rational mapping cjJ: Cn -+ C m as an m vector (cjJl(X), ... , cjJm(x», whose components are rational functions on C". Using the rational mapping cjJ, we can define a ring homomorphism cjJ*: PF(Cm )
RF(Cn )
--+
by the rule cjJ*(p)
=
p(cjJ(x»
for a rational mapping cjJ. For example, if 2)
cjJ(x)
X
= ( x2 + 1 '
5x 3
X
+ 3x + 2
cjJ*(p(X» = p(cjJ(X» = 1 + X2
'
P(Yl' Y2) = 1 + Yl
+ y~,
2
+ 1 + ( 5x 3 +x 3x + 2 )2 .
x
We say a rational map cjJ is a submersion if the map cjJ*: PF(Cm ) is 1-1. The basic result needed for system-theoretic studies is
-+
RF(Cn )
Theorem 9.12 Suppose cjJ is a submersion. Let X be the set of points in en at which cjJ is well defined. Then cjJ(x) is almost all of em, i.e. cjJ is almost onto.
Intuitively, the theorem states that, except for a negligible set of y's, the equations cjJ(x) = Y,
XE
C",
yEC m
are solvable. Here negligible means a set of algebraic dimension < m. We note that Theorem 9.12 critically depends on the assumptions that x and Y are complex. The result is not true for real x and Y without further hypotheses, as the map f:R-+R,
shows. We would like to develop some simple, easily testable conditions under which a given map cjJ is a submersion. To this end, we consider the Jacobian matrix J(x) of the map cjJ, J(x) =
[:~;J
i
= 1,2, ... , n,
j
= 1,2, ... , m.
288
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
In terms of lex), we can give the following easy test for cjJ to be a submersion: Theorem 9.13 submersion.
If at some point x* E en, rank l(x*) = m, then cjJ is a
REMARK 1 In coordinate-free terminology, the role of lex) is taken on by the Frechet differential dcjJ of cjJ, which is defined by the formula
dcjJ(x*)(x) =
:t cjJ(x* + tX)!t=o
The theorem is then stated in terms of the rank of dcjJ(x*). REMARK 2 Note that the rank condition needs to be verified at only a single point of en for the theorem to hold. REMARK 3 These results imply that the image of cjJ is such that not only does em - 4>(X) have measure zero (in the Lebesgue sense), but it is contained in a finite union of algebraic sets of dimension less than m.
9.14 POLE PLACEMENT FOR LINEAR REGULATORS
Earlier we saw that the "poles" of the linear system
x = Fx + Gu, could be moved to arbitrary locations in the complex plane by linear feedback u = - Kx, if and only if the pair (F, G) is completely reachable. Now we want to consider a different version of this question and ask whether or not the same conclusion can be obtained if the feedback matrix K is the solution to the LQG optimal control problem with criterion
{Xl [(x, Qx) + (u, Ru)] dt,
R >0.
From Chapter Eight, we know that the optimal control law K satisfies K = -R-1G'P,
where P is the solution of the algebraic Riccati equation Q + PF
+ F'P -
PGR-1G'P = O.
Thus, if we assume F, G and R are fixed, our problem becomes: choose Q such that the closed-loop matrix has its characteristic roots at arbitrary predefined locations. Note that here we do not impose the customary condition Q z O. Furthermore, we shall admit the possibility that Q is complex.
9.14
289
POLE PLACEMENT FOR LINEAR REGULATORS
Since the LGQ problems is equivalent (via the Maximum Principle) to the system
(1) = (~Q y
=
(I
-G~;,lG,)G)
G)u,
+
O)G).
we consider the map
X
S;
--+ sp(n,
C),
given by
T[~
-GR-1G'] r:> -F'
,
where Sp(n, C) is the Lie group of complex symplectic matrices, sp(n, C), the algebra of infinitesimally symplectic matrices, and Sn is the space of complex symmetric matrices. The map tp is a polynomial map, so in light of the previous section we must determine whether or not it is onto. Calculating the Jacobian of
(:0
-G~~lG')]
+ (e5~
~).
where L·J denotes the usual Lie bracket (commutator) of matrices. Since dsp is onto if and only if Ud
U=[ ~p ~J where P is a solution of the algebraic Riccati equation. Defining UJTU- l , we now need only show that
+ [0 e5Q
J p -GR~lG' _ )p'
V('P, bQ) = [.'P, ( 0
is onto. If we partition
'P =
OJ
0
'P as
'P z )
-'P~
,
we obtain V(\}I, bQ)
=
P'P l =- 'PiP - G!-lG'\}I3 ( -F''P 'P F + JQ 3 -
3
P'P z
+ 'PzP' + GR-1G''P'l + 'Pi GR-1G') F''P'l -
'P'lP' - 'P
3
GR - 1 G'
.
290
Al
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
If Q is chosen so that if Al and A2 are characteristic values of F, then + A2 "# 0, and we see that V('P, e5Q) is onto if and only if the function
is onto. Since every symmetric matrix P is the solution of the algebraic Riccati equation for some Q,
A is onto, and A;(F) + AlF) "# 0,
i "# j.
We now show that the above conditions are satisfied by almost all systems (F, G). Instead of showing that A is onto, it is a little easier to deal with the adjoint of F, and show that A * is one-to-one. A simple calculation shows that
A*(B)
=
([F, B], - t(BGR- 1G'
+ GR- 1G'B')).
Thus, re-stating our earlier result about
if
there exists a
[F, B] = 0, BGR- 1G'
+ GR- 1G'B' =
0,
implies B == O. But, it is easy to verify that if (F, G) is not reachable there is a B "# 0 such that [F, B] = 0 and GB = O. Thus, if (F, G) is not reachable, then
The dual of this result then concludes our proof. Theorem 9.16 The set ofsystems (F, G)for which A* is one-to-one contains a non-empty Zariski open set, i.e., such systems are generic.
The implication of these results is that any desired behavior of the system may be coded for in the state weighting matrix Q, suggesting that the system co-states A can contain a substantial amount of information about overall system behavior.
9.15
291
MULTIVARIABLE NYQUIST CRITERIA
9.15 MVLTIVARIABLE NYQUIST CRITERIA
In Chapter Seven we saw that the Nyquist diagram provided a simple and effective geometric criterion for determining the stability of a closed-loop single-inputjsingle-output system. One of the great attractions of this criterion is that the geometric curve in the complex plane that is used to decide stability of a given feedback law is determined solely from the open-loop transfer matrix Z(A), i.e., the response diagram is independent of the feedback law u = - k' y. In this section we show how modern algebraic geometric ideas can be employed to generalize the Nyquist criterion for multi-input/multioutput systems. What we will show here is that the natural analogue of the response diagram is not a curve in the complex plane, but a curve in the Grassmann space Gm(V), where V is an (m + p)-dimensional space. Here, as usual, m is the number of system inputs while p is the number of outputs. A feedback gain then corresponds to a point in a p-dimensional subspace and the analogue of the Nyquist locus passing through the point -ljk is that of the curve of m-dimensional subspaces intersecting the p-dimensional subspace representing the multidimensional feedback matrix K. If Z(A) is the open-loop transfer matrix of our system, the basic multivariable feedback equations are Z(A)U
=y
U
=
-Ky,
which we can compactly write as
This system will have a solution if and only if the kernels of [Z(A), -1 pJ and Um, KJ have nontrivial intersection. This observation motivates Definition 9.12 The response diagram (Nyquist curve) of the pxm transfer matrix Z(A) is given by the points
Re A = 0
in Gm(V). We also need the idea of a Schubert hypersurface. Definition 9.13
Let W be a m-plane in V. Then the subset of Gm(V) given by I1(W)
= {U:dim(W n
U) ~ I}
is called the Schubert hypersurface associated with W.
292
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
By our above remarks on intersection properties of the p-dimensional subspace of K with am(V), we can use the Schubert hypersurface to characterize pole placement by output feedback. Theorem 9.17 The closed-loop poles corresponding to a feedback law K occur at points Ai where the algebraic curve ker [ Z( A), - I p] c m( V) intersects the Schubert hypersurface a(ker[lm' K]).
a
This is an obvious consequence of the fact that a(ker[lm, K]) = {W E Cm(V): W n ker[lm' K] # O}. Finally, we can put all of these ideas together and prove the generalized Nyquist theorem. Theorem 9.18 Let Z(2) be a proper rational transfer matrix with no poles on Re 2 = O. Further, suppose the Nyquist locus does not intersect the Schubert hypersurface a(ker[lm, K]). Let Jl be the number of closed-loop poles in Re 2 > 0 and let v be the number of poles of Z(2) in Re 2 > O. Then Jl = P
+ v,
where P is the number of times the Nyquist locus encircles the Schubert hypersurface a(ker[l m' K]) in the positive direction. PROOF A direct consequence of fact that u - v is equal to the change in argument in det(Z(2)K + I), plus the fact that this number is precisely the number of encirclements of a(ker[l m' K]) made by the Nyquist locus. Details are given in the Chapter References. Note that this result recaptures the most important aspect of the scalar Nyquist criterion of Chapter Seven, namely that it involves a fixed curve obtained from Z(2) that does not need to be changed as we vary the gain K.
9.16 ALGEBRAIC TOPOLOGY AND THE SIMPLICIAL COMPLEX OF l:
To this point we have employed several algebraic and geometric objects to study the various properties oflinear systems. Each "gadget", whether it be a k[z] - module, a vector bundle or a Grassmann variety, illuminated a different aspect of the overall category called Linear Systems. In this brief section, we introduce still one more algebraic object, a simplicial complex, in order to further amplify our prejudice that the study of Linear Systems is really applied algebra in disguise. At the most basic level of set theory, a simplicial complex arises whenever we try to geometrically characterize a binary relation defined on two finite sets. Let A and B be such finite sets with card A = n, card B = m, and let 2 c A x B be a binary relation, i.e., a, is 2 - related to b, if and only if (a i , bj ) E 2, i = 1,2, ... , n;j = 1, 2, ... ,m.
9.16
ALGEBRAIC TOPOLOGY AND THE SIMPLICIAL COMPLEX OF l:
293
We can represent such a relation by an n x m incident matrix A, whose elements are given by 1, (ai' b)EA [Alj = { 0, otherwise. Thus, we identify the rows of A with the elements of A, the columns with the elements of B. Clearly, by interchanging the roles of A and B we obtain the conjugate relation A* c B x A, and the corresponding incidence matrix A* = A'. While A is an algebraic characterization of the relation A, the idea of a simplicial complex arises when we attempt to geometrically represent the relation. Let us identify the elements of A with the vertices of a complex KA(B; A), while the elements of B will represent the simplices. Thus, if the element bkEB is A-related to the elements {al' a 2 , a 3 } , for instance, then b, would constitute the 2-simplex, bk =
82....- - - - - - - - - - - - - - - - - . 8 3 Now assume that the simplex bjis given by bj =
8
3
with the face
294
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
The above identification of vertices with the set A and simplices with the elements of B, with connections among the simplices being determined by the relation A, enables us to associate the geometrical object KA(B; A) with any such binary relation. Similary, by reversing the roles of A and B it is possible to generate the conjugate complex KB(A; A*). Thus, the preceding considerations enable us to produce three totally and completely equivalent mathematical objects: the binary relation A c A x B, the incidence matrix A and the simplicial complex KA(B; A). In algebraic topology, an important role is played by the so-called Betti numbers of K, which measure the number of "holes.. present in the complex at various dimensional levels. To formalize this idea, we need the concept of the boundary of a simplex.
Definition 9.14
>
Let (J = <X 1X 2' ... , x n be a simplex in K. Then the bound-
ary of (J, is given by
n+l
V(J=
L
k=l
(-1)k<XIX2,···,Xk Xk+l'···'X n+ 1 )
,
where x denotes omission of vertex X k, k = 1,2, ... , n + 1. Note that if dim (J = n, then dim v(J = n - 1. Here the summation is understood to denote a chain of simplices, e.g., the simplex b, =
-
>+
la 3 ) -
which geometrically is the hollow triangle
....... 8 3
8 1 ....--------l~-------
with the edge orientation being as shown. Any chain p+l
c, =
L
j=l
(Xj(Jj'
9.16
ALGEBRAIC TOPOLOGY AND THE SIMPLICIAL COMPLEX OF 1:
295
whose boundary oc; = 0 is called a cycle. It is easy to verify that o(oc;) = 0 for any cycle (i.e. the boundary operator iJ is idempotent), so it is of interest to find those cycles in K which are not the boundary of some higher dimensional chain. That is, we want to find cycles that are not bounding cycles. It is such cycles that bound the "holes" in K. If we let C p denote the p-dimensional chains of K, Zp the p-cycles and B; the bounding p-cycles, then the boundary operator iJ acts such that
and we can define the pth factor group as p = I, 2, ... , n + 1. H p is called the pth homology group of K. When H p = 0, there is only a single equivalence class in the factor group and this is B p , i.e., every p-cycle is a bounding cycle. When H p -# 0, there must be at least one z; E Z p such that z; E B p • We note that H o -# 0, so that when we say the homology of K is trivial, we mean H p = 0 for all p > o. Since H p is a finitely-generated abelian group, the number of generators !3p = dimHp- The numbers {!3;}f:J are called the Betti numbers of K and are topological invariants of the complex. Now let us return to system theory. In order for algebraic topology to make contact with linear system theory, we must be able to associate an abstract simplicial complex with some meaningful aspect of a linear system L. One way to do this is to consider the controllability matrix
of L. For simplicity, assume L is a single-input system so that
Thus, CC is just the ordered collection of n vectors listed above. For ease of notation, let us label the elements of"£ as Xl' X 2, ••• , X n , respectively, so that we have an abstract set X = {x l' x 2 , ••• , x n } . We will define X to be the vertex set of our complex Ky(X; A). The simplex set Y will consist of all possible combinations oflinearly independent elements from X. Thus, the relation Ais defined as
if and only if the vector
Xi
belongs to the simplex Yj.
296
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Let us adopt the customary notation Xi, 1\ Xi2 1\ ... 1\ x ik for a typical elements of Y. An example of the preceding set-up is given by the 3dimensional system in control canonical form
Then the sets X and Yare
=
{Xl'
xz, X3},
Y =
{Xl'
XZ, X3, Xl
X
1\
Xz,
Xl 1\
X3' Xz
1\
X3, Xl
1\
Xz
1\
X3}
= {YI' Yz, Y3' Y4' Ys, Y6' Y7' Ys} The incidence matrix A for the complex Ky{X, A.) is X
YI Yz Y3 Y4 Ys Y6 Y7 Ys
0 1 0
o o 1 1
0 1 0
1 1
1 1.
0
o
1 1
o
0 1
0 0
1
It is easy to see here that the geometric complex Ky(X; A.) is just the triangle <XIXZX3), together
x, ...- -
.. x
3
9.16
297
ALGEBRAIC TOPOLOGY AND THE SIMPLICIAL COMPLEX OF 1:
with all of its edges and vertices. Furthermore, almost by inspection one can verify that the homology is trivial for this complex i.e., H p = 0, p = 1, 2. The foregoing example provides the underlying motivation for the following basic result. Theorem 9.19
The single-input system 1: = (F, g, -) is completely reachable
if and only if the associated complex Ky(X; A) has trivial homology, i.e., Hi = 0,
i = 1, 2, ... , n - 1. PROOF
Assume Hi = 0, i > 0. Then, in particular, we have H n -
z=I il
2
= 0. Let
(Xi,i2···in_l<Xi,Xi,···Xik···Xin_) n-l
be an (n - 2)-cycle, i.e., OZ = 0. But, since H n - 2 = 0, there exists an (n - Ij-chain y E K such that oy = z, i.e., z is a bounding cycle. By definition of 0, we have
oz = I I I
k it
'
Since this equation must hold identically in the symbols xi,' X i2,···, X in' == 0. Thus, we have a linear relation between the coefficients (Xi, ... in' insuring that there exists a scalar 13 and a simplex <Xi, ... Xi) E K such that ail ... in
o(P<x i , •.. x in» = z, i.e., we choose y = 0 (P<xi, ... Xi»)' But since <xi, '" Xi) E K, it must be that Xi" X i2"'" x in are linearly independent, implying that 1: is reachable. Now let 1: be reachable. Then the simplex <X lX 2 ' " x n E K. Furthermore, each non-empty subset of {Xl' X 2, ••• , x n } is also linearly independent, hence each simplex associated with such a subset is also in K. Thus, all Hi = 0, i > 0, completing the proof.
>
The intuitive content of Theorem 9.19 is clear: the abstract space associated with a reachable 1: has no "holes". Thus, the appearance of an unreachable part in 1: is geometrically equivalent to "punching a hole" in the underlying abstract space. There are some corollaries of the above result that are also of interest. Corollary 1 1: is completely reachable if and only if the Betti numbers ofK are 1, Pi = 0, i > 0. Furthermore, these numbers constitute a set of arithmetic invariants of 1: with respect to state and control coordinate changes.
Po =
Corollary 2 1: is completely reachable if and only K, x(K) = 1.
if the Euler characteristic of
298
9
PROOF
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Follows immediately from the relation n-l
X(K) =
L
(-lYPi'
i=O
Unfortunately, the above results do not carry through to the case of multiinput system without further modification involving more advanced notions of algebraic topology. The reader can easily verify this by consideration of the system F=
3 01 01] , [ 1 1 2 1
This system is easily seen to be reachable, yet the homology groups of its associated abstract complex are
Ho=Z,
HI =0,
ZEFJZEFJ···EFJZ Hz =' , 10 times
The necessary modifications for the multi-input case are developed in the Miscellaneous Exercises at the end of the chapter. MISCELLANEOUS EXERCISES
1. Show that the state set X f defined as n/ker f is isomorphic to the set = fen) and, consequently, that the module-theoretic description of 1: from the text could be developed using Xf instead of X f. In X r- state transitions are defined by
Xf
F: [coJ! ---+ [z· coJf = z- [coJf' How would they be defined in Xf ? 2. Consider the action of GL(n, k) on pairs (F, G): (F, G) ---+ (TFT- 1 , TG) = (p, G), T E GL(n, k), k an arbitrary field. The stabilizer of (F, G) in GL(n, k) is {T E GL(n,.k): (F, G) = (p, G)}. Show that if (F, G) is completely reachable, then the stabilizer = identity. Show that the converse is false by constructing a counterexample. (Hint: consider the field k = Z/2). 3. Consider the following pattern discrimination problem: we want to construct a linear system that will recognize an input cP as cP mod t/J ¥- 0 and will recognize a different input (J as (J = 0 mod t/J. Show that a solution t/J exists if and only if (J is not a factor of cP. 4. Let 1: = (F, G, H) and define the k-linear dual of 1: as 1:* = (F ', H', G'), where denotes matrix transpose. Show that the state set X 1:* of 1:* can be given the structure of a k[z -1 J-module in the following fashion: I
299
MISCELLANEOUS EXERCISES
(i)
regarded as a vector space, Xl:' is the dual of X regarded as a k-vector space; (ii) define the scalar product in Xl:' (Z-l . x*)(x) = x*(Fx). The states of Xl:' are called the costates of E, (Compare this with Exercise 1). Now show that l: = (F, -, H) is completely observable if and only if H' generates Xl:' and prove the Duality Principle that the observable costates of l:* are the reachable states of l:*. 5. Consider a behavior sequence B = {a 1,a 2,a 3 , ••• }, where the {aJ are scalars. Show that the rank criterion rank
~r+
lom
= rank ~rm
= rank ~r.m+
1
is satisfied infinitely often for the principal submatrices of the Hankel matrix ~ of B. 6. Let GL(n, k) act on pairs (F, G) in the usual state coordinate transformation fashion, i.e, (F, G) -+ (TFT- 1 , TG). Define the matrix R(F, G) = (G, FG, F 2G, ... ,F"-lG, F"G),
and assume that (F, G) is completely reachable. Extend the action of GL(n, k) to R(F, G) in the obvious way. Order and number the columns of R(F, G) as 01,02, ... , Om; 11, ... , nm and define a multiindex v to be nice if the elements ofv have the property that if jt E v, then all indices j't, 0 ~ j' ~ j are also in v. A successor (1 of v is a multi-index such that (1 = v - jt + pq, where p is such that p'q E V for all 0 ~ p' < p. (a) (b)
Prove that there exists at least one nice v such that det R(F, G), ¥- 0 and, conversely, that if det R(F, G), = 0 for all nice v, then (F, G) is not reachable. Next show that the orbits of (F, G) under GL(n, k) form a quasiprojective variety V and that the nice neighborhoods det R(F, G), cover V. Further, each neighborhood is isomorphic to k/": (Hint: consider the Pliicker map 4J: (F, G) -+ (det R(F, G)p), where p = nice or a successor. For details of this construction, see Kalman, R. "Algebraic Geometric Description of the Class of Linear Systems of Constant Dimension," 8th Princeton Conf. on Info. Sci. and Systems, March 1974.)
7. Let K, L, M be modules over a fixed ring R. The sequence
is called exact if (i) B is onto, (ii) 0 is one-to-one, and (iii) M ~ L/K. Let f: n -+ r be an external description of l: and let K = ker f, L = n. Show that if the sequence is exact, then M is the canonical state set X f of l:.
9
300
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
What interpretation can be given to the homology modules H 0 = ker i/im Ii = M/imn and HI = ker s/im d? 8.
Consider the following diagram of maps
:(~1:
w
n------y--+r where f: W ~ y(Z-I), g: W~ [w]J' h: y(-)~ y(Z-l), with g being onto and h one-to-one. Here X J is the canonical state module, which is assumed to be torsion. Let X(z) be the characteristic polynomial of X J' Show that if w is an isomorphism, that is, f is canonically factored as f = how g, then Z J is a submodule of r having generators qi(Z)/X(z), where q;(z) is the image in r of the generator gi of n. In other words, X J and Z J are equivalent objects thereby demonstrating the existence of transfer functions and their equivalence to a state-variable model for a system ~. 9. Consider the two polynomials f(z) = z" + flZn-1 + fzz n- Z + ... + fn' 0
g(z) = glzn-1
+ gzzn-z + .,. + gn
and the associated matrices
o F =
0 1 0 0 1
o
o o o
-fn
-f,,-1 -fn-z ,
0
If «5 represents the usual controllability matrix
«5 = [GIFGI" ·IFn - 1 G] ,
and (f, g) is the monic polynomial which is the greatest common factor of f and g, show that (a) «5 = g(F), (b) Ai(g(F)) = g(4)i), where 4>i = ith root of f, (c) deg (f, g) = n - rank «5, (d) (f, g) = g/h, where h(z) is the minimal polynomial of G relative to F, i.e., rr:« + h 1F n-r- IG + ... + hn-rG = 0,
with r minimal.
301
MISCELLANEOUS EXERCISES
10. Let k be a Noethrian integral domain with identity and let K be its quotient field. Show that a given behavior sequence B = {Al , A 2 , •.• ,} over k has a finite-dimensional realization over k if and only if B has a finitedimensional realization over K. In other words, it is not necessary to go osutside the domain of "numbers" expressing the data to construct a model of that data. 11. Consider the system F=
(a)
(b)
0 10] [0 0 0 0
0
0 ,
Show that under the feedback group, this system has the canonical form
where a;j are invariants of the group action. Show that decoupling is possible only if one of the following conditions holds: (i) a 12a 2 3 of- a 22a 13 , unless either a l2 = a 13 = a 2 2 = a 2 3 = 0, (ii) all a 23 of- 0, if a 12 = a 13 = 0, (iii) a 2 1a 1 3 of- 0, if a 22 =:= a 2 3 = o.
°or
m
12. Let (F, G) be completely reachable with = rank G. Further, let k l ~ •.. ~ k m be the ordered control invariants of (F, G) and let l/Jl' .. " l/Jr be
arbitrary monic polynomials subject to
(i) l/J;il/J;-l' i = 1, 2, ... ,r -1; r s m, (ii) deg l/Jl ~ k l , deg l/Jl + deg l/J2 ~ k,
+ k2 , · . ·
Show that there exists a matrix L such that F - GL has the invariant factors l/Jl' l/J2'···' l/Jr' In other words, the structure of the system x = Fx + Gu can be altered arbitrarily by linear feedback, subject to the above lower bound on the size of the cyclic blocks given by the control invariants. (Remark: The control, or Kronecker, invariants {kJ are defined as follows: write the elements of [G/FGI·· ./pn-1GJ as
Then k; is defined as the smallest positive integer such that Fkig; is linearly dependent upon the elements preceding it in the above list.)
302
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
13. The natural state space of a finite-dimensional1inear system is either the torsion quotient module O/ker f or the torsion submodule f(O).
(a)
Show that a quotient module of 0 is a finitely generated torsion k[z]module if and only if it is of the form O/DO, where D is a nonsingular m x m polynomial matrix, i.e., it is of the form Coker(D). (b) Let Z c r. Show that Z is a finitely generated torsion k[z]-submodule of r if and only if Z = ker(D) for some nonsingular p x p polynomial matrix D. (This result enables us to test whether a given quotient or submodule could form the natural state set of a finite-dimensional linear system.) 14. Let ~ = {(F, G): F = n x n, G = n x m real matrices} and let the feedback group §' act on ~ as in the text. Choose a e ~ and let §'(u) denote its orbit. Show that the tangent space to §'(u) at a (when identified with a linear subspace of ~ using the vector space structure of ~) is given by {([T, F] - GK, TG): T e ute, R n), K eL(R", Rm)},
where [ . , . ] denotes the usual Lie product in R", i.e., the matrix commutator. Using this result, show that the projection n of the orbit onto L(R n, Rn) is given by n[gu] = T(F - GK)T-l, where a e~, 9 e fJi. Consequently, prove that n(§'(u» contains a Zariski open subset of L(R n, R n). 15. Let Ul = (Fl' G 1 ) , U z = (F z, G z) be two systems over an algebraically closed field k. We say a 1 is state-space equivalent to a z if there exists a coordinate change TeGL(n,k) such that F z '= TF 1T-1, Gz = TG 1 • Show that a 1 is state space equivalent to a z if and only if (i) XF,(Z) = XF2(Z); (ii) rank (F 1 ® I - I ® F 1 ) = rank(F z ® I - I ® F z) = rank(F 1 ® I - I ® F z); (iii) dim g(u 1 ) = dim g(u z); (iv) dim O(cT 1 ) = dim O(cT z),
where XF(Z) is the characteristic polynomial of F, ® denotes the Kronecker product, g(u) is the F submodule of k" generated by GL(n, k), and O(cT) is the orbit under GL(n, k) of the parallel sum of a with itself i.e., a EB a = (F EB F, G EB G). (This theorem gives a decision procedure for establishing whether any two systems are state-space equivalent. If we restrict attention to the subset of reachable and observable systems then, of course, a 1 '" U z if and only if they have the same transfer function.)
303
MISCELLANEOUS EXERCISES
16. Consider the scalar transfer function Z(2) monic.
(a)
Show that fraction
~
= nix, where deg n < deg X. X
can be written as the finite (terminating) continued
Z(2)=-----:....~0(2
1-
IX I
~
PI 1 _ ct2
~ :
P2
(b)
where all cti #- 0 and all polynomials Pi are monic. Show that the following ladder structure provides a realization for Z(2). y (output)
(c) Show that a canonical internal model for Z(2) matrices
G~ rn
H = [0 ... 0
i
0(1
0(2
0(2
F=
1 0 ... OJ,
kth term
Fl F2 ct3
F3
IS
given by the
304
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
where each F, has the form
o
0
1 0 F
= 0 1
o o o
-)'l,k-2
1
-Y11
-)'lk -)'l,k-l ,
with the v., being the coefficients of Pi' k = deg Pi' In the matrix F, all entries not explicitly indicated are zero. (For many more details on this construction and its use in the partial realization problem, see Kalman, R., "On Partial Realizations, Transfer Functions and Canonical Forms," Acta. Poly. Scand., Ma 31 (1979),9-32). 17. Let R = Z, the ring of integers and consider the scalar behavior sequence B = {2, 2, 2, ...}.
Show that the minimal realization over R is F = [lJ, G = [2J, H = [ll (b) Show that this realization is not canonical. (c) Show that P = [1J, G = [1J, fj = [2J is a canonical realization of B. (a)
Let ~ = (F, G, H) be a completely reachable and completely observable system over a field k. If we let GL(kn ) act on ~ in the usual way, i.e.,
18.
~.!.
t = (TFT-l,
TG, HT- 1 ) , T
E
GL(k n ) , show that the following numbers
completely determine the orbit of this action
i
= 1,2, ... , n;
A = 1,2, ... ,p;
j1. =
1,2, ... ,m,
where h p .) denotes the Ath row of H, while g(fJ) is the j1.th column of G and Tr( . ) denotes the matrix trace operation. 19. A category K consists of a class of objects {A, B, C, .. .}, and a set of morphisms Hom(A, B) with domain A and codomain B for any two objects A, B E K with the additional properties (i) for each object A E K, there is an identity morphism 1A E Hom(A, A); (ii) if f: E Hom(A, B) and 9 E Hom(B, C), there exists a unique morphism h e Hom(A, C), called the composition of f and g, i.e., h = go f. The operation 0 is assumed to be associative. Let ~ be any system (linear or not) and let [wJ!; denote the state resulting from an input sequence WEn, and let {wh be the class of all states which always give the same output when the same input ca is applied.
305
MISCELLANEOUS EXERCISES
(a)
(b)
Show that if we take as objects all dynamical systems ~ which realize a fixed behavior sequence B and as morphisms any relation
(c)
(d) (e) (f)
isomorphism
1- 1
Show that if ~ is observable (i.e. every set {x}I. = x) then any morphism tp: ~ -+ ~ is a partial map (i.e., a restriction to the observable states of ~). Prove that if ~ is reachable and.~ is observable then the morphism tp: ~ -+ ~ is a unique map. Putting the above results together, prove that any two canonical objects in Realj, are isomorphic. Specialize these results to the case of a linear system ~.
20. Consider the fractional linear transformation of the transfer matrix Z(A) -+ [0(21(,1,)
+ 0(2z{A)Z(A)J [0(11(,1,) + O(12(A)Z(A)r 1 == 2(,1,),
i.e., Z(A) -+ O(A)Z(A),
where 0(,1,)
= [0(11(,1,). O(12(A)J. 0(21 (A) 0(22(,1,)
Assume that the entries of 0(,1,) are all proper rational functions such that lim;._oo O(ij(A) exists, and Iim;._oo 0(21(,1,) = O. Write Z(A) = N(A)D- 1(A). Then
(~) (a)
(b)
= o(A)(~}
Show that the Macmillan degrees of Z and 2 are the same if O(s) can be continuously deformed into the identity matrix, such that at each stage of the deformation (N, D) are co-prime, i.e., there exist polynomial matrices X(A), yeA), such that X(A)N(A) + Y(A)D(A) = 1. Consider the special case of "pure feedback," where 0(21 = 0, 0(11 = 0(22 = I, 0(12 = -K(A). Then 2(,1,) = Z(A)(I - KZ)-l and N = N, D = D - KN. If we make the deformation Nt=N,
D, = D - tKN,
o s t s 1,
306
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
show that the Macmillan degree remains invariant ifthe deformation is continuous in t and A and
»)
DrC A rank ( Nr(A) = constant for all t, A..
21. Let 1: = (F, G, H) be a canonical system and assume t = (F, G, H) is one of the canonical forms considered in Chapter Five (Lur'e-LefschetzLetov form or Kronecker form). Prove that the map T: 1: --+ t is continuous if and only if m = 1 or p = 1, i.e., if and only if the system has a single-input or a single output. 22. Let 1: = (F, G, - ) be a completely reachable system over k[1T l' IT2 ' " ' ' ITr ] and suppose the Kronecker indices of 1: are constant as functions of (IT l' IT2' ... , ITr) == IT for all IT E k:, where k is the algebraic closure of the field k. (a)
(b)
Prove that 1: is coefficient assignable, i.e., we can find a feedback matrix K such that the characteristic polynomial of F - GK has assignable coefficients. If the a, are delay operators and if the conditions above are satisfied, show that 1: is feedback equivalent to a system involving no delays.
23. Let T(s) be a complex analytic function of a complex variable s. The Schwarzian derivative of T is defined as
(a)
Show that if W(s) = (aT numbers such that
+ b)j(cT + d)
det(: (b)
(d)
C,
d complex
= 1,
then L1 W = L1 T, and conversely. If we require that T and W be scalar transfer functions, then b = O. Hence, show that L1 W = L1 T if and only if W=
(c)
~)
with a, b,
aT . cT + a-I By normalizing T at a single point, prove that if T and Ware scalar transfer functions whose values agree at a single point in the finite complex plane, then T and W differ only by feedback if and only if L1W = L1T. Can this result be extended to the case of matrix transfer functions?
307
MISCELLANEOUS EXERCISES
24. A quiver Q is a finite connected directed graph, i.e., Q consists of a finite set of points P and a finite set A of arrows between points of P. Loops and multiple arrows are allowed. A representation V of Q over a field k assigns a kvector space V(P) for each pEP and a k-homomorphism F(a): V(p(a)) --+ V((p'(a)), for each a E A, p, p' E P, where a connects p, p', A representation is called indecomposable if it cannot be written as a direct sum V = VI EEl V2 , VI' V2 #- O. Finally, two representations V, Ware isomorphic if ther exist isomorphisms for all PEP,
cp(p): V(p)--+ W(p)
such that the following diagram commutes V(p(a))
J.!&.... V(p'(a))
CP(P)j
jcP(P')
W(p(a))
g(;i) W(p'(a)).
A quiver Q is of finite type if there exist only finitely many indecomposable representations (up to isomorphism). The quiver Q is tame if there are infinitely many isomorphism classes of indecomposable representations but these classes can be parametrized by a finite set of integers, together with an irreducible polynomial over k; otherwise Q is wild. Gabriel's Theorem shows that Q is of finite type if and only if its underlying undirected graph is one of the following Dynkin diagrams An:
•1
•
3
• ------.n
n;;>l
2
>-----e--; 1
Dn:
n;;>4,
2 ES:
E7
•
;
E8 :
•
•
I
•
I
•
I
•
•
•
•
•
•
•
•
9
308
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
By Nazarova's Theorem, all tame quivers have underlying undirected graphs that are one of the following extended Dynkin diagrams
A' n'
n;;'O
)-.--<
2
n+l
I -----------r .. •
•
•
•
£7: ••
I
•
(a)
•
•
•
Consider the following important quivers from linear system theory
o
•
o
...
(i)
(b)
(ii)
Show that both (i) and (ii) are wild. Define the quadratic form of a quiver as N
K(x 1 ,
X
z, · · · , x N ) =
I
xf -
i= 1
I
Xs(a)X,(a)'
aeA
where s(a), r(a) are the beginning and end nodes of the arrow a, respectively, and N = card P. Show that Q is of finite type, tame, or wild as K > 0, ~ 0 or indefinite, respectively. Apply this result to the quivers of system theory. 25. Consider the open-loop transfer matrix Z(A) = H(AI - F)-lG and the feedback gain matrix K. Assume 1: = (F, G, H) is canonical, i.e., Z(A) is proper. Show that the elements p, 11, v ofthe Multivariable Nyquist Theorem 9.18 are given by
+ I)], [det(Z(A)K + I)]
11 = # zeros [det(Z(A)K
v = # poles
309
MISCELLANEOUS EXERCISES
as A ranges over the imaginary axis (A = iOJ, OJ real). Thus, P = J1- - v = net change in the argument of det(Z(A)K + I)/2n. 26. Let Z(A) be a non-degenerate p x m transfer matrix of Macmillan degree n = mp. For all choices (AI"'" An), show that it is possible to find d(m, p) solutions K to the equation det(Z(Ai)K
+ I = 0,
i
= 1,2, ... , n,
where
d
) _ 1!2! ... (p - 1)'21 ... (m - 1)!(mp)! 1'2' . . .... (m+p- 1)'.
(m,p -
(This problem shows the rather astounding number of solutions possible to the output pole-placement problem). 27. Let Ky(X: A) be the simplicial complex of the single-input system ~ = (F, g, -) as defined in Section 16. Given two simplices up and a; in Ky(X; A) we say they are joined by a chain of connection if there exists a finite u~s such that sequence of simplices U~I' U~2"'" (i) (ii) (iii)
is a face of up' is a face of a., U~, and U~i+ I have a common face, say, u Pi ' U~l
u~.
i = 1, 2, ... , s - 1.
If we adopt the standard terminology that dim a, = i, then we say that such a chain is a q-connection if
(a) (b)
q = min{ lXI' PI' P2,"" Ps-l' IX s } · Verify that q-connection is an equivalence relation on Ky(X; A) for each q = 0, 1, 2, ... , dim K = N. Define the structure vector Q of K as Q
= (QN' QN-l,"" Ql' Qo),
where Qi = number of distinct i classes under the relation of q-connection, i = 0, 1, ... ,N. Show that ~ is completely reachable if and only if the structure vector of Ky(X; A) has the form Q
(c)
= (11
1 .. ·1).
Let Kx(Y; A*) be the complex conjugate to Ky(X; A). Show that ~ is completely reachable if and only if the conjugate complex has the structure vector Q* = (2n
-
1,2 n -1, ... ,2n -1, 1, 1, ... ,1),
i i i
(n -1)st
{Jth
position
position
Oth position
310
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
where
p = nil k=O
(nk -- 2). 1
(Here n = dim F) and ( . ) denotes the usual binomial coefficient. 28. Consider the m-input system L = (F, G,- ), with n x nm controllability matrix C(j =
[gl"'" gm' Fg I , · · · , Fg mF2g l, .. -, F 2gm, ... , Fn-lg h
••• ,
Fn-lg m ].
Define an abstract simplicial complex K~m as an object with mn distinct vertices (corresponding to the entries of C(j) and the rule that an ordered sequence (Xl' x 2 , ••• , x q of q distinct vertices is a simplex in K~m if and only if q ~ p, p = 1, 2, ... , n.
>
(a)
Use the Mayer-Vietoris sequence to show that the homology groups of K satisfy the relation Hp(K~~l)
~ Hp(K~~ll)EBHp_l(K~m-I),
l:S;;p:s;;n-1.
(b)
Call a system p-generic if any p vectors in the set C(j are linearly independent. Assume that L = (F, G, -) is (p + I)-generic. Use the above resuslt to prove that the homology groups of the complex of L satisfy where
1)
k = (nm p+1 ' p = 1,2, ... , n - 1. (This result generalizes the single-input case treated in Section 16, at least for generic control systems). NOTES AND REFERENCES
Section 9.1 The concepts and techniques of modem algebra and geometry take their sharpest form when employed for the analysis of linear systems. Nevertheless, their principal utility lies in the ease with which most of the ideas presented in this chapter can be extended to broad classes on nonlinear processes. A recent summary of much of the work in this currently very active area of algebraic-geometric nonlinear system theory is given in Casti, 1., "Nonlinear System Theory," Academic Press, New York, 1985.
NOTES AND REFERENCES
311
Some good background references to modern algebra and geometry are Birkhoff, G., and Mac Lane, S., "A Survey of Modern Algebra," 3rd ed., Macmillan, New York, 1985. Herstein, I. N., "Topics in Algebra," Blaisdell, New York, 1964. Auslander, L., and Mackenzie, R., "Introduction to Differentiable Manifolds," McGraw-Hill, New York, 1963. Singer, I. M., and Thorpe, J., "Lecture Notes on Elementary Topology and Geometry," Scott, Foresman, Glenview, Illinois, 1976.
Some excellent general references to the use of algebra in system theory are provided by Fuhrmann, P., Algebraic system theory: An analyst's point of view, J. Franklin lnst.; 301, 520-540 (1976). "Algebraic and Geometric Methods in Linear Systems Theory" (Byrnes c., and Martin, C., eds.) Lectures in Applied Math., vol. 18, American Math. Soc., Providence, R. I., 1980.
Section 9.2-9.3 The definitive statement of the module-theoretic treatment of linear systems is presented in: Kalman, R, Falb, R, and Arbib, M., "Topics in Mathematical System Theory," McGraw-Hill, New York, 1969.
The material synthesized in this volume is based upon the earlier works of R. E. Kalman, the most important being Kalman, R, Algebraic structure of linear dynamical systems, Proc. Nat. Acad. Sci. USA, 54, 1503-1508 (1965). Kalman, R., Algebraic Aspects of the Theory of Dynamical Systems in "Differential Equations and Dynamical Systems" (Hale, J., and LaSalle, J., eds.) Academic Press, New York, 1967. Kalman, R, "Lectures on Controllability and Observability," Centro Internazionale Matematieo Estivo Summer Course 1968, Cremonese, Rome.
Section 9.4 For additional examples of how a linear system can be used as a pattern recognition device, especially in the context of brain modeling, see Kalman, R, On the mathematics of model building, in "Neutral Networks" (Caianiello, E., ed.) Springer, New York, 1968.
Section 9.5 The historical use of transfer functions to describe a linear system stems from work in electrical circuit design. To some extent this is unfortunate, since a number of issues extraneous to the mathematical relationship between transfer functions and linear systems were obscured for several decades For instance, the usual discussions of transfer functions rely upon various convergence arguments for the contour integral defining the Laplace transform, conveying the erroneous impression that only stable systems can be studied by transform means. The algebraic treatment arising from the module-theoretic view shows that no questions of convergence enter
9
312
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
at all. The mathematical basis for transfer functions is the existence of a k[z]module structure on I" for which the input/output map f is a homomorphism. This is a purely algebraic fact. Additionally, the module machinery allows us to treat both continuous and discrete-time processes using the same formalism. Thus, both the Laplace transform and the" z-transform" methods are unified in the algebraic treatment. For some classically-oriented views on transform methods, see Horovitz, I., and Shaked, D., The superiority of transfer function over state variable methods in linear, time-invariant feedback systems designs, IEEE Tran. Auto Control, AC-20 84-97 (1975).
Zadeh, L., and Desoer, C, "Linear Systems Theory," McGraw-Hill, New York, 1963. Rubio, J., "The Theory of Linear Systems," Academic Press, New York, 1971. Smyth, M., "Linear Engineering Systems," Pergamon, New York, 1972.
Section 9.6 Two seminal semi-classical papers which led directly to the algebraic treatment of transfer function realization are Kalman, R., Mathematical description oflinear dynamical systems, SIAMJ. Control, 1, 152-192 (1963).
Kalman, R., Irreducible realizations and the degree of a rational matrix, SIAMJ. Control, 13, 520-544 (1965).
Both of these papers contain a wealth of concrete examples of the abstract realization procedures presented in the text. The Invariant Factor Theorem can be proved in many different ways at several levels of algebraic generality. For instance, see Gantmacher, F., "Matrix Theory," VoL 1, Chelsea, New York, 1959. Albert, A., "Fundamental Concepts of Higher Algebra," D. Chicago Press, Chicago, 1965. Jacobson, N., "Lectures in Abstract Algebra, VoL 2: Linear Algebra," Van Nostrand, New York, 1953.
Section 9.7 There are numerous formally different, but equivalent, methods for realizing a given transfer matrix. For example, see Rosenbrock, H., Computations of minimal representations of a rational transfer function matrix, Proc. IEEE, 115, 325-327 (1968). Mayne, D., A Computational procedure for the minimal realization of transfer function matrices, Proc. IEEE, 115, 1368-1383 (1968). Gilbert, E., Controllability and observability in multivariable systems, SIAM Control J., 1, 120-151 (1963).
The first effective procedure for carrying out the realization procedure for input/output data given in "Markov" form is Ho, B. L., and Kalman, R., Effective construction of linear state variable models from input/ output functions, Reqelunqstechnik, 14, 545-548 (1966).
NOTES AND REFERENCES
313
Other procedures improving upon aspects of Ho's method are presented in Silverman, L., Realization of linear dynamical systems, IEEE Tran. Auto-Control, AC-16, 554-567 (1971). Rissanen, J. Recursive identification of linear systems, SIAM Control J., 9; 420-430 (1971). Willems, J., Minimal realization in state space form from input/output data, Mathematics Institute Report, U. of Groningen, Groningen, Netherlands, May, 1973. Guidorzi, R., Canonical structures in the identification of multivariable systems, Automatica, 11, 361-374 (1975).
A summary of much of this material and the various realization procedures is given in Kalman, R., Realization theory of linear dynamical systems, in "Control Theory and Topics in Functional Analysis," Vol. II, Int'!. Atomic Energy Agency, Vienna, 1976.
We have not touched upon the important case of non-stationary behavior sequences, for which the coefficient matrices F, G, H may be time-varying. An algebraic tretment for the realization of such behavior is developed in Kamen, E., and Hafez, K., Algebraic theory of linear time-varying systems," SIAM J. Control Optim., 17, 500-510 (1979). Kamen, E., New results in realization theory for linear time-varying analytic systems, IEEE Tran. Auto. Control, AC-24, 866-878 (1979).
Section 9.8 The idea of partial realization is actually equivalent to the classical idea of Pade approximation, which concerns the problem of finding for a given Laurent series L a.z :", a strictly proper rational function f with denominator of minimal degree whose Laurent expansion in z - 1 agrees with the given series through the first r terms, for some specified finite r. The first systematic account of the partial realization problem is given in Kalman, R., On partial realizations of a linear input/output map, in "Guillemin Anniversary Volume," (de Claris, N., and Kalman, R., eds.) Holt, Rinehart & Winston, New York, 1968.
A definitive sharpening of these results is Kalman, R., On partial realizations, transfer functions, and canonical forms, Acta Polytechnica Scandinavia, Ma 31, 9-32 (1979).
An algebraic treatment of the partial realization problem avoiding use of the Hankel matrix and the above noted Pade data constraints, but based upon a generalization of the Berlekamp-Massey algorithm for recursive decoding of cyclic codes, is found in Sain, M. K., Minimal torsion spaces and the partial input/output problem, Info. and Control, 29, 103-124 (1975).
314
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Section 9.9 According to popular legend, the Pole-Shifting Theorem was first proved around 1959 by J. Bertram and later by R. Bass. The first published proof for multi-input systems seems to be Wonham, W. M., On pole assignment in multi-input controllable linear systems, IEEE Tran. Auto. Control, AC-12, 660-665 (1967).
We say that ~ = (F, G) is coefficient assignable if given any monic polynomial p(z), there exists a m x n matrix K such that det(zI - F + GK) = p(z). Over a field, coefficient assignability is equivalent to pole assignability; however, over a general ring R, coefficient assignability is a much stronger property. For an example over a general ring showing that complete reach ability is not enough to ensure coefficient assignability for multi-input systems, see Bumby, R., and Sontag, E., Reachability does not imply coefficient assignability, Notice Amer. Math. Soc., 1978.
An algebraic approach to the pole-shifting problem using the transfer matrix only is given in Seshadri, V., and Sain, M. K., An approach to pole assignment by exterior algebra, Allerton Can! on Circuits & Systems, Monticello, Illinois, Sept. 1976.
Section 9.10
Two excellent surveys on systems over rings are
Sontag, E., Linear systems over commutative rings, Rich. di Auto., 7,1-34 (1976). Naude, G. and Nolte, C., A survey of the theory of linear systems over rings, NRIMS Tech. Rpt.; TWISK 161, Pretoria, South Africa, June, 1980.
See also Sontag, E., On linear systems and non-commutative rings, Math. Sys. Th., 9, 327-344 (1976).
Section 9.11 The ideas of this section have been extensively pursued and expounded by R. Hermann in a continuing series of books and research articles Hermann, R., "Interdisciplinary Mathematics," Vols. 8, 9, 11, 13, 20, 21, Maths Sci. Press, Brookline, Mass. 1974-1980.
Probably the most complete and accessible work on algebraic geometry and its uses in linear system analyses is Tannenbaum, A., "Invariance and System Theory: Algebraic and Geometric Aspects," Lecture Notes in Mathematics, Vol. 849, Springer, Berlin, 1981.
NOTES AND REFERENCES
315
See also the works Hermann, R., and Martin, c., Applications of algebraic geometry to systems theory, part I, IEEE Tran. Auto. Control, AC-22, 19-25 (1977). Brockett, R., Some geometric questions in the theory of linear systems, IEEE. Tran. Auto. Control, AC-21, 449-464 (1976). Byrnes, c., and Hurt, N., On the moduli of linear dynamical systems, Adv. in Math. Supp. Series, 4, 83-122 (1979). Byrnes, c., and Falb, P., Applications of algebraic geometry in systems theory, Amer. J. M aths., 101,337-363 (1979). Hazewinkel, M., "A Partial Survey of the Uses of Algebraic Geometry in Systems and Control Theory," Report 7913/M, Erasmus u., Rotterdam, 1979. "Algebraic and Geometric Methods in Linear Systems Theory," (Byrnes, C; and Martin, C., eds.) American Math Society Lectures in Applied Math, vol. 18, Providence, R.I., 1980.
Section 9.12 The feedback group and the Kronecker indices seems to have first been introduced into linear system theory by Brunovsky in Brunovsky, P., A classification of linear controllable systems, Kybernetiku, b, 176-188 (1970).
See also, Kalman, R., Kronecker invariants and feedback, in "Ordinary Differential Equations," (Weiss, L., ed.) Academic Press, New York, 1971.
The relationship between the Kronecker indices and the McMillan degree of a transfer matrix is spelled out in detail in Martin, C, and Hermann, R., Applications of algebraic geometry to systems theory: The McMillan degree and Kronecker indices of transfer functions as topological and holomorphic invariants, SIAM Control J., 16, 743-755 (1978).
Related results are Brockett, R., Lie algebras and rational functions: Some control-theoretic questions, in "Proc, Queens' Symposium on Lie Theory and Its Applications," (Rossmann, W., ed.) Queens University, Kingston, Ontario, 1978. Brockett, R., and Byrnes, C., Multivariable Nyquist criteria, root loci and pole placement: A geometric viewpoint, IEEE Tran. Auto. Control, AC-26, 271-284 (1981). Hazelwinkel, M., Moduli and canonical forms for linear dynamical systems-III: The algebraicgeometric case, in "Geometric Control Theory," (Martin, C., and Hermann, R., eds.) Math. Sci. Press, Brookline, Mass., 1977. Friedland, S., Classification of Linear Systems, "Proc, AMS Conf. on Linear Algebra and its Role in System Theory," (Datta, M., ed.), Providence, RI (to appear 1986). Delchamps, D., Global structure of families of multivariable linear systems with an application to identification, Math. Systems Theory, 18, 329-380 (1985).
Section 9.15 The results of this section follow the work Brockett, R., and Byrnes, c., Multivariable Nyquist criteria, root loci, and pole placement: A geometric view, IEEE Tran. Auto. Cont., AC-26 (1981).
316
9
A GEOMETRIC-ALGEBRAIC VIEW OF LINEAR SYSTEMS
Some interesting related work is reported in De Carlo, R., and Saeks, R., The encirclement condition: An approach using algebraic topology, Int'l., J. Control, 26, 279~287 (1977). Owens, D., On structural invariants and the root-loci of linear multivariable systems, Int'l., J. Control, 28, 187~ 196 (1978).
Section 9.16 The initial results on associating a simplical complex with a linear system are reported in Casti, J., Polyhedral dynamics and the controllability of dynamical systems, J. Math. Anal. & Applic. 68, 334-346 (1979).
The extension of these results to multi-input systems is given in Ivascu, D. and Burstein, G., An exact homology sequence approach to the controllability of systems, J. Math. Anal. & Applic. (forthcoming).
C RAPTER
10
Infinite-Dimensional Systems
10.1 FINITENESS AS A SYSTEM PROPERTY
In mathematics one of the most highly-prized properties of a given structure is some type of finiteness; we can classify finite groups, but infinite groups still remain a mystery; finitely-determined function germs form the basis for the theory of singularities of smooth functions, while infinitelydetermined germs exhibit pathological behavior; the Hilbert Basis Theorem shows that the ring of invariants of a transformation group is finitely generated. In system theory, as in the rest of mathematics, finiteness properties also enter in a basic way as we have seen in the preceding chapters. For example, the Cayley-Hamilton Theorem, a finiteness condition on the subspaces generated by the powers of a matrix, plays the central role in controllability and observability conditions. Similarly, the crucial assumption underlying all realization theory results is that the external behavior sequence jr = {J l' J 2'" .}, possesses a finite-dimensional realization. In this chapter we shall explore the consequences of dropping the finiteness assumption regarding the dimension of the system state-space. As will be seen below, relaxation of this assumption greatly complicates the study of the properties of our system 1:, both from a technical as well as conceptual point of view. A variety of non-equivalent notions of controllability, observability, realization and stability emerge depending upon the choice made for the mathematical structure of the state-space, and there is no clear-cut answer as to which notion is the "right" one. 317
10 INFINITE-DIMENSIONAL SYSTEMS
318
In view of the mathematical and conceptual complications, it is reasonable to inquire as to the utility of studying such systems. Is there a broad enough class of such objects to justify the major efforts needed to create a decent theory of infinite-dimensional systems? Unfortunately, the answer to this question is an unambiguous and emphatic, yes! In fact, it is very likely the case that virtually all real systems arising in nature have infinite-dimensional state-spaces, and it is only by looking at subsystems that we encounter the finite-dimensional versions treated in the preceding chapters. For instance, any system whose space-time behavior is governed by a diffusion equation
ax at
-=Ax+f
'
x(O)
=
g
has a natural state-space that is infinite-dimensional, consisting of an appropriate function space containing the initial function g. Similarly, any system whose dynamics involve time-delays such as dx dt
= f(x(t),
x(t)
x(t - r)
+ u,
-r>o,
= get),
also has a natural infinite-dimensional state-space determined by the nature of the initial function g. There are many other examples of such systems, as well, some of which will be encountered as we proceed. Systems involving non-local interactions, multiple time-scales and time varying coefficients all lead to infinite-dimensional systems justifying the major effort needed to understand the nature and behavior of such processes and their models. In a short introduction, it is impossible to do justice to the richness and variety of the many types of issues that arise in a thorough treatment of infinite-dimensional processes. So here we shall try only to give the spirit and flavor of the nature of the questions that have to be addressed and the type of results that can be obtained. However, even to do this much it will be necessary to depart from the cozy' mathematical confines of finite-dimensional vector spaces and matrices and invoke a higher level of mathematical sophistication involving Banach and Hilbert spaces, together with the properties of various classes of linear operators defined on these spaces. Since we have no space here to develop a "crash-course" in elementary functional analysis, it will be assumed at the outset that the reader comes prepared with the equivalent of a one-semester introductory course in such matters. If not, any of the excellent texts cited in the bibliography may be consulted to fill-in background gaps.
10.2
319
REACHABILITY AND CONTROLLABILITY
10.2 REACHABILITY AND CONTROLLABILITY
We consider the system
x=
Fx
+ Gu,
x(O) = x o,
where now x E X, a reflexive Banach space. We assume that F generates a strongly continuous semigroup T(t): X -+ X, and that the operator G: U -+ X is bounded. Further, the space of controls U is also taken to be a reflexive Banach space. It is useful to write ~ in the integrated "mild" form x(t)
= T(t)x o + {T(t - s)Gu(s) ds.
Consider the operator B: U[O, t; U] -+ X
given by Bu
==
I
T(t - s)Gu(s) ds,
U E
U[O, t; U].
If we want to reach the state x* at time t using a control u, then we must require ~(B) == Range B = X,
since T(t)xo is fixed. This is a very strong condition, so strong, in fact, that ifit is satisfied it is possible to extend the semigroup T(t) to a group. Thus, generally speaking we cannot expect to.reach x* exactly, and must be content with getting arbitrarily close. In this case we need only require 9l(B) = X, and we say that ~ is approximately reachable, while if 9l(B) = X, we speak of ~ as being exactly reachable. For finite-dimensional systems, we have the very elegant, convenient and computable criterion for reachability that ~ is reachable if and only if the matrix has full rank. Our first line of attack on the infinite-dimensional case is to see to what degree the above test can be generalized. To this end, we define the set U" =
rUE
U: Gu E
n:'=l
E0(F")}.
(Here E0(T) denotes the domain of the operator T). Then we have the following result. Theorem 10.1
The system
~
is approximately reachable n
if the
= 0,1, ...}>= X,
set
320
10
INFINITE-DIMENSIONAL SYSTEMS
i.e., if the closure of the set of elements {PGU*} generates all of the statespace X. PROOF By elementary operator identities, it is easy to see that L is approximately reachable if and only if the adjoint equation
G*T*(t)x*
= 0 = x* = 0,
for all t. Thus, if L is not approximately reachable there exists an x* #- 0 such that <x*, T(t)Gu)
=0
for all u E U*.
Differentiating this identity countably many times at t = 0 yields <x*, FnGU*) = 0,
n = 0, 1,2, ... ,
implying f7I #- X, completing the proof. REMARK The converse of this theorem is not true as is seen by consideration of the case
~(F)
=
{x: x, ~;,
~:~
EX}.
Then if the system is x = Fx + glU 1 + gzu z with u 1 , U z E LZ[O, f], gl' gz EX, with gl(t), gl(t - h) vanishing, together with its derivatives outside [-h, 1], h > 0, it turns out that f7I is then a proper subspace of X. Theorem 10.1 appears to be the right generalization of the finite-dimensional rank condition, although in practice it may be difficult to apply as the set U* is not always easily obtained. Nonetheless, there are many situations in which we can readily obtain U* and, consequently, effectively determine the approximate reach ability of L. EXAMPLE
10.1 Consider the system
x = Fx + gu, where F is the right-shift operator having the matrix representation
F=
000 100 0 1 0 001
10.2
321
REACHABILITY AND CONTROLLABILITY
in the standard basis {e;}~ Then
1
of l z. Assume that g has the form g = (1 0 0 .. .y.
and, in general, Fng = en+ r- It follows that g is a cyclic vector for F and, consequently, Bl = X, implying that ~ is approximately reachable. EXAMPLE 10.2 We consider the simple one-dimensional heat equation with pointwise control
z(O, t) = z(l, t) = 0.
The semigroup generated by F
= dZjdx Z with
these boundary conditions is
00
I
T(t)qJ = J2
qJne -n
2
,,2
sin (nnx),
t
n=I
qJn
=
J2
We have U = R I and G: U --+H-I/Z-E[O, 1] G*: H I/2+E[O, 1] --+ R I is given by G*f = f(x I ) ,
f
E
for
HI/Z+TO,
any
s > 0. Also,
1]
(here H±(I/2+E) is a Sobolev space on [0, 1]). Thus, we can easily see that G*T*(t) = J2
I
00
qJn e -
n
2
,,2
t
n=I
so that
~
sin(nnx I ) ,
is approximately reachable if 00
J2
I
n=I
qJn e-n
2 ,, 2
t
sin(nnx I )
=
°
=>
qJn
= 0,
for all t and all n. It is elementary to verify that this will be the case only if Xl is an irrational number. The preceding results have established a basis for checkingc the approximate reachability of a given system. But, what about exact reachability? Can
10
322
INFINITE-DIMENSIONAL SYSTEMS
we give any corresponding results for determining those (rare) situations when it is possible to steer 1: to a desired state exactly? Perhaps the best general result in this direction is Theorem 10.2 Let U and X be a reflexive Banach spaces. Then 1: is exactly reachable if and only if there exists an oc > 0 such that
oc/lG*T*(· )x*llq where lip
~
+ 11q = 1. (Here /I·ll q denotes the
/Ix*/IX*, norm in the space U[O, t; U*J).
The proof of this result can be found in the Chapter References. As an illustration of the use of Theorem 10.2, consider the controlled wave equation
z(O, t) = z(l, t) =
o.
We can rewrite this in operator form as
w= The operator F = d given by T(t)
21dx 2
Fw
+ Gu.
generates the strongly continuous semigroup T(t)
W I] [21:[<WI' qJn>H cos met + ~me <w qJn>H sin mrtJqJn ] [w = . 21:[ -nn<w l, qJn>H sin nnt + <w qJn>H cos nntJqJn 2,
2
2,
acting on the Hilbert space H = L 2[0, 1]. Here we also have G =
[~}.
It is easy to verify that T*(t) = T( - t) and G* = [0 1J, so if u E L 2[0, t I; HJ, the system will be exactly reachable if and only if there exists an a > 0 such that
a/lG*T*(·)wI1 2 ~
/Iwl H ·
After some tedious calculations, this condition reduces to 2
4( t 1
-
. 22 nnt 1) SIn 4n 2 n 2
2
n n
2
~
2
(1 - cos 2nnt l) ,
together with tl > [
sin 2nntl] 2nn
.
n = 1,2, ... ,
10.3
323
OBSERVABILITY AND DUALITY
These conditions together reduce to t1 >
[
Sin met1] , me
which is satisfied for any t 1 > 0 implying that such an a> 0 can be found enabling us to conclude that the system is exactly reachable. 10.3 OBSERVABILITY AND DUALITY
One of the foundational results of linear system theory (nonlinear, too, for that matter) is the duality between reachability and observability: the system
x = Fx + Gu,
x(O)
= 0,
(L)
is completely reachable if and only if the dual system
x'=x'F', y= G'x,
(L*)
is completely observable. Loosely speaking, L* is formed from L by transposing all matrices and vectors and setting up the vector-matrix multiplications to be consistent (and also reversing the direction of time for time-dependent systems). It's natural to conjecture that by paying attention to the details, the same principles can be applied in the infinite-dimensional setting by using dual spaces and adjoint operators. However, since we have seen that there are at least two different notions of reachability, we will have to introduce corresponding concepts of observability and match them to generalize the finite-dimensional duality results. . Consider the system i
= Fz,
z(O)
=
Zo,
y=Hz,
(L*)
where F is the infinitesimal generator of a strongly continuous semigroup T(t) on a reflexive Banach space Z with Zo E ~(F), while H E L(Z, Y), with Y also a reflexive Banach space, The mild solution of L* is y
= HT(t)zo·
Z
-+
Let us define the map £1):
Lq[O, t*; Y]
Zo 1-+ HT(t)zo.
°
Definition 10.1
f3 > such that
We call L* continuously observable on [0, t*] if there exists a for all ZEZ.
324
10
INFINITE-DIMENSIONAL SYSTEMS
Definition 10.2 The system L* is initially observable on ker @ = {O}. Now let us make the following dual identifications
u= Y*,
G=H*, 1
F=F*,
[0, t*J if
Z=X*,
1
-+-=1. P q With the foregoing definitions and identifications, we can easily prove the following duality theorem. Theorem 10.3 (a) (b)
The system ~* is initially observable on [0, t*J approximately reachable on [0, t*]. ~* is continuously initially observable on [0, t*J exactly reachable on [0, t*].
PROOF
if and only if
is
~
if and only if ~ is
See references cited at the end of the chapter.
EXAMPLE 10.3 heat equation
Using Theorem 10.2, it can be seen that the controlled
02Z
oZ
ot = ow2 z(O, t)
+ u,
(~)
= z(l, t) = 0,
is not exactly reachable on L 2[0, IJ using controls u E L 2[0, t*; Z]. As a consequence, the dual system
x(O, t)
=
x(l, t)
= 0,
(~*)
yet, w) = x(t, w),
is not continuously initiallyobservable on [0, t*]. In Chapters 3 and 4, we have drawn the distinction between reachability and controllability, as well as between observability and constructibility. Let us focus upon controllability for the moment. In this case, we are interested in whether or not there exists a control that will drive the system
x= to the origin. We say that
Fx ~
+ Gu,
x(O)
= x o =f. 0,
is exactly controllable on [0, t*J if
range fJB
:=>
range T(t*),
(~)
lOA
and
325
STABILITY THEORY ~
is completely controllable on [0, t*] if range fJI :::J range T(t*).
Conditions for these inclusions to hold can be easily obtained from the standard fact that if V, W, Z, are reflexive Banach spaces, with FE L(V, Z), G E L(W, Z), then ker(G*)
c:
ker(F*)¢>range(G)
:::J
range(F).
In terms of constructibility, the controllability concepts above correspond to the notions of continuously final and final constructibility. We say that ~* is continuously constructible on (0, t*] if there exists a fJ > 0 such that fJll(Qxllu[o,t*;Yl ~ II T(t*)xllx,
for all x E X, and
~*
is constructible on [0, t*] if ker
(!) c:
ker T(t*).
With these definitions, we can state the following duality theorem linking controllability and constructibility. Theorem 10.4 (a) (b)
The system
~*
is
constructible on [0, t*] if and only if ~ is approximately controllable; continuously constructible if and only if ~ is exactly controllable. 10.4 STABILITY THEORY
The two most important stability results for the linear system
x=
Fx
+ Gu,
x(O) = X o
*0
(~)
are (1) (2)
Asymptotic Stability (in the sense of Lyapunov), where we have Ilx(t)1I ~ 0 if and only if Re A;(F) < 0 (assuming G = 0), and the Pole-Shifting Theorem (stabilizability), which asserts that if ~ is reachable (stabilizable), there exists a feedback matrix K such that F - GK has prescribed roots (Re Ai(F - GK)
< 0).
For infinite-dimensional systems, neither of these results extends directly without the imposition of additional structure on F and/or a strengthening of our notion of stability. Let us begin with the problem of the asymptotic stability of the free motion of ~. By a well-known counterexample due to Hille and Phillips, there exists a semi-group T(t) whose generator F has an empty spectrum, although II T(t) II = e", t ~ 0, showing that there is no possibility of basing the
10
326
INFINITE-DIMENSIONAL SYSTEMS
asymptotic stability test on the spectrum of F without further assumptions. It turns out that a convenient way to proceed is to introduce the following stronger notion of stability. Definition 10.3 The uncontrolled system there exist constants M, W > 0 such that
I T(t) II
~
S Me-rot,
is called exponentially stable if t ~
o.
We can now establish the following result Theorem 10.5 The system ~ is exponentially stable sup Re 1(F) < 0 in any of the following situations: (a) (b) (c)
if
and only
if
F bounded, T(t) an analytic semigroup, T(t) compact for some t* > O.
Instead of the spectrum test which involves finding the location of the spectrum F, it is often convenient to be able to invoke the test that ~ is asymptotically stable if and only if there exists an Hermitian matrix P > 0 such that F*P
+ PF= -Q,
for all Q > O. To extend this result to our current setting, we must assume that X = H, a Hilbert space with inner product Co). We then have Theorem 10.6 The system ~ is exponentially stable an Hermitian operator B on H such that 2Re(BFx, x) = (BFx, x)
=
if and
only
if there
exists
+ (F*Bx, x)
-lIxIl 2 •
Now let us consider the problem of stabilizing the controlled motion of ~ by means of linear feedback. We would like to be able to have a theorem along the lines that if ~ is approximately reachable, then ~ is stabilizable, and conversely. However, consider the system X = [2,
with
U
= reals,
10.5
327
REALIZATION THEORY
For any feedback control u =
Fx
+ Gu =
{Xl + gl
°
+ g2
It can be shown that E A(F + Gk), implying that 1:, which is clearly approximately reachable, is not stabilizable. So we must impose more than just approximate reachability upon 1: to assure stabilizability, The following result of Slemrod gives some indication of how much more than just approximate reachability is needed.
Theorem 10.7 Let F generate a strongly continuous group T(t) on a Hilbert space H. Assume that for some f. > 0, there exists a fJ > 0, such that 2
f:IIG*T*( -t)xIl dt for all
X
E
~ fJllxll 2 ,
H. Then 1: is stabilizable.
The key assumption in this result is that F generates a group. In practice, it may be difficult to verify the hypotheses underlying Theorem 10.7, and it is desirable to have a result with a more direct connection between the controllability and stability properties of 1:. Perhaps the most straightforward general result along these lines is Theorem 10.8
lfthe mild solution x(t) = T(t)x o + {T(t - s)Gu(s) ds,
of 1: is controllable in [0, t*J on a Hilbert space H, using controls u E L 2[0, t*; UJ, with U also a Hilbert space, then 1: is stabilizable. 10.5 REALIZATION THEORY
A finite-dimensional linear system is specified by the triple 1: = (F, G, H), which gives rise to the weighting pattern W(t) = HeFtG or its Laplace transform, the transfer matrix Z(A) = H(Al - F)-lG. As we have seen in earlier chapters, if 1: is reachable 'and observable, the poles of 1: correspond to the spectrum of F and the representation 1: is then minimal. Unfortunately, this correspondence between a canonical system 1: and a minimal realization no longer holds for distributed processes, and it is necessary to broaden our notion of a realization in order to recapture some of the spirit of the finitedimensional results. Consider the system x = Fx + Gu, y=Hx,
10
328
INFINITE-DIMENSIONAL SYSTEMS
with »(r) E X, a Hilbert space, with F an operator on X (possibly unbounded) generating a strongly continuous semigroup. We assume that U(t)E U, yet) E Y, finite-dimensional Hilbert spaces of dimensions m and p, respectively. In this setting, we note that the transfer matrix 2(2) should be interpreted with care since it only exists in certain regions of the complex plane due to the fact that, unlike in the finite-dimensional setting, F may possess more than a point spectrum. Definition 10.4 We say that 1: = (F, G, H) forms a regular realization of Wet), whenever F generates a semigroup, G and H are bounded and Wet) = ne-e: In a variety of important settings (for instance, when observations are taken only on the boundary of X), it is desirable to alIow the operator H to be unbounded. For this, we must extend our notion of a realization. Definition 10.5 HeFtG and
(i) (ii)
We call 1: = (F, G, H) a balanced realization if Wet)
=
range G c E0(F), H is linear and F-bounded,
i.e.,
IIHxll r ~ ktllFxll x + k2l1xllx, for some k t , k 2 > 0, and all x
E
E0(F).
While regular and balanced realizations of Ware quite different objects, there are some strong interconnections as the folIowing result demonstrates. Theorem 10.9 Wet) has a balanced realization if and only if it has a regular realization. Furthermore, the infinitesimal generator F can be taken to be the same in both realizations. PROOF
See the Baras and Brockett article cited in the References.
A key concept in realization theory is the idea of a canonical, i.e., reachable and observable model 1:. In our setting, we say that 1: is reachable if G*eF*tx = 0 implies x = 0 for all t ~ 0, and observable if HeFtx = 0 implies x = 0 for all t ~ O. Let us assume that we are given a (regular) realization 1: = (F, G, H) of W. How can it be reduced to a canonical realization? Theorem 10.10 Let a (regular) realization 1: = (F, G, H) be given with state space X for the weighting pattern W. Let
M = {x
E
X: HeFtx = 0, t ~ O}\
N = {x E M: PMG*eF*tx
= 0, t ~
O}',
10.5
329
REALIZATION THEORY
where PM is the orthogonal projection onto M, P N the corresponding projection onto N. Then the realization
G=
P = PNFIN'
PNG,
H=
HPN
is a canonical (regular) realization of W with state space N.
The main importance of canonical realizations is the State-Space Isomorphism Theorem, which asserts that any two canonical realizations of a given weighting pattern W differ only by a change of basis in the state-space X. Unfortunately, the natural notions of reachability and observability introduced above do not lead to canonical models admitting such a result for infinite-dimensional systems. To extend the State-Space Isomorphism Theorem, we need a more restricted concept of reachability and observability.
Definition 10.6 The system 1: = (F, G, H) is called exactly reachable if the
lim t*--+oo
r eFtGG*eF*t dt t*
Jo
exists as a bounded and boundedly invertible operator. Similarly, 1: is exactly observable if lim t*-+
00
i
tO
eF*tH*HeFt dt
0
exists as a bounded and boundedly invertible operator. We can now establish Theorem 10.11 Let 1: = (F, G, H), ~ = (p, G, H) be two realizations of W. Then 1: and ~ are similar, i.e. there exists a bounded and boundedly invertible operator P such that PF = PP,
PG=G,
if either of the following conditions are satisfied: (i) (ii)
1: and ~ are reachable and exactly observable or, 1: and t are observable and exactly reachable.
Now we come to the practical question of how to identify those weighting patterns W that possess a regular realization. Theorem 10.12 (1)
Let Wet) be a pxm weighting matrix. Then
If W admits a regular realization, each element of W must be continuous and of exponential order and
10
330
(2)
INFINITE-DIMENSIONAL SYSTEMS
if every element of W is locally absolutely continuous and the derivative of W is of exponential order, then W admits a regular realization.
Assuming that W admits a regular realization, our final task is to explicitly construct the operators F, G, H. To this end, we introduce the Hankel operator JIf: L 2(0, 00; U) ~ L 2(0, 00; Y)
{oo W(t + a)u(o) da.
u(t)~
Assuming that W is square-integrable, JIf is well-defined and bounded. Next, introduce the left-translation operator on a space X by eFt:X~X
+ t),
f(a)~f(a
t
~
O.
An explicit regular realization of W is then given by Theorem 10.13
A regular realization
of~
of W is given by
X = range£; F
= infinitesimal generator of the left-translation semigroup on X,
(Gu)(a) = W(a)u, Hx = x(O). Moreover, this realization is reachable and exactly observable. 10.6 THE LQG PROBLEM
In correspondence with the finite-dimensional results of Chapter 8, it is reasonable to expect that with a little care in defining spaces and operators all (or, at least, almost all) the results pertaining to optimization of quadratic cost functions subject to linear dynamical side constraints can be recaptured in our current, more general setting. In view of the quadratic nature of the cost criterion, the most natural setting for such extensions is a Hilbert statespace. Consider the linear system
x=
Fx
+ Gu,
x(O) = x o,
where x(t) eX, a Hilbert space, and G is a bounded operator from a Hilbert space U to X. Assume that F generates a semigroup T(t) on X. As cost functional, we take J(u) = (Mx(T), x(T)x
+ IT«Qx, x)x + (Ru, u)u) dt,
10.6
331
THE LQG PROBLEM
where M, Q E L(X, X) are self-adjoint, non-negative operators, R E L( U, U) is such that (Ru, u) ~ ml/ul/ 2,m > oand (·,.)ydenotes the inner product in the corresponding Hilbert space Y. After an operator version of completing-the-square, whose details can be found in the standard references cited later, the (unique) solution to the above optimization problem is given by
Theorem 10.14 The unique optimizing control for the functional leu) is given in feedback form by u*(t) = -R-1G*P(t)x(t),
a.e.,
where pet) is the (unique) solution of the inner product Riccati equation d
- dt (P(t)x, y) = (Qx, y)
+ (P(t)x, Fy)
+ (Fx, P(t)y) peT)
(P(t)GR-1G*P(t)x, y),
o :5; t :5; T,
= M,
for all x, y E P}(F). Furthermore, the optimal cost l*(u*) is given by l*(u*) = (P(O)x o, xo)x. REMARK The preceding set-up does not apply to the case when the control u is exerted only upon the boundary. In this event, G is not a bounded operator from U -+ X so additional assumptions are needed to make the conclusions of Theorem 10.14 hold. See the Problems and Exercises for an account of how this can be done.
Of considerable importance is the infinite-time (T = co) version of the foregoing regulator problem. Here we wish to minimize
leu)
=
l~(QX,
X)x
+ (Ru, U)u) dt,
and it is straightforward to show that the optimal control is given by
u*(t) = -R-1G*Px(t), where P satisfies the algebraic Riccati equation
(Qx, y)
+ (Fx, Py) + (Px, Fy) -
(PGR-1G*Px, y)
provided that (F, Ql /2) is observable and (F, G) is reachable.
= 0,
332
10
EXAMPLE
INFINITE-DIMENSIONAL SYSTEMS
Consider the controlled heat equation oW
02W
at = ow -
OX2
ow
(0 t) = -
ox'
+ U(X, t),
(1 t) = 0
ox'
0 < x < 1, t > 0,
,
W(x,O) = wo(x),
with the quadratic cost functional J(u)
= L:O
f
[w 2 (t, x)
+ u2 (t, x)] dx dt.
Letting £P;(x) = fi cos nix, £Po = 1, it is straightforward to show that the (unique) solution of the operator algebraic Riccati equation is 00
P=
L (Jn 4/ + 1 -
n 2l)£P/·, £P),
j=l
and the optimal feedback control is 00
u*(x, t) = -
L (Jn 4/ + 1 -
n2l)wj(t)£pj'
j=O
where wj is the jth coefficient in the expansion w(t, x) =
00
L wk(t, 'x)£Pk'
k=O
The optimal cost is J*(u*)
=
00
L (Jn 4j4 + 1 j=O
n 2j2)<wo, £Pj)2.
10.7 OPERATOR RICCATI EQUATIONS AND GENERALIZED X- Y FUNCTIONS
Earlier we saw that for systems in which the number of inputs and outputs are much less than the number of states, a major computational savings could be obtained by employing the so-called "generalized" X- Y functions to calculate the optimal feedback gain K = -R-1G*P, rather than solving the Riccati equation directly. This observation holds with even greater force when dealing with infinite-dimensional problems, since here it is usually the case that the controls and/or observations affect the state at only a finite number of points, or at least over some proper subset (or subspace) of X.
10.7
333
OPERATOR RICCATI EQUATIONS
Thus, we would expect the X - Y functions to provide a means for improving the computational tractability of the LQG problem. Here we sketch the means for extending these functions to the infinite-dimensional setting, i.e., the generalization to X - Y operators. Consider the linear system
x = Fx + Gu, with F, G, x, u as above, and introduce the observation equation y=Hx.
Here y E Y, a Hilbert space. We wish to minimize the quadratic cost functional J(u) = R
IT
[(y(t), y(t)y
+ (Ru, u)uJ dt,
> O. The solution as given in the last section is u*(x) = -R-1G*P(t)x = -K(t)x,
where P(t) satisfies the operator Riccati equation of Theorem 10.14 with Ql l 2 = H, (i.e., (Qx, z) = (Hx, H*z»). The generalized X-Y operators provide a means to compute the operator K directly, without the intermediate computation of the Riccati operator P. Theorem 10.15 system
The optimal feedback gain operator K(t) is given by the dK dt =
-R-1G*L*(t)L(t),
dL
dt = L(t)F + L(t)GK(t), where K(t) E L(X, U), L(t) E L(X, Y). The initial conditions are K(T) = 0,·
L(T) = H.
Furthermore, the solution of the operator Riccati equation P(t) is given by P(t)F
+ F*P(t) =
-K*(t)K(t)
+ H*H -
L*(t)L(t).
REMARK The equations for the operators K and L should be interpreted in the same inner product sense as that given in Theorem 10.14 for P. The proof is a direct extension of that given in Chapter 8 for the finite dimensional case. Again we emphasize the point that in practice the spaces U and Yare usually finite-dimensional. An easy way to see the importance of
10
334
INFINITE-DIMENSIONAL SYSTEMS
this fact for actual generation of the optimal control is to consider the kernels k(t, x), I(t, x) of the operators K(t) and L(t) in the case when F is a diagonal operator. In this case we have the kernel equations
Ok~;
x)
Ol~~
x) = F*I(t, x)
=_
o d~
[Ix R -lg(~)I(t,
k(T, x) = 0,
]V(t, x),
+ v« x{Ixg(~)I(t,
~) d~
1
I(T, x) = h(x).
The conditions for k, 1 on ax are given by the state equation at t = 0, i.e., x(O) = Xo' The functions g(x), h(x) are determined from the operators G and Hby [Gu(t)](x)
=
gT(X)U(t),
Hx(t)
=
Ixh(~)x(t,
o d~.
Here, if k(t, x) is of dimension m l , the number of input variables, while I(t, x) is of dimension m2 , the number of system outputs, then solution of the above system involves m l + m 2 conventional partial differential equations, rather than the doubly infinite operator Riccati equation for the kernel p(t, x, ~) of the operator P(t). EXAMPLE
Consider the one-dimensional heat equation oW
02W
at = ox2 + b(X w(O, t) = w(l, t)
I
2)U(t),.
0 ~ x ~ 1,
= 0,
where the point control u(t) is exerted at the mid-point of the rod. We have the spaces X={w(x):w'EL 2[0,I;R], O~x~I}, U~
Y=R.
Let the observation be the average temperature over the rod: y(t) =
f
w(t, x)h(x) dx,
where h(x) is the weight function determined from the operator y discussed earlier. Let the control u(t) be chosen to minimize J =
faT [y2(t) + u 2(t)] dt.
= Hx,
as
335
MISCELLANEOUS EXERCISES
According to our earlier results, the optimal feedback control u*(t) is given by u*(t) = -
s:
k(t, x)w(t, x) dx,
where k(t, x) is the solution of ok(t, in x) = -l( t, 1)1( 2 t, x ) , ol(t, x) = _ 02/(t, x) ot ox2
+
l( l)k( ) t, 2 t, x ,
k(t,O) = k(t, 1) = l(t,O) = l(t, 1) = 0, k(T, x) = 0,
I(T, x) = h(x).
These equations are relatively straightforward to solve numerically, once the observation weight function h(x) is prescribed. Numerical results are given in the papers cited in the references.
MISCELLANEOUS EXERCISES
1. Let the input, state and output space U, X and Y be complex vector spaces and let A E C, the complex numbers. Define Z(A) to be the space of all pairs (u, y) E U X Y such that there -exists an x E X such that for a given A
AX = Fx
+ Gu,
y=Hx, where F, G, H are appropriate operators on X, U and Y, respectively. As A varies, we obtain a map C -+ ~(U
Ei1 Y),
where ~ is the Grassmann space. Call this map the transfer function of the system ~ = (F, G, H). (a)
show that if the resolvent (AI - F) - 1 exists, then ~(U
Ei1 Y)
= {(u, y): y = H(AI - F)-lGU},
so that the transfer function can be identified with the curve
A -+ H(A.! - F)-lG in the space of linear maps.
10
336
INFINITE-DIMENSIONAL SYSTEMS
(b)
Show that usually differential operators do not satisfy the above condition. (c) Consider the linear differential equation d 2w dw a2 dt 2 +aldi+aow=f(t).
Show that for A E C, the transfer function is given by the set of all pairs (u, y) E U X Y such that Dy - Ay = uf,
where D is the differential operator d2
d
= a2 dt 2 + a 1 dt + aoI.
D
2. Consider the nonlinear diffusion equation olp
at =
jPlp ox2
+ Alp -
2 J1,lp ,
A, J1,
c: 0, 0 s
x
S 1.
Show that if A < n 2 then lp = 0 is an asymptotically stable equilibrium in the L 2 sense. (Hint: consider the candidate Lyapunov function V
= Illp 1112[0,1)'
3. Consider a transmission line with capacitance c(x), inductance l(x) > 0, resistance r(x) and conductance g(x), x E [0, 1]. The energy of a currentvoltage distribution
G)
in the line is E
=
s:
(cv 2
+ [j2) dx.
Let
Assume the line is short-circuited at x = 0 and connect the end x = 1 to a lossless reference line having c = 1 = 1. Assume that signals are sent down the reference line toward the end x = 1 of the line under study. (a) Show that the dynamical equations for this system are
:t G) FG) + =
g=HG}
Gu,
337
MISCELLANEOUS EXERCISES
where F is the unbounded operator
acting on the space
~(F)
=
{G)EH:
E <
00,
i(l) = v(l), v(o) =
o}-
The operators G and Hare Gu = 15(x -
(b)
1)(_~)u,
HG)
= v(l)
+ i(l).
Establish that if we consider the same line in impedance form, i.e., the input is i(l) while the output is x(I), then the system equations are as before and the operator F remains unchanged, but we have
~(F)
=
Gu
{G)EH:
= 15(x -
E<
00,
v(o) = i(l) =
a},
1)G)U, HG) = v(l).
4. The delay system x(t) = Fox(t)
+ F lX(t -
1)
+ Gu(t),
x(t)
=
-1 ::;; t ::;; 0,
y(t) = Hx(t),
is called output reachable on [0, t*] if, for every y* and tp, there exists a u(t) E Li[O, t*] for which y(t*) = y*. (a)
Consider the table HG HFoG HF~G
°
HF1G HF1FoG + HFoF1G
°° °°
HFiG
°
338
10
INFINITE-DIMENSIONAL SYSTEMS
Show that the delay system is output reachable if and only if the above array has rank p, where p is the number of system outputs ( = number rows of H). REMARK If 0 < t* :$; 1, then only the first column of the array is considered; if 1 < t* :$; 2 then the first two columns are considered, and so forth. (b) Extend the above result to the case when the system has control and observation delays as well, i.e.,
x(t)
= Fox(t) + F1x(t -
1) + Gou(t)
y(t)
= Hox(t) + H1x(t - 1).
+ G1u(t -
1),
5. Consider the scalar transfer function
Z(A
pie-.lhiAn-l + ("!_ pi e-.lh')An-2 + ... + ("!_ pie-.lh,) -.lh) = "!_ L..,-o 1 L..,-o 2 L..,-o n n ,e An + ocie - Ahi)A 1 + + oc~e .lh; ,
where oct, (a)
L:=o
(L:=o
Pi are constants, ho =
0, hi > 0, i = 1,2,
, l are constant delays. Show that the following delay system canonically realizes Z(A, e-.lh): x(t)
I
L Fix(t -
=
+ Giu(t -
hi)
i=O
y(t)
=
hi)'
I
L Hix(t -
hi),
i=O
where
o o o
Fi
-
[ -oc~
o o o
o
1
o o
-
0
OC~-l
o o
1
o
•••
1
]
i = 1,2, ... , l,
-oci '
o, = [0],
o o o
i
= 1,2, ... , t,
1 Hi = [P~P~-l'···'
pn,
i
= 0, 1, 2, ... , l.
339
MISCELLANEOUS EXERCISES
(b)
What is the corresponding "observer" canonical form for this realization? 6. Let f: n --.. r be the input/output map of a (possibly) infinite-dimensional, continuous-time constant linear system. Take n = E(_00, OJ' the space of distributions with compact suport in ( - 00, OJ, r = E[O, 00)' the space of smooth functions on [0, (0). As the state-space of an internal model, take X = Ei-oo.oj!ker f. Using this state-space, it has been shown by Kalman and Hautus that the corresponding internal model is exactly reachable and observable. Now consider the internal model
x = Fx + Gu,
XEX,UE
U,
(~)
where X and U are Banach spaces. Under very mild conditions, this system cannot be exactly reachable with L 1 (or L 00) inputs with bounded support. (a) (b)
Show by an example that even if we extend the input space of ~ to Ei-oo, OJ the system cannot be made exactly reachable. Let X be a locally compact Hausdorff space. A subset T of X is a barrel if (i) T is convex, (ii) T is balanced in that ax E T for all Ia I ~ 1 and x E T, (iii) T is absorbing, i.e., for each x E X there exists a scalar a i= 0 such that ax E T, (iv) T is closed.
We call X barreled if every barrel is a: neighborhood of O. Show that if we restrict the state-space of E to be the set of reachable states is topologically isomorphic to Ei- oo,Oj if and only if gt(~) is barreled. 7. Consider the system gt(E) and if E is observable, then gt(~)
yet)
=
oe
L hnxn,
n=l
where
x; E X = 12 and {gn}' {hn} E 12 • (a) (b)
Show that if A. n i= A. m, n i= m, and On' hn i= 0, then the reachable set is dense in 12 and the system is completely observable. Show that there are many non-isomorphic realizations of the input/ output behavior of this system.
10
340 8.
INFINITE-DIMENSIONAL SYSTEMS
Consider the continuous-time control process with distributed delays x(t) = Fx
+ Box(t -
k)
+ J:D 1(S)U(t -
+
f
B 1(s)x(t - s) ds
+ Gu(t) + Dou(t -
h)
S) ds,
x(t) = e(t),
t
E
[to - k, to],
u(t) = 1](t),
t
E
[to - h, to), u(t)
L 2 [t o, t 1 ].
E
Assume we wish to minimize J
=
f
tl
[(x, Qx)
to
(a)
+ (u, Ru)] dt + (x(t 1), PX(t1))'
Q, P :2: 0, R > 0.
Show that the optimal feedback control u* is given by u*(t) = -R- 1{[G'E o(t) +
+ E~(t,
O)]}x(t)
f[G'El(t,S)+E~(t,O,S)]X(t-S)dS
+ f[G'Eit, s)
+ E 5(t, 0, s)]u(t -
s) ds,
where the matrices Eo - E 5 satisfy the generalized Riccati system -
iJE O ot = F'Eo(t)
+Q-
(a~1
+ 0~1)
+ Eo(t)F + E 1(t, 0) + E'1(t, 0) [Eo(t)G
+ E 4(t, 0)]R- 1[G'E o(t) + E~(t,
= F'E,1(t, s)
0)],
+ E o(t)B 1(s) + Eit, 0, s)
+ Eit, 0)]R- 1 [G'E 1 (t, s) + E~(t, 0, s)],
- [Eo(t)G x
s E [0, k),
OE 3 OE3 OE3) - ( ----at + & + a;: = B ,1(s)Eit, r) + E ,1(t, s)D 1(r ) - [E'1(t, s)G
+ E 3(t, s, 0)]R- 1[G'Eit, r)
+ E 5 (t, 0, r)],
s E [0, k), r E [0, h),
341
MISCELLANEOUS EXERCISES
OE 4 OE4) - ( at + & = E o(t)D
1(s)
+ F ,E 4(t, s) + Eit, 0, s)
+ E 4(t, 0)]R- 1 [G'Eit, s) + E 5(t, 0, S)],
- [Eo(t)G X
OE5 OE5 OE5) - ( at +& +& = D
1(s)E4(t,
[0, h),
, + E 4(t, s)D 1(r)
+ E 5(t, s, 0)]R- 1 [G'Eit, r) + E 5(t, 0, r)J, s, r E [0,
- [E~(t, X
r)
SE
s)G
E 1(t, k) = Eo(t)B o, [0, k],
E 2(t, s, k) = Ei(t, s)B o,
SE
Eit, k, r) =
r E [0, h],
r),
B~Eit,
Eit, h) = Eo(t)Do, E 5(t, s, h) =
(b)
s)D o,
S E
[0, h],
Show that the optimal value function for J is given by l*(t) = (x(t), Eo(t)x(t»
+
J: s:
rr
+2 +
2(
+" (2X(t),
J:
E 1(t - s)x(t - s) dS)
(x(t - s), E 2(t, s, r)x(t - r) dr ds (x(t - s), E 3(t, s, r)u(t - r) dr ds
x(t): fEit, s)u(t - s) dS)
+ {J:(U(t 9.
E~(t,
s), E 5(t, s, r)u(t - r) dr ds.
Consider the hyperbolic control system
~
at
(W) _(01 v
°1) ~ax (W)v _F(X)(W)v = g(x)u(t),
h],
342
10
INFINITE-DIMENSIONAL SYSTEMS
where F( . ) is a continuous matrix and
g(x)
= (gl(X») gix)
is a vector function in LZ[O, 1]. The boundary conditions are
+ bov(O, t) = 0,
aow(O, t)
laol 2
+ Ibol z #- 0.
Assume the control u(t) is given in linear feedback form by u(t)
:j)
fk*(X)(:(~'
=
dx,
for some 2-dimensional vector function k E LZ([O, 1]; R Z). Associated with the closed-loop dynamics is the group of bounded operators Sk(t) having generator
L{:) == G~)
:x (:)
+
F(X)(:) + g(x) f k*(S)(:~:
Let the elements of the spectrum of Sk(t) be denoted by Aj,j Show that the Aj can be assigned any values if
I IA j -
e,
j
CTjl <
:j) as. = 0, ± 1, ±2, ....
00,
where the gj are the expansion coefficients of g relative to the eigenfunctions of the operator L o given by
defined on {(:) E
H1([0, 1]; R Z) : aow(O)
+ bov(O) = a1w(l) + b1v(l) = O}
and CTj = a + 2nij + O(I/j) for some complex a, j = 0, ± 1, ± 2, ... , i.e. the set {CTj} is the spectrum of L o . . 10. Let the system under consideration be described by the self-adjoint parabolic equation on an infinite interval ow(x, t) at = F'kw y(t) =
+ g(x)u(t),
J:oo h(x)w(x, t) dx,
w(x,O) = wo(x)
E
Lz(R).
xER, t
~
0,
343
MISCELLANEOUS EXERCISES
Assume 9 E LiR), U E L 2(R +), hE LiR), k an integer, m a positive integer. The operator Fk is a linear unbounded operator defined as
F k: ~(Fk)
-+
LiR),
where ~(Fk)
= {w(x)
d
E
L 2(R): Fkw
2
Fkw = dx 2 w(x)
+ (k
- x
E
LiR)},
2)w(x),
F'k = mth power of Fk'
dn Let Hix) = (_1)n exp x 2 -dn exp( _x 2 ) be the nth Hermite polynomial, - 00
(a)
<x<
x
00,
and define bnx == (2 nn!)-1/2 n-l/4Hix) exp( -1/2x 2).
Prove that I: is approximately reachable and observable if and only if
f:oo g(x)bix) dx -# 0, (b) (c)
n = 0, 1,2, ...
and the characteristic values A'kn of F'k are all simple, i.e., of multiplicity one, n = 0, 1,2, .... Show that for k > 1 and m even, I: is neither approximately reachable nor observable. Let m be odd, k ~ 1. Then the number of non-negative characteristic values of F'k is nk , the integer part of (k + 1)/2. Show that there exists a feedback control of the form U
=
f:oo h(x)w(x, t) dx
that stabilizes I: if and only if
f:oo g(x)bn(~)
dx -# 0,
n = 1,2, ... , nk •
11. Consider the scalar weighting pattern W(t)
= e-t_1_e-l/4t
and its associated transfer function
2.}m3
10
344
INFINITE-DIMENSIONAL SYSTEMS
where we choose the branch which gives ReJY > the branch cut along the negative real axis. (a)
Show that the following realization is canonical X = L 2[0, eFt
00),
= left translation semigroup restricted to [0, 00),
g= Whx =
(b)
°whenever Re A. > 0, with
1'"
dW
dt
,
e-tx(t) dt.
Show that this realization is equivalent to the parabolic equation ax
iJ2x
at = x(O, z)
= 0,
x(t,O)
=
x,
az2 -
u(t),
the control,
limx(t, z) = 0,
z ....co
y(t, z)
(c)
12.
=
x(t, 1)
the output.
Establish that the spectrum of F is the closed left half-plane, while the spectrum of the transfer function Z is {A.: - 00 ~ A. ~ - I}. This example shows that in contrast to finite-dimensional systems, the spectrum of the transfer matrix, i.e., the poles of Z(A.), do not necessarily coincide with the characteristic values of the system operator F.
Given the diffusion equation
1 [0 °OJ
av(x, t)
iJt
+
[0. -IJ° 1
t) -
c 21 v 1(1, t)
[0° OJ ( )
x E [0, 1], t
v(x, t) = (v 1 , v 2 )', C llV 1(O'
av(x, t) = ax
C 12V2 (O'
1 v x, t ,
~
0,
t) = 0,
+ C2 2 v2( 1, t) = 0,
v 1(x,0)
=
p(x),
°
show that if p(x) is arbitrary, C 12 = C2 2 = and the measurements are V 1(Xi, t), i = 1,2, ... , N, the system is unobservable for any N if the Xi
z(t) =
345
NOTES AND REFERENCES
are rational. On the other hand, if e l 2 #
°and the measurement is
z(t)
=
vt(O, t) then the system is observable for any p(x).
NOTES AND REFERENCES
Section 10.1 In view of the central role played in finite-dimensional linear system theory by the transfer matrix (ratios of polynomials) and its relationship to the characteristic polynomial ofthe associated system matrix F, it is no exaggeration to say that the Hilbert Basis Theorem provides the underpinning for all of finite-dimensional linear system theory. This observation can even be extended to many classes of nonlinear systems having some sort of polynomial structure. A treatment of the overall role of finiteness as an ingredient in the definitive resolution of a wide spectrum of mathematical issues remains to be written, but should prove to be a fascinating theme to pursue across the entire extent of the mathematical universe. Basic general references on the topic of distributed parameter control processes are: Curtin, R., and Pritchard, A., "Infinite-Dimensional Linear System Theory," Springer, Berlin, 1978.
Banks, S. P., "State-Space and Frequency-Domain Methods in the Control of Distributed Parameter Systems," Peregrinus, London, 1983.
Some earlier general references include Butkovskii, A., "Theory of Optimal Control of Distributed Parameter System," Elsevier, New York, 1969. Wang, P. K. c., Control of distributed parameter systems in "Advances in Control Systems, Vol. I" (c. T. Leondes, ed.) Academic Press, New York, 1964.
Section 10.2-10.3 The treatment of these sections follows that of the CurtinPritchard and Banks books cited above. See also Delfour, M., and Mitter, S., Controllability and observability for infinite-dimensional systems, SIAM J. Control, 10, 329-333 (1972).
Section 10.4 In addition to the general references given above, see Hahn, W., "Theory and Applications of Lyaponov's Direct Method," Prentice-Hall, Englewood Cliffs, N.J., 1956. Lee, E., and Markus, L., "Foundations of Optimal Control Theory," Wiley, New York, 1967. Russell, D., Controllability and stabilizability theory for linear partial differential equations: Recent progress and open questions, SIAM Review 20, 639-739 (1978).
Section 10.5 Most of this section follows the development in Baras, 1., and Brockett, R., H 2 functions and infinite dimensional realization theory, SIAM J. Control, 13, 221-241 (1975).
See also Baras, 1., and Dewilde, P., Invariant subspace methods in linear multivariable distributed systems and lumped distributed network synthesis, Proc. IEEE, 64, 160-178 (1976).
346
10
INFINITE-DIMENSIONAL SYSTEMS
Helton, J. W., Systems with infinite-dimensional state space: The Hilbert space approach, Proc. IEEE, 64, 145-160 (1976). Raibman, N. S., et. aI., Identification of distributed parameter systems, Auto. Remote Cont. 43, 703-731 (1982). Polis, M., and Goodson, R., Parameter identification in distributed systems: A 'synthesizing overview, Proc. IEEE, 64, 45-61 (1976). Kobayaski, T., Parameter identifiability for distributed parameter systems of hyperbolic type, Int. J. Sys. Sci., 11, 247-259 (1980).
Section 10.6 One of the most active groups studying the theory and computation of distributed parameter control processes is at INRIA in France, headed by J. Lions and A. Bensoussan. A good introduction to the approach (and results) of their groups is Lions, 1., Optimal control of deterministic distributed parameter systems, in "Distributed Parameter Systems: Identification, Estimation and Control," (W. H. Ray and D. G. Lainiotis, eds.,) Dekker, New York, 1978. Bensoussan, A., Control and stochastic partial differential equations in Ray and Lainiotis book cited above.
See also Lions, J., "Some Aspects of the Optimal Control of Distributed Parameter Systems," CBMS Conf. Proc., vol. 6, SIAM, Philadelphia, 1972. Gibson, J. S., The Riccati integral equations for optimal control problems on Hilbert spaces, SIAM J. Control, 17, 537-565 (1979). Zabczyk, J., Remarks on the algebraic Riccati equations in Hilbert space, Appl. Mth. & Opt.; 3, 383-403 (1976). Baum, R., Necessary conditions for distributed parameter systems with controls of fewer variables than state variables, J. Optim. Th. & Applic., 30,666-681 (1980). Balas, M., The Galerkin method and feedback control oflinear distributed parameter systems, J. Math. Anal. Applic., 91,527-546 (1983). Ahmed, N., and Teo, K., "Optimal Control of Distributed Parameter Systems," North-Holland, New York, 1981.
Section 10.7
This material was first presented in
Casti, J., and Ljung, L., Some new analytic and computational results for operator Riccati equations, SIAM J. Control, 13,817-826 (1975). Ljung, L., and Casti, J., Reduction of the operator Riccati equations, in "Proc, Int'1. Symp. on Control Theory, Numerical Methods and Computer Systems," Springer, Berlin, 1974.
Subsequently, the X- Y operators have been used in numerical work by J. P. Yvon, J. Quadrant and M. Sorine at INRIA. See Yvon, J. P., Some optimal control problems for distributed systems and their numerical solutions, in "Proc, IFAC Congress," Boston, 1975.
Index
A
Adaptive control, 72 Air pollution, 79-80 Algebraic equivalence, 118 Algebraic set, 286 Algebraic variety, 59 Almost-onto map, 286 Annihilator, 254 Arithmetic invariants, 285 Arms races, 195, 213-214 Attractor, 151
B Bang-Bang principle, 66 Bass-Roth theorem, 237 Berlekamp-Massey algorithm, 313 Betti numbers, 294-295, 297 Bezoutiant matrix, 101-104 Bode diagram, 28 Bundle isomorphism theorem, 282 C Calculus of variations, 200 Canonical form, 89 Catastrophe theory, 181 Category, 304
Cayley transform, 161 Cayley-Hamilton theorem, 48 Center, 150 Chain complex, 294 Chain of connection, 309 Characteristic vector method, 216 Circle theorem, 163-164 Closed-loop control, 12, 148 Closed-loop system structural stability, 219-220 Commutative ring, 274 Conjugate relation, 293 Constructibility, 9 Constructible continuously, 325 Continued fraction, 303 Control canonical forms, 91-97,104 Control energy, 140 Control invariants ordered, 113 Control law feedback, 113 neutral, 113 Controllability, 7 conditional, 57-58 minimal energy, 65 positive, 52-54 347
348 relative, 55-57 structural, 58-60 ControIlability index, 120 Controllability matrix extended, 50-51,60 Controllable autonomously, 64 exactly, 324 structural, 58 Controllable modes, 170 Correspondence theorem, 264 Costate, 188, 299 Cramer-Rao inequality, 239 Cycle, 295 bounding, 295
INDEX
Event constructible, 74 controllable, 36, 40-41 observable, 74 reachable, 37,41 unconstructible, 73, 75 unobservable, 73, 75 Exact sequence, 299 Extension theorem, 273 External description, 27-28 F
Degeneracy pointwise, 43 Delay operator, 275 Delays distributed, 340 Detectable, 205, 208, 210 Distribution, 339 Duality, 9, 81, 299 Dynamical system autonomous, 25 constant, 24 finite-dimensional, 25 linear, 25 smooth,25 Dynkin diagram, 307-308
Factorization algorithm, 135 Factorization canonical, 144 Feedback, 12 Feedback group, 104-105, 108,301 Feedback solutions, 192-196 Feedforward control law, 113 Fiber bundle, 279 Fibonacci sequence, 128 Filtering theory, 227-230 discrete-time, 233-234 Finiteness, 317 Fisher information matrix, 239 Floquet's theorem, 67, 177 Focus, 150 Fractional linear transformation, 283, 305 Frequency domain, 28-30 Frequency response curve, 28 Fundamental theorem of linear system theory, 249
E
G
Economic planning, 8 Energy function, 156 Equilibrium asymptotically stable, 152, 159 isolated, 152 Equivalence differentiable, 175-176 linear, 175 Nerode, 250 state-space, 302 topological, 175-176 Estimation, 18, 19 Estimation loop, 174 Euler characteristic, 297 Euler equation, 235
Gabriel's theorem, 307 Generalized X-y functions, 196-203, 234-235 Gramian observability, 139 reachability, 138 Grassmann manifold, 283 Grassmann space, 335 Grassmann variety, 279
D
H
Hankel matrix block,125 Harmonic oscillator, 150
349
INDEX Heat equation, 275, 321, 324, 332, 334, 344 nonlinear, 336 Hilbert basis theorem, 345 Hill's equation, 42 Ho algorithm, 140-141,268-269 Homogeneous space, 283 Homology groups, 295, 297, 310 Homology modules, 300 Hurwicz determinants, 155 Hyperbolic control system, 341
Impulse response function, 31-32 matrix, 100, 141 Impulse response matrix separable, 119 Industrial production, 2 Initial value problem, 192 Input functions, 22 Input values, 22 Input/output economics, 78-79 Input/output map, 2, 248 Invariant factor theorem for modules, 254-255 Invariant factor theorem for polynomial matrices, 258 Invariant factors, 64, 84, 258 Invariant topological, 176 Inverse problem, 220-227 Iterative method, 216-217
M
Macroeconomics, 37-39 Markov data, 145 Matrix circulant, 276 commutator, 302 Hankel,267-269 incidence, 293 Jacobian, 287 signature, 144 stability, 178 structural, 58 Toeplitz, 276 Maximum principle, 185, 187-189 Mayer-Vietoris sequence, 310 McMillan degree, 143,284-285,305-306,309 Meerov criterion, 177 Michailov criterion, 156 Minimal control fields, 180 Minimal polynomial, 255 Modal control, 169-173 Module torsion, 253, 255 Moments, 53, 83-84 Morphisms,304-305 N
Kalman filter, 18, 232 Kronecker canonical form, 95-96 Kronecker indices, 97, 109, 111,284,301 Kronecker product, 162
National settlement planning, 45-46 Natural numbers, 129-130 Nazarova's theorem, 308 Negative exponential method, 218-219 Node, 150 Noetherian domain, 277, 301 Nyquist criterion multivariable, 291-292, 308 Nyquist locus, 292 Nyquist plot, 28 Nyquist theorem, 162-163
L
o
Laplace transform, 30, 34 discrete, 34 Leontif matrix, 3 LQG problem, 183 Lur'e-Lefscbetz-Letov form, 92, 94, 98, 112, 169 Lyapunov function, 158, 160 Lyapunov matrix equation, 160 Lyapunov method, 156-162
Observability, 7 by functional analysis, 82-83 complete, 9 Observability index, 120 Observable autonomously, 85 continuously, 323-324 exactly, 329-330 initially, 324
K
350 Observable modes, 171 Observation energy, 140 Observation problem, 73 Observer canonical form, 97-98 Observers, 173-174 Open-loop control, 12, 148 Open-loop solutions, 185-190 Optimality vs. stability, 204 Orlando's formula, 178 Oscillatory circuit, 150 Output functions, 22 Output pole placement, 309 Output reachable, 337-338 Output regulation, 184 Output values, 22
p Pade approximation, 144,313 Parabolic equation, 342 Partial realization theorem, 271 Pattern discrimination, 298 Pell numbers, 270 Pharmacokinetics, 74-75 Phase space, 23 Pluecker coordinates, 280 Pluecker map, 299 Pole placement, 288-290 Pole-shifting theorem, 166-169,273-274, 314,325 Population migration, 10 Predator-prey systems, 13 Prime decomposition theorem, 260 Pull-back map, 281
INDEX
Reachability, 37 complete, 7 Reachable approximately, 319, 327, 343 exactly, 319, 321-322, 329 Realization, 6, 124 algorithm, 127, 137 balanced, 138-140,328 canonical, 328-329 construction, 263-271 impulse, 118 minimal, 123, 127 multi-input/multi-output, 130 partial, 129, 133-138 regular, 328-330 transfer functions, 260-263 uniqueness, 132-133 Reconstruction problem, 73 Reduced-order models, 138-140 Regulator loop, 174 Representation theorem, 257 Fticcatiequation, 24 algebraic, 207,331 asymptotic properties, 212 operator, 332-334 matrix, 192-193, 197 nonnegative solutions, 212 Riccati group, 243 Ring realization theorem, 277 RLC network, 35- 36 Routh criterion, 154 Routh-Hurwicz criterion, 11, 152-155
S
Q Q-connection, 309 Quadratic criteria, 14 Quasiprojective variety, 299 Quiver, 307 finite type, 307 tame, 307 wild, 307 R
Rank condition theorem, 269 Rational function, 286 Rational map, 287
Sampling, 46 Satellite problem, 76-77 Schubert surface, 291-292 Schwarzian derivative, 306 Semigroup, 321 Semigroup property, 23-24 Separation principle, 17-18, 231-232 Shift operator, 24, 268 Sign function method, 217 Simplicial complex, 292-298 Simulation, 259 Sobolev space, 321 Space barreled, 339
351
INDEX
Stability and feedback control, 164-169 asymptotic, 11, 151-152, 325 structural, 148, 174-177 Stabilizability, 12 Stabilizable, 205, 207-209 State, 3 State module, 255 State transition function, 22, 23 State-space decomposition theorem, 124 Stochastic control theory, 231-232 Storm sewer system, 14-17 Structure vector, 309 Submersion, 287-288, 290 Superposition principle, 190 System coefficient assignable, 306, 314 completely controllable, 36, 42, 47, 325 completely reachable, 297 constant, 47 discrete-time, 43-45, 77 exponentially stable, 326 Hamiltonian, 188, 192 internal description, 3, 4 linear, 1 module structure, 249-253 n-d,276 N-step observable, 77-78 over a field, 247 p-generic, 310 pole assignable, 314 stabilizable, 172, 327 structured, 58 time-dependent, 39-43 Systems almost constant, 177 dynamical, 21-22 equivalent, 175 over rings, 274-278 with delay, 62-64
T
Toynbee theory of history, 20 Transfer function, 30-32, 257-260 invariance, .99-101 realization, 131-132 Transformation group, 89 Transformations state-variable, 90-91 Transition matrix method, 218 Transmission line, 336 U
Unbiased estimator, 239-240 Urban traffic flow, 55- 57
v VTOL-type vehicle, 48-49
w Water quality control, 184-185 Water reservoir, 5 Wave equation, 322
x X-y operators, 333-334
y Young's diagram, 105, 109
z Zariski closed set, 286, 290 Zariski open set, 286, 290 Zero-input response, 29, 32 Zero-state response, 29
This page intentionally left blank
Mathematics in Science and Engineering Edited by William F. Ames, Georgia Institute of Technology
Recent titles
R. P. Gilbert and J. Buchanan, First Order Elliptic Systems J. W. Jerome, Approximation of Nonlinear Evolution Systems Anthony V. Fiacco, Introduction to Sensitivity and Stability Analysis in Nonlinear Programming Hans Blomberg and Raimo Ylinen, Algebraic Theory for Multivariable Linear Systems T. A. Burton, Volterra Integral and Differential Equations C. J. Harris and J. M. E. Valenca, The Stability of Input-Output Dynamical Systems George Adomian, Stochastic Systems John O'Reilly, Observers for Linear Systems Ram P. Kanwal, Generalized Functions: Theory and Technique Marc Mangel, Decision and Control in Uncertain Resource Systems K. L. Teo and Z. S. Wu, Computational Methods for Optimizing Distributed Systems Yoshimasa Matsuno, Bilinear Transportation Method John L. Casti, Nonlinear System Theory Yoshikazu Sawaragi, Hirotaka Nakayama, and Tetsuzo Tanino, Theory of Multiobjective Optimization Edward J. Haug, Kyung K. Choi, and Vadim Komkov, Design Sensitivity Analysis of Structural Systems T. A. Burton, Stability and Periodic Solutions of Ordinary and Functional Differential Equations V. B. Kolmanovskii and V. R. Nosov, Stability of Functional Differential Equations
This page intentionally left blank
