CONTROL AND DYNAMIC SYSTEMS
Advances in Theory and Applications Volume 73
CONTRIBUTORS TO THIS VOLUME MOHAMMED ABBAD ZIJAD A GANO VIC SIVA S. BANDA BOR-SEN CHEN GIANNI FERRETTI JERZY A. FILAR ZORAN GAJIC LANG HONG P. J. LYNCH CLAUDIO MAFFEZZONI SEN-CHUEH PENG RICCARDO SCA TTOLINI XUEMIN SHEN JITENDRA K. TUGNAIT EDWIN ENGIN YAZ HSI-HAN YEH
CONTROL AND DYNAMIC SYSq'EMS ADVANCES IN THEORY AND APPLICATIONS
Edited by
C. T. LEONDES University of California, Los Angeles Los Angeles, California
VOLUME 73:
TECHNIQUES IN DISCRETE-TIME STOCHASTIC CONTROL SYSTEMS
ACADEMIC PRESS San Diego New York Boston London Sydney Tokyo Toronto
This book is printed on acid-free paper.
Copyright 9 1995 by ACADEMIC PRESS, INC. All Rights Reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Academic Press, Inc. A Division of Harcourt Brace & Company 525 B Street, Suite 1900, San Diego, California 92101-4495
United Kingdom Edition published by Academic Press Limited 24-28 Oval Road, London NWl 7DX
International Standard Serial Number: 0090-5267 International Standard Book Number: 0-12-012773-3
PRINTED IN THE UNITED STATES OF AMERICA 95 96 97 98 99 00 QW 9 8 7 6
5
4
3
2
1
CONTENTS CONTRIBUTORS ................................................................................... vii PREFACE ................................................................................................ ix
Techniques for Reduced-Order Control of Stochastic Discrete-Time Weakly Coupled Large Scale Systems ..................................................
Xuemin Shen, Zijad Aganovic, and Zoran Gajic Techniques in Stochastic System Identification with Noisy Input & Output System Measurements ................................................................ 41
Jitendra K. Tugnait Robust Stability of Discrete-Time Randomly Perturbed Systems
89
Edwin Engin Yaz Observer Design of Discrete-Time Stochastic Parameter Systems ...... 121
Edwin Engin Yaz The Recursive Estimation of Time Delay in Sampled-Data Control Systems .................................................................................................... 159
Gianni Ferretti, Claudio Maffezzoni, and Riccardo Scattolini Stability Analysis of Digital Kalman Filters
207
Bor-Sen Chen and Sen-Chueh Peng Distributed Discrete Filtering for Stochastic Systems with Noisy and Fuzzy Measurements ............................................................................... 237
Lang Hong
vi
CONTENTS
Algorithms for Singularly Perturbed Markov Control Problems: A Survey ..................................................................................................
257
Mohammed Abbad and Jerzy A. Filar Control of U n k n o w n Systems via Deconvolution
................................
289
Hsi-Han Yeh, Siva S. Banda, and P. J. Lynch
I N D E X .....................................................................................................
315
CONTRIBUTORS
Numbers in parentheses indicate the pages on which the authors' contributions begin.
Mohammed Abbad (257), Departement de Mathematiques et Informatiques, Faculte des Sciences de Rabat, Morocco Zijad Aganovic (1) Cylex Systems, Inc., Boca Raton, Florida 33487 Siva S. Banda (289) Flight Dynamics Directorate, Wright Laboratory,
Wright-Patterson Air Force Base, Ohio 45433
Bor-Sen Chen (207) Department of Electrical Engineering, National Tsing Hua University, Hsin Chu, Taiwan, R.O.C. Gianni Ferretti (159) Dipartimento di Elettronica e Informazione, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, Italy Jerzy A. Filar (257) School of Mathematics, University of South Australia, The Levels, South Australia 5095 Zoran Gajic (1) Department of Electrical and Computer Engineering, Rutgers University, Piscataway, New Jersey 08855 Lang Hong (237) Department of Electrical Engineering, Wright State University, Dayton, Ohio 45435 P. J. Lynch (289) Flight Dynamics Directorate, Wright Laboratory, WrightPatterson Air Force Base, Ohio 45433 Claudio Maffezzoni (159) Dipartimento di Elettronica e Informazione, Politecnico di Milano, Piazza Leonardo da Vinci 32, 20133 Milano, Italy Sen-Chueh Peng (207) Department of Electrical Engineering, National YunLin Polytechnic Institute, Huwei, Yunlin, Taiwan, R.O.C. vii
viii
CONTRIBUTORS
Riccardo Scattolini (159) Dipartimento di Informatica e Sistemistica, Universitgt degli Studi di Pavia, 27100 Pavia, Italy Xuemin Shen (1) Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1 Jitendra K. Tugnait (41) Department of Electrical Engineering, Auburn University, Auburn, Alabama 36849 Edwin Engin Yaz (89, 121) Electrical Engineering Department, University
of Arkansas, Fayetteville, Arkansas 72701 Hsi-Han Yeh (289) Flight Dynamics Directorate, Wright Laboratory, Wright-Patterson Air Force Base, Ohio 45433
PREFACE Effective control concepts and applications date back over millennia. One very familiar example of this is the windmill. It was designed to derive maximum benefit from windflow, a simple but highly effective optimization technique. Harold Hazen's 1932 paper in the Journal of the Franklin Institute was one of the earlier reference points wherein an analytical framework for modem control theory was established. There were many other notable milestones along the way, including the MIT Radiation Laboratory series volume on servomechanisms, the Brown and Campbell book, Principles of Servomechanisms, and Bode's book, Network Analysis and Synthesis Techniques, all published shortly after mid-1945. However, it remained for Kalman's papers of the late 1950s (which provided a foundation for modem state-space techniques) and the tremendous evolution of digital computer technology (which was founded on the continuous giant advances in integrated electronics) to establish truly powerful control systems techniques for increasingly complex systems. Today we can look forward to a future that is rich in possibilities in many areas of major significance, including manufacturing systems, electric power systems, robotics, aerospace systems, and many other systems with significant economic, safety, cost, and reliability implications. Thus, this volume is devoted to the most timely theme of "Techniques in Discrete-Time Stochastic Control Systems." The first contribution to this volume is "Techniques for Reduced-Order Control of Stochastic Discrete-Time Weakly Coupled Large Scale Systems," by Xuemin Shen, Zijad Aganovic, and Zoran Gajic. There are cases of large scale complex stochastic systems whose discrete-time control can be greatly simplified by approaching their design through decomposition into weakly coupled systems with no significant impact on overall system performance. "Techniques in Stochastic System Identification with Noisy Input and Output System Measurements," by Jitendra K. Tugnait, describes parameter estimation and system identification for stochastic linear systems, a topic of active research for over three decades. In earlier works it was often assumed that the measurements of the system output are noisy, but measurements of the input to the system are well defined. This contribution is an in-depth and comprehensive treatment of the important problem of the identification
x
PREFACE
of stochastic linear systems when the input as well as the output measurements are noisy. Illustrative examples demonstrating the effectiveness of the results are presented. The next contribution is "Robust Stability of Discrete-Time Randomly Perturbed Systems," by Edwin Engin Yaz. Maintaining a control system's stability in the presence of parameter perturbations (i.e., the issue of robustness) is. of prime importance in system design, and therefore, much research effort to improve sufficient conditions for stability robustness has been expended in this area. In addition to presenting a comprehensive treatment of important results to date a number of significant new results are presented. "Observer Design for Discrete-Time Stochastic Parameter Systems," by Edwin Engin Yaz, is an in-depth treatment of linear system observer design to reconstruct the state vector of discrete-time systems with both white (timewise uncorrelated) and jump-Markov-type parameters based on noisy observations. Such stochastic parameter systems occur in many significant applications as noted in this contribution. The significant area of robustness in observer design is also examined. The next contribution is "The Recursive Estimation of Time Delay in Sampled-Data Control Systems," by Gianni Ferretti, Claudio Maffezzoni, and Riccardo Scattolini. Many engineering systems have inherent varying time delays in their system dynamics. This contribution is an in-depth treatment of the principal approaches to the recursive delay estimation problem, their applicability assumptions, and their related performance in a number of simulation experiments. The next contribution, "Stability Analysis of Digital Kalman Filters," by Bor-Sen Chen and Sen-Cheuh Peng, is an in-depth treatment of methods to ensure the stability of state estimators by Kalman filters with the finite word length inherent to digital (computer) filters. Means for treating (stabilizing) filters which might otherwise be unstable are also presented. "Distributed Discrete Filtering for Stochastic Systems with Noisy and Fuzzy Measurements," by Lang Hong, presents algorithmic techniques for distributed discrete filtering of both noisy and fuzzy measurements. Illustrative examples of application of the techniques to the multisensor integration problem are presented. The computational efficiency and capability of dynamic sensor integration are clearly demonstrated by these examples. Mohammed Abbad and Jerzy A. Filar present a comprehensive treatment of algorithms for controlled Markov chains in "Algorithms for Singularly Perturbed Markov Control Problems: A Survey." Since the uncontrolled case forms a fundamental building block, this area is also examined in depth, and numerous illustrative examples are presented. The final contribution to this volume is "Control of Unknown Systems via Deconvolution," by Hsi-Han Yeh, Siva S. Banda, and P. J. Lynch. This contribution deals with methods for on-line real-time control of systems or
PREFACE
xi
plants which are unknown, a problem which occurs in diverse engineering system problems, as noted in this contribution. The technique of on-line deconvolution generates control signals that steer the output of an unknown plant to follow a reference trajectory as closely as the numerical accuracy of the instrumentation allows, provided that the unknown plant is stable and has minimum phase, both of which might be generally expected in practice. The contributors to this volume are all to be highly commended for their contributions to this comprehensive treatment of discrete-time stochastic control systems. They have produced a work which should provide a unique and most highly useful reference on this broad subject internationally for many years to come.
This Page Intentionally Left Blank
Techniques for Reduced-Order Control of Stochastic Discrete-Time Weakly Coupled Large Scale Systems
Xuemin Shen Department of Electrical and Computer Engineering University of Waterloo Waterloo, Ontario, Canada, N2L 3G1
Zijad Aganovic Cylex Systems Inc. 6001 Broken Sound Parkway Boca Raton, FL 33487
Zoran Gajic Department of Electrical and Computer Engineering Rutgers University Piscataway, NJ 08855-0909
I. INTRODUCTION The weakly coupled systems were introduced to the control audience by (Kokotovic et al., 1969). Since then many control aspects for the linear weakly coupled systems have been studied (Medanic and Avramovic, 1975; Ishimatsu et al., 1975; Ozguner and Perkins, 1977; Delacour et al., 1978; Mahmoud, 1978; Petkovski and Rakic, 1979; Washburn and Mendel, 1980; Arabacioglu et al., 1986; Petrovic and Gajic, 1988; Gajic and Shen, 1989, 1993; Harkara et al., 1989; Gajic et al., 1990; Shen, 1990; Shen and Gajic, 1990a, 1990b, 1990c; Su, 1990; Su and Gajic, 1991, 1992; Qureshi, 1992). The general weakly coupled systems, in different set ups, have been studied by Siljak, Basar and their coworkers (Ikeda and Siljak, 1980; Ohta and Siljak, 1985, Sezer and Siljak, 1986, 1991; Kaszkurewicz et al., 1990; Siljak, 1991; Srikant and Basar, 1989, 1991, 1992a, 1992b; Skataric, 1993; Skataric et al., 1991, 1993; Riedel, 1993). The weak coupling has been also considered in the concept of multimodeling (Khalil and Kokotovic, 1978; Ozguner, 1979; Khalil, 1980; Saksena and Cruz, 1981a, 1981b; Saksena and Basar, 1982; Gajic and CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
2
XUEMIN SHEN ET AL.
Khalil, 1986; Gajic, 1988; Zhuang and Gajic, 1991) and for nearly completely decomposable Markov chains (Delebecque and Quadrat, 1981; Srikant and Basar, 1989; Aldhaheri and Khalil, 1991). The nonlinear weakly coupled control systems have been studied in only a few papers (Kokotovic and Singh, 1971, Srikant and Basar, 1991, 1992b; Aganovic, 1993; Aganovic and Gajic, 1993). The discrete-time linear control systems have been the subject of recent research (Shen and Gajic, 1990a, 1990b). In this chapter we first give an overview of the obtained results on filtering and control of discrete-time stochastic systems and then present some new results. For the reason of completeness, we study the main algebraic equations of the linear control theory, that is, the Lyapunov and Riccati equations. Corresponding parallel reduced-order algorithm for solving discrete Lyapunov and Riccati equation of weakly coupled systems are derived and demonstrated on the models of real control systems. Algorithms for both the Lyapunov and Riccati equations are implemented as synchronous ones. Their implementation as the asynchronous parallel algorithms is under investigation.
II. RECURSIVE METHODS FOR WEAKLY COUPLED DISCRETE-TIME SYSTEMS In this section parallel reduced-order algorithms for solving discrete algebraic Lyapunov and Riccati equations of weakly coupled systems and the corresponding linear-quadratic optimal control problem are presented.
A. PARALLEL ALGORITHM FOR SOLVING DISCRETE ALGEBRAIC LYAPUNOV EQUATION Consider the algebraic discrete Lyapunov equation
ATpA-P---Q,
A
Q>_0
(1)
In the case of a weakly coupled linear discrete system the corresponding matrices are partitioned as
eA3
A4'
Q-
eQ~
Q3 '
eP~
Pa
where Ai, i = l, 2, 3, 4, and Qj, j = l, 2, 3, are assumed to be continuous functions of e. Matrices /91 and /93 are of dimensions n • n and m • m, respectively. Remaining matrices are of compatible dimensions. The partitioned form of (1) subject to (2) is -
t,1 +
+
+
+
-
0
(3)
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
3
AT p1A2 - P2 + Q2 + AT p2A4 + AT pDA4 + e2AT p T A 2 -- 0
(4)
ATpDA4 - P3 + Qa + e2(ATp1A2 + A T p 2 A 4 + A T p T A 2 ) -- 0
(5)
Define, O (e 2) perturbations of (3)-(5) by
A T P l A I - Pl + QI -- O
ATl f'IA2 + AT['2A4 + ATpDA4 -
#2 +
(6)
Q2 - 0
(7)
A T PaA4 - P3 + Qa - 0 !
(8)
!
Note that we did not set e - 0 in Ais and Q:is. Under the assumption made in (1), A < 0, it follows that for sufficiently small e the matrices A1 and A4 are stable. Then the unique solutions of (6)-(8) exist. Define errors as
1'1- 1'1 + eEl _
-192-- P2 + e E 2
(9)
P3 - PD + eEa Subtracting (6)-(8) from (3)-(5), the following error equations are obtained
AT E1A1 - E1 - - A T peA3 - AT p T A 1 -- AT pDA3 AT EDA4 - Ea - - A T p1A2 - AT p2A4 - AT p T A 2
(10)
AT E2A4 - E2 - -AT1E1A2 - AT p T A 2 -- AT EaA4 The proposed parallel synchronous algorithm for the numerical solution of (10) is as follows (Shen, et al., 1991).
Algorithm 1: AT E ~ i + I ) A 1 _ E~ '+1)
_
_ A T p(2i)Aa _ A T P(i)TA1
_
AT p(i)Aa
= - ~ 2 " l **2 - **2 " 2 **4 - A T
A2
(11)
AT E~i+I)A4 - E~ i+1) = - A T E~i+I)A2 - AT p (i)T A2 - A T E(3i+l)A4 with starting points E~ ~ - E~ ~ - E (~ - 0 and p(i)_pj+e2E~i),
j_1,2,3;
i-0,1,2...
(12)
4
XUEMIN SHEN ET AL.
Now we have the following theorem. Theorem 1 Under stability assumption imposed in (1) and for ~ sufficiently small, the algorithm (11)-(12) converges to the exact solutions for Ejs with the rate of convergence of O(e2). For the proof of this theorem see (Gajic and Shen, 1993). 1. CASE STUDY: DISCRETE CATALYTIC CRACKER A fifth-order model of a catalytic cracker (Kando et al., 1988), demonstrates the efficiency of the proposed method. The problem matrix A (after performing discretization with the sampling period T = 1) is given by
Ad
--
0.011771 0.014096 0.066395 0.027557 0.000564
0.046903 0.056411 0.252260 0.104940 0.002644
0.096679 0.115070 0.580880 0.240400 0.003479
0.071586 0.085194 0.430570 0.178190 0.002561
-0.019178"] -0.022806| -0.11628 / -0.048104| -0.0006561
The small weak coupling parameter is e = 0.21 and the state penalty matrix is chosen as Q = I. The simulation results are presented in Table 1.
B. PARALLEL ALGORITHM FOR SOLVING DISCRETE ALGEBRAIC RICCATI EQUATION The algebraic Riccati equation of weakly coupled linear discrete systems is given by
P - A T P A + Q - A T P B ( B T P B + R ) - I B TPA,
R > O, Q >_ 0
(13)
with cA3
Q-
A4
~2
'
eB3
Q3
0
B4
(14)
R2
and e is a small weak coupling parameter. Due to block dominant structure of the problem mamces, the required solution P has the form P-
[
ePf
eP2] /'3
(15)
DISCRETE-TIME WEAKLY COUPLED SYSTEMS .
5
..
F i 1.00030 0.00135 0.00135 1.00540
0.54689 0.40537 -0.10944 2.08640 1.54650 -0.41752
1.93020 0.68954 -0.18620 0.68954 1.51110 -0.13802 -0.18620 -0.13802 1.03730
1.01390 0.05290 0.052897 1.20180
0.66593 0.49359 -0.13322 2.54040 1.88290-0.50820
2.20320 0.89183 -0.24071 0.89183 1.66100 -0.17841 -0.24071 -0.17841 1.04820
1.016200.06184 0.06184 1.23600
0.69091 0.51209 -0.13821 2.63570 1.95350-0.52722
2.26010 0.93400 -0.25208 0.93400 1.69230 -0.18683 -0.25208 -0.18683 1.05040
,,
I/
iJ
ii
2
!
3
1
.
0
1
6
7
0
0
.
0
6
3
7
1
i 006 , 4 i[ 1.01680 0.06409 / 0.06409 1.24450 i
,
ii
0.69604 0.51590 -0.13923 2.65520 1.96800 -0.53113
2.27170 0.94260 -0.25439 0.94260 1.69860 -0.18855 -0.25439 -0.18855 1.05090
0.69710 0.51668 -0.13944 2.65930 1.97100-0.53193
2.27410 0.94437 -0.25487 0.94437 1.70000 -0.18891 -0.25487 -0.18891 1.05100 2.27460 0.94473 -0.25497 0.94473 1.70020 -0.18898 -0.25497 -0.18898 1.05100
..
5
1.016800.06417 0.06417 1.24480
0.69731 0.51684 -0.13948 2.66010 1.97160-0.53210
6
1.01680 0.06418 0.064 18 1.24490
0.69736 0.51687 -0.13949 2.66010 1.97170-0.53213
7
1.016800.06419 0.06419 1.24490
0.69737 0.51688 -0.13950 2.66030 1.97180-0.53214
i
ii
2.27470 0.94481 -0.25499 0.94481 1.70030-0.18899 -0.25499 -0.18899 1.05100 I
II
2.27470 0.94482 -0.25499 0.94482 1.70030 -0.18900 -0.25499 -0.18900 1.05100 ,.
Table 1" Reduced-order solution of discrete weakly coupled algebraic Lyapunov equation (P(') = P,=,,a)
The main goal in the theory of weakly coupled control systems is to obtain the required solution in terms of reduced-order problems, namely subsystems. In the case of the weakly coupled algebraic discrete Riccati equation, the inversion of the partitioned matrix B T P B -t- R will produce a lot of terms and make the corresponding approach computationally very involved, even though one is faced with the reduced-order numerical problems. To solve this problem, we have used the bilinear transformation to transform the discrete-time Riccati equation into the continuous-time algebraic Riccati equation of the form
ArT P~ + Pr
+ Qr - P~S~Pr - O,
S~- B~R[1B T
such that the solution of (13) is equal to the solution of (16).
(16)
6
XUEMIN SHEN ET AL.
The bilinear transformation states that equations (13) and (16) have the same solutions if the following relations hold, that is
Ac = I -
2D -T
S~ - 2(I + A ) - I S d D -1, Sd -- B R - 1 B T
Q~ - 2D-XQ(I + A) -1
(17)
D - (I + A) T + Q(I + A ) - I S d assuming that (I + A)-1 exists. It can be seen that for weakly coupled systems the matrix
(I + A)_~ _ [ 0(1) O(e)
O(e)] 0(1)
(18t
is invertible for small values of e. It can be verified that the weakly coupled structure of the matrices defined in (14) will produce the weakly coupled structure of the transformed continuous-time matrices defined in (17). It follows from the fact that Sd from (17) and Q from (14) have the same weakly coupled structure as (18), so does D in (17). The inverse of D is also in the weakly coupled form as defined in (18). From (17) the weakly coupled structure of matrices Ac and Qc follows directly since they are given in terms of sums and/or products of weakly coupled matrices. Using the standard result from (Stewart, 1973), it follows that the method proposed in this section is applicable under the following assumption. Assumption 1 The system matrix A has no eigenvalues located at-1. It is important to point out that the eigenvalues located in the neighborhood o f - 1 will produce ill-conditioning with respect to matrix inversion and make the algorithm numerically unstable. Let us introduce the following notation for the compatible partitions of the transformed weakly coupled matrices, that is
[All ,A12] sc [sll ,s12] Ac
-
P~-
6A21
A22
5
P3 '
~2
'
eS~2
$22
~Q~2
Q22
[
(19)
(20)
These partitions have to be performed by a computer only, in the process of calculations, and there is no need for the corresponding analytical expressions. The solution of (16) can be found in terms of the reduced-order problems by imposing standard stabilizability-detectability assumptions on the subsystems. The efficient recursive reduced-order algorithm for solving (16) can be found in
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
7
(Gajic and Shen, 1993). It will be briefly summarized here taking into account the specific features of the problem under study. The O(e 2) approximation of (13) subject to (14)-(15) can be obtained from the following decoupled set of reduced-order algebraic equations P 1 A l l -+- A1T1P1 -- P 1 S l l P 1 + Qll - 0 PaA22 -+- A~2Pa
-
(21)
PsS22Ps + Q92 - 0
and (22)
P2A2 + A~P2 - - ( P I A 1 2 + A ~ P , + 012 - P1s12P,) where /~1(~)- [All(e)-
S~(e)P~(e)],
A 2 ( e ) - [A22(e)- S22(e)Ps(e)]
(23)
The unique positive semidefinite stabilizing solutions of (21) exist under the following assumption. Assumption 2 The triples (All(e), v/Sii(e), v/Qii(e)), i = 1, 2, are stabilizable-detectable. Under Assumption 2 matrices/~1 (e) and A2(e) are stable so that the unique solution of (22) exists also. If the errors are defined as
Pi = Pj +,2Ei,
j - 1, 2, 3
(24)
e(P2 + e2E2)] Ps + e2E3 J
(25)
then the exact solution will be of the form P
P1
e(P2
+e2E1 -+- eZE2) T
The fixed point parallel reduced-order algorithm for the error terms, obtained by using results from (Gajic and Shen, 1993), has the form. Algorithm 2:
EI'+I)A1 + A~EI '+~) _ p~,)s12p~,) ~ + P~')~P~') '.-722.r-2
--
1 -- -t-x21-,- 2
E(i+I)A2 + ' '~2 ~'3 ~-('+1)-
-]-P(i)Ts12p(i)
"F-
(26)
$11
_,,,-(')~s, 1-2 -(') +
(27)
--12~2 q-
P(i)TA12
$22
A~E~'+I) + E~'+I)A: + E~'+I)AI: + A~,EI '+~) :P(~')r,-'12- 2 +,~(~'+1)s11~(')2
+
~i '+~) $12 ~(;)
+
E~')$22 ~i '+~) / #
(28)
8
XUEMIN SHEN1~1 AL.
with E[ ~ - 0, E~ ~ - 0, E (~ - 0, where
P~') -
Pj + , ~ E J ' ) ,
j - 1, 2, 3;
i = 1, 2, 3, ...
(29)
and
(3o)
This algorithm satisfies converges to the exact solution of E with the rate of convergence of O(e~), that is
II
or
("11
i = 0, 1, 2,..
(31)
or equivalently
II
II- o
(32)
In summary, the proposed parallel algorithm for the reduced-order solution of the weakly coupled discrete algebraic Riccati equation has the following form: 1) Transform (13) into (16) by using the bilinear transformation defined in (17). 2) Solve (16) by using the recursive reduced-order parallel algorithm defined by (26)-(30). 1. CASE STUDY: DISCRETE MODEL OF A CHEMICAL PLANT A real world physical example (a chemical plant model (Gomathi et al., 1980)) demonstrates the efficiency of the proposed method. The system matrices are obtained from (Gomathi et al., 1980) by performing a discretization with a sampling rate T = 0.5.
A-
B T _
10 -2
95.407 40.849 12.217 4.1118 0.1305
10_2[ 0.0434 [ -0.0122
1.9643 41.317 26.326 12.858 0.5808 2.6606 -1.0453 Q = Is ,
0.3597 16.084 36.149 27.209 1.8750 3.7530 -5.5100 R = I~
0.0673 4.4679 15.930 21.442 3.6162
0.0190 1.1971 12.383 40.976 94.280
3.6076 -6.6000
0.4617 -0.9148
]
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
9
The small weak coupling parameter e is built into the problem and can be roughly estimated from the strongest coupled matrix (matrix B). The strongest coupling is in the third row, where e-
b31 3.753 -- = 0.68 b32 5.510
Simulation results are obtained using the MATLAB package for computer aided control system design. The solution of the algebraic Riccati equation, obtained from Algorithm 2, is presented in Table 2. For this specific real world example the proposed algorithm perfectly matches the presented theory since convergence, with the accuracy of 10 -4 , is achieved after 9 iterations (0.68 is = 10-4). Note very dramatic changes in the element P~J) per iteration. Thus, in this example only higher order approximations produce satisfactory results. The obtained numerical results justify the necessity for the existence of the higher order approximations for both the approximate control strategies and the approximate trajectories.
III. PARALLEL REDUCED-ORDER CONTROLLER FOR STOCHASTIC DISCRETE-TIME SYSTEMS In this section, we study the linear-quadratic Gaussian control problem of weakly coupled discrete-time systems. The partitioned form of the main equation of the optimal linear control theory m the Riccati equation, has a very complicated form in the discrete-time domain. In Section II, that problem is overcome by using a bilinear transformation which is applicable under quite mild assumption, so that the reduced-order solution of the discrete algebraic Riccati equation of weakly coupled systems can be obtained up to any order of accuracy, by using known reduced-order results for the corresponding continuous-time algebraic Riccati equation. Although the duality of the filter and regulator Riccati equations can be used together with results reported in (Shen and Gajic, 1990b) to obtain corresponding approximations to the filter and regulator gains, such approximations will not be sufficient because they only reduce the off-line computations of implementing the Kalman filter which will be of the same order as the overall weakly coupled system. The weakly coupled structure of the global Kalman filter is exploited in this section such that it may be replaced by two lower order local filters. This has been achieved via the use of a decoupling transformation introduced in (Gajic and Shen, 1989). The decoupling transformation of (Gajic and Shen, 1989) is used for the exact block diagonalization of the global Kalman filter. The approximate
10
XUEMIN SHEN ET AL.
20.9061 0.9202 0.9202 1.2382
1.8865 1.4365 18.5536 0.5259 0.3219 2.1852
1.2937 0.1971 1.2516 0.1971 1.1514 1.2887 1.2516 1.2887 21.0090
39.2244 2.5453 2.5453 1.5406
3.4212 2.3932 28.8267 0.7575 0.4428 3.3277
1.4754 0.2982 2.0621 0.2982 1.2067 1.7456 2.0621 1.7456 25.1919
4.2746 2.8594 32.9119 0.8637 0.5006 3.8272
1.5558 0.3423 2.4450 0.3423 1.2304 1.9451 2.4450 1.9451 26.7777
4.6785 3.0634 34.4250 0.9111 0.5250 4.0179
1.5911 0.3609 2.5959 0.3423 1.2399 2.0161 2.5959 2.0161 27.2107
I !
50.6375 3.6481 3.6481 1.6827
] ]
L
u
3
56.1732 4.2167 4.2167 1.7492
4
58.6366 4.4773 4.4773 1.7788
4.8566 3.1498 34.9986 0.9314 0.5351 4.0888
1.6063 0.3686 2.6519 0.3423 1.2436 2.0416 2.6519 2.0416 27.3486
5
59.6956 4.5906 4.5906 1.7915
4.9327 3.1858 35.2222 0.9400 0.5392 4.155
1.6127 0.3717 2.6727 0.3423 1.2451 2.0510 2.6727 2.0510 27.3982
60.1433 4.6387 4.6387 1.7969
4.9646 3.2008 35.3112 0.9436 0.5409 4.1258
1.6154 0.3729 2.6800 0.3729 1.2451 2.0546 2.6800 2.0546 27.4171
i
u
ii
i i
!~ 9
60.4410 4.6707 4.6707 1.8004
! i
1 ||
i
12 i_1_
i
1.6171 0.3737 2.6853 0.3729 1.2461 2.0567 2.6853 2.0567 27.4288
4.9857 3.2106 35.3676 0.9459 0.5420 4.1321 .....
60.4621 4.6730 4.6730 1.8006
4.9872 3.2113 35.3715 0.9461 0.5420 4.1326
1.6172 0.3738 2.6857 0.3729 1.2461 2.0569 2.6857 2.0569 27.4295
16 [~ 60.4636 4.6732
4.9873 3.2113 35.3717 0.9461 0.5420 4.1326
1.6172 0.3738 2.6857 0.3729 1.2461 2.0569 2.6857 2.0569 27.4296
ii
4.6732 1.8006 u
i
P, =
l
=
i
p
=pl
Table 2: Reduced-order solution of the discrete weakly coupled algebraic Riccati equation
feedback control law is then obtained by approximating the coefficients of the optimal local filters and regulators with the accuracy of O(e.N). The resulting feedback control law is shown to be a near-optimal solution of the LQG by studying the corresponding closed-loop system as a system driven by white noise. It is shown that the order of approximation of the optimal performance is O (eN), and the order of approximation of the optimal system trajectories is O(E.2N).
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
11
All required coefficients of desired accuracy are easily obtained by using the recursive reduced-order fixed point type numerical techniques developed in Section II. The obtained numerical algorithms converge to the required optimal coefficients with the rate of convergence of O(e2). In addition, only loworder subsystems are involved in the algebraic computations and no analyticity requirements are imposed on the system c o e f f i c i e n t s - which is the standard assumption in the power-series expansion method. As a consequence of these properties, under very mild conditions (coefficients are bounded functions of a small coupling parameter), in addition to the standard stabilizability-detectability subsystem assumptions, we have achieved the reduction in both off-line and online computational requirements. The results presented in this section are mostly based on the doctoral dissertation (Shen, 1990) and on the recent research papers (Shen and Gajic, 1990a, 1990b). Consider the linear discrete weakly coupled stochastic system
x~(n + 1) = Al~xl(n) + eA~2z2(n) + Bllul(n) + eB~2u2(n) +Gttw~(n) + eG~2w2(n) x2(n + 1) = eA2xz~(n) + A22x2(n) + eB2~u~(n) + B22u2(n)
(33)
with the performance criterion
s
(34)
-
n--0
where xi E ~ n i , i -- 1, 2, comprise state vectors, ui E ~ rl~i , i = 1, 2, are control inputs, Yi C ~ti, i = 1, 2, are observed outputs, wi E ~"~ and vi E ~t~ are independent zero-mean stationary Gaussian mutually uncorrelated white noise processes with intensities Wi > 0 and Y~ > 0, respectively, and zi E ~,i, i = 1, 2, are the controlled outputs given by
zx(n) = Dllxl(n) + eD12x2(n) z2(n) = eD21zl(n) + D22x2(n)
(35)
All matrices are bounded functions of a small coupling parameter e and have appropriate dimensions. In addition, it is assumed that Ri, i = 1, 2, are positive definite matrices.
12
XUEMINSHENETAL. The optimal control law is given by (Kwakernaak and Sivan, 1972) (36)
u(n) = - F ~ ( n )
with ~(n +
I) =
(37)
A ~ ( n ) + B u ( n ) + K[y(n) - C~(n)]
where
A_[All ,A12] B_[Bll ,B12] C_[Cll cA21
A22 '
eB21
B22
[eK2x K22 '
'
eC21
eF21
~C12 ] C22
(38)
F22
The regulator gain F and filter gain K are obtained from F - ( R + B r P B ) - 1BT P A
(39)
K - A Q C T ( V + C Q C T ) -1
(40)
where P and Q are positive semidefinite stabilizing solutions of the discrete-time algebraic regulator and filter Riccati equations, respectively, given by P - D TD + A TPA - A ~'PB(R + B TPB)-xB TPA
(41)
Q - AQA T - AQC r (V + CQC T)-xCQAT + GWG T
(42)
with R = diag(R1 R2),
and
W - diag(W~ W2),
o_[oll ,o12] eD21
D22
V - diag(V~ V2)
[ 11 c12]
'
eG~l
G22
(43)
44,
Due to the block dominant structure of the problem matrices the required solutions P and Q have the form
[ (e12],
P- ~P/~2 P22
0-
[Qll eQ12] EQ~2 Q22
(45)
In order to obtain the required solutions of (41) and (42) in terms of the reduced-order problems and to overcome the complicated partitioned form of the
DISCRETE-TIMEWEAKLYCOUPLEDSYSTEMS
13
discrete-time algebraic Riccati equation, we have used the method developed in the previous section, to transform the discrete-time algebraic Riccati equations (41) and (42) into the continuous-time algebraic Riccati equations of the form P S R P + D T D R -- O, SR -- B R R R 1 B T
(46)
A F Q + Q A T - QSFQ + G F W F G T -- O, SF -- C T V F 1 C F
(47)
ATRp + P A R -
such that the solutions of (41) and (42) are equal to the solutions of (46) and (47), that is P-P,
Q=Q
(48)
where AR -
BRR;~B~ -
I-
2(I +
T
A)-~BR-~BT/x~~
(49)
DT DR -- 2 A ~ I D T D ( I + A) -1 A R -- (I + A T) + D T D ( I + A ) - X B R - 1 B T
and - Z-
2(A
- 2(I + A
)-Ic
v-IcAT
G F W F G T - 2 A F 1 G W G T (I + AT) -1 A F -- (I + A) + G W G T (I
(50)
+ AT)-IcTv-1c
It is shown in Section II that the equations (46) and (47) preserve the structure of weakly coupled systems. These equations can be solved in terms of the reduced-order problems very efficiently by using Algorithm 2, which converges with the rate of convergence of O(e2). Solutions of (46) and (47) are found in terms of the reduced-order problems by imposing standard stabilizability-detectability assumptions on subsystems. Getting approximate solution for P and Q in terms of the reduced-order problems will produce savings in off-line computations. However, in the case of stochastic systems, where the additional dynamical s y s t e m - f i l t e r - has to be built, one is particularly interested in the reduction of on-line computations. In this section, the savings of on-line computation will be achieved by using a decoupling transformation introduced in (Gajic and Shen, 1989). The Kalman filter (37) is viewed as a system driven by the innovation process. However, one might study the filter form when it is driven by both
14
XUEMIN SHEN ET AL.
measurements and control. The filter form under consideration is obtained from (37) as ~ ( n + 1) - (AI~ - B ~ F ~
- e2B~9.F~2)~(n)
+e(A12 - BllF12 - B12F22)~2(n) + K l l v l ( n ) + eK12vz(n)
(51) ~2(n + 1) = e(A21 - B21Fll - B22F21)~l(rt) +(A22 - e2B21F12 - B22F22)~,2(n) + e K e l v l ( n ) + K22vz(n) with the innovation process
,~(,~) = y~(~) - , c ~ , e ~ ( n )
(52)
- c~e~(,~)
The nonsingular state transformation of (Gajic and Shen, 1989) will block diagonalize (51) under the following assumption. Assumption 3 The subsystem matrices ( A l l - B l l F l l - E2B12F21) and ( A 2 2 - B 2 2 F 2 2 - e2B21F12) have no eigenvalues in common. The transformation of interest is given by
[~l(n)
-
E2/~H e/-/
1
-~/;
(n) ]
~2 1
(53)
with
T -
I,,, -ell
eL ] I,~ - eZHL
(54)
where matrices L and H satisfy equations L(a22 -I- e.Hal2)
_
(all --
~2
a l 2 H ) L 2t- a12 -- 0
H a l l - a22H + a21 - e2Ha12H = 0
(55)
(56)
with a l l -- A l l -
BllFll-
e2B12F21
a12 -- A 1 2 - B11F12 - B12F22
a2x = A2x -
B2xFll
-
B22F21
a22 = A22 -
B22Fz2
-
e2BzlFx2
(57)
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
15
The optimal feedback control, expressed in the new coordinates, has the form
ui(n) - - f i . ~ i ( n ) - efi2r u2(n) : - - , f 2 i ~ l ( n ) - f2zPp.(n)
(58)
with ~)i(n + 1) = a i O i ( n ) + / 3 i i v i ( n )
+ eflizv2(n)
(59)
~)2(n + 1) = az~)2(n) + eflzivi(n) + fl22v2(n) where
f i i - Fll -e2F12H.
A2-
f2~ = F2~ - F22H, C~l -"
F12 + (Fll - e2F12H)L
f22 = F22 + e2(F2~ - F 2 2 I t ) L
-- e2al2H,
all
flii = K i i - e 2 L ( H + K z i ) .
(60)
c~2 : a22 -~- e2Hal2 fli2 = K i 2 - L K 2 2 - e 2 L H K i 2
fl2i = H K i l + Kzi,
fl22 = K22 + e2HKi2
The innovation processes vi and v2 are now given by v i ( n ) = y i ( n ) - diiT)i(n) - edi2~2(n)
(61)
v2(n) = y2(n) - ed2i~i(n) - d2z7)2(n) where d i i -- C i i - e~Ci2H,
di2 = C i i L ,u Ci2 - e2C12HL
d21 = C21 - C2~H.
d22 = (722 + e 2(C2i - C22H)L
(62)
Approximate control laws are defined by perturbing coefficients F i j . Kij. i . j = 1. 2; L and H by O(ek). k = 1. 2 . . . . . in other words by using k-th approximations for these coefficients, where k stands for the required order of accuracy, that is
u~)(~)
_
-
e.(k)^(k). ,2~ ,i t ~ ) -,
(63)
.(k)^(k ~2~ 0~ )(~)
with _
~k)(,j,/,
+
+ e,~(k)
~(~)v(~)(n)
^(k )(~/') 1) _ O~k)'2
+
~,(k) Vl(k),lJ/,), ']'~21
(k)
(,~)
(64)
(k)v~k) (7"/,) +/~22
where
v~k)(n)
-
yl(n)
-
d (k);'(k) ~i .,~ (n) - e d(k);'(k ..~ .,~ )(,~)
,,~)(,~)
--
y~(,~)
--
e,(k) ^(k) ( n ) , ~ ,7~
-
,(k) ~^(k) (,~) 12 a22
(65)
16
X U E M I N SHEN ET AL.
and f};) - s.~ + o(,~),
dl~ ) - d,j + o(, ~) (66)
.,~ - .,~ + o(,~1. ~}~) - ~,~ + o(, ~1 i,j = 1,2
The approximate values of j(k) are obtained from the following expression j(k)
(~
1 _ -~E E
(n)DT Dz(k)(n) + o,k, (n)Ru(k)
(67)
n--O
1
T
)r
= -~tr{D Dq~)-q-f,k Rf(k)q~k)} where z2
T
r f~k)
u(k)(n) u(k)_ [u!k)(n)],
f(k,_ [ef2~)
^(k)) (68)
,jr(k)
f~2)]
The quantities qlk) and qlk) can be obtained by studying the variance equation of the following system driven by white noise ,~(~)(n + 1)
f~(~)C
~(~) - t~(~)d(k)
,~(~)(n)
(69)
+ [ G0 /3(k)] 0 [ w(n)
~(,,)]
where c~(k)_
c~ k)
0
0 4~' '
O(k)_
,~l(k )
)
,~,~ ; ~ , ~ , ~ - , ~ ,
11
;:~
(~0)
Equation (69) can be represented in the composite form
r(~)(,~ + :) - A(~)r(~)(n) + rt(k)w(n)
(71)
with obvious definitions for A(k), II (k), I'(k)(n), and w(n). The variance of F(k)(n) at steady state denoted by q(k), is given by the discrete algebraic Lyapunov equation (Kwakernaak and Sivan, 1972)
q(k)(n + 1) -- A(k)q(k)A(k)r + II(k)WII (k)T, W - diag(W, V)
(72)
DISCRETE-TIME W E A K L Y C O U P L E D SYSTEMS
17
with q(k) partitioned as q(k) 11 T q(k) -- [q~k2)
q~k2) q~)]
(73)
On the other hand, the optimal value of J has the very well-known form, (Kwakernaak and Sivan, 1972)
jopt _ l t r [ D T D Q + P K ( C Q C T + V ) K T]
(74)
where P, Q, F, and K are obtained from (39)-(42). The near-optimality of the proposed approximate control law (63) is established in the following theorem. Theorem 2 Let z x and x2 be optimal trajectories and J be the optimal value of the performance criterion. Let x~k), z~k), and j(k) be corresponding quantities under the approximate control law u (k ) given by (63). Under the condition stated in Assumption 3 and the stabilizability-detectability subsystem assumptions, the following hold
jopt _ j(k) __ O(ek) Var{z,-x,
(k)
}-O(e2k),
i-1,2,
k = O, 1, 2, ....
(75)
The proof of this theorem is rather lengthly and is therefore omitted here. It follows the ideas of Theorems 1 and 2 from (Khalil and Gajic, 1984) obtained for another class of small parameter problems n singularly perturbed systems. These two theorems were proved in the context of weakly coupled linear systems in (Shen and Gajic, 1990c). In addition, due to the discrete nature of the problem, the proof of our Theorem 2, utilizes a bilinear transformation from (Power, 1967) which transforms the discrete Lyapunov equation into the continuous one and compares it with the corresponding equation under the optimal control law. More about the proof can be found in (Shen, 1990). A. C A S E S T U D Y : D I S T I L L A T I O N C O L U M N A real world physical example, a fifth-order distillation column control problem, (Kautsky et al., 1985), demonstrates the efficiency of the proposed method. The problem matrices A and B are
A-
10 -3
989.50 117.25 8.7680 0.9108 0.0179
5.6382 814.50 123.87 17.991 0,3172
0.2589 76.038 750.20 183.81 1.6974
0.0125 5.5526 107.96 668.34 13.298
0.0006 0.3700 11.245 150.78 985.19
18
XUEMIN SHEN ET AL.
BT_10_3[
0.0192 --0.0013
6.0733 --0.6192
8.2911 --13.339
9.1965 --18.442
0.7025 ] --1.4252
These matrices are obtained from (I~autsky et al., 1985) by performing a discretization with the sampling rate A T - 0.1. Remaining matrices are chosen as
6"-
[11000] 0
0
1
1
1 '
Q-Is,
R-h
It is assumed that G - B, and that the white noise intensity matrices are given by Wz-1, W2-2, Vz-0.1, V2-0.1 The simulation results are presented in Table 3. In practice, how the problem matrices are partitioned will determine the choice of the coupling parameter which in turn determines the rate of convergence and the domain of attraction of the iterative scheme to the optimal solution. It is desirable to get as small as possible a value of the small coupling parameter. This will speed up the convergence process. However, the small parameter is built into the problem and one can not go beyond the physical limits. The small weak coupling parameter e can be roughly estimated from the strongest coupled matrix ~ in this case matrix B. Apparently the strongest coupling is in the third row, that is b3z 8.2911 -- ~ 0.62 b32 13.339
e-
It can be seen that despite the relatively big value of the coupling parameter e = 0.62, we have very rapid convergence to the optimal solution.
j(k)
y(k) _ j
0.6989 x 10 -3
0.80528 x 10- 2 ,-
1
4'
0.75977 x 10-2 .
.
2
.
"
0.2438 x 10 -3
.
0.74277 x 10- 2
0.7380 x 10 -4
0.73887 x 10-2
0.3480 x 10 -4
,,
.
optimal
.
.
.
.
0.73546 x 10- 2
0.5000 x 10 -6
0.73539 x 10 -2
< 1.000 x 10 -7
0.73539 x 10- 2 =
.,-
Table 3: Approximate values for criterion
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
19
IV. NEW METHOD FOR OPTIMAL CONTROL AND FILTERING OF LARGE SCALE SYSTEMS In this section the algebraic regulator and filter Riccati equations of weakly coupled discrete-time stochastic linear control systems are completely and exactly decomposed into reduced-order continuous-time algebraic Riccati equations corresponding to the subsystems. That is, the exact solution of the global discrete algebraic Riccati equation is found in terms of the reduced-order subsystem nonsymmetric continuous-time algebraic Riccati equations. In addition, the optimal global Kalman filter is decomposed into local optimal filters both driven by the system measurements and the system optimal control inputs. As a result, the optimal linear-quadratic Gaussian control problem for weakly coupled linear discrete systems takes the complete decomposition and parallelism between subsystem filters and controllers. In this section, we introduce a completely new approach pretty much different than all other methods used so far in the theory of weak coupling. It is well known that the main goal in the control theory of weak coupling is to achieve the problem decomposition into subsystem problems. Our approach is based on a closed-loop decomposition technique which guarantees complete decomposition of the optimal filters and regulators and distribution of all required off-line and on-line computations. In the regulation problem (optimal linear-quadratic control problem), we show how to decompose exactly the weakly coupled discrete algebraic Riccati equation into two reduced-order continuous-time algebraic Riccati equations. Note that the reduced-order continuous-time algebraic Riccati equations are nonsymmetric, but their O(~ 2) approximations are symmetric. The Newton method is very efficient for solving these nonsymmetric Riccati equations since the initial guesses close O(e z) to the exact solutions can be easily obtained. It is important to notice that it is easier to solve the continuous-time algebraic Riccati equation than the discrete-time algebraic Riccati equation. In the filtering problem, in addition of using duality between filter and regulator to solve the discrete-time filter algebraic Riccati equation in terms of the reduced-order continuous-time algebraic Riccati equations, we have obtained completely independent reduced-order Kalman filters both driven by the system measurements and the system optimal control inputs. In the last part of this section, we use the separation principle to solve the linear-quadratic Gaussian control problem of weakly coupled discrete stochastic systems. Two real control system examples are solved in order to demonstrate the proposed methods.
20
XUEMIN SHEN ET AL.
A. L I N E A R - Q U A D R A T I C C O N T R O L P R O B L E M In this section, we present a new approach in the study of the linearquadratic control problem of weakly coupled discrete systems. By applying the new algorithm the discrete algebraic Riccati equation of weakly coupled systems is completely and exactly decomposed into two reduced-order continuous-time algebraic Riccati equations. This decomposition allows us to design the linear controllers for subsystems completely independently of each other and thus, to achieve the complete and exact separation for the linear-quadratic regulator problem. Consider the weakly coupled linear time-invariant discrete system described by (Gajic et al., 1990; Gajic and Shen, 1993) x l ( 0 ) = ~1o x2(0) = ~2o
(76) with state variables xt E R "1, x2 E R n~, and control inputs ui E R mi, i = 1, 2, where e is a small coupling parameter. The performance criterion of the corresponding linear-quadratic control problem is represented by
J-
1 oo [~ k)~ u(k)rRu(k) E ( o (k) + ]
(77)
k=0
where
z(k) - [xl(k)] u(k) - J ut(k) ] Q _ [ Q1 x2(k) ' u2(k) ' eQ~
R_[R1
0
O]
R2
eQ2 ] > 0 Qa -
(78)
> 0
It is well known that the solution of the above optimal regulation problem is given by
u(k) - -R-XBT)~(k + 1) -
-(1{ + B T p r B ) - t B T p r A x ( k )
- -Fx(k) -
[Ft -
~F3
eF2 ] F~
(79) ~(k)
where )~(k) is a costate variable and P,. is the positive semi-definite stabilizing solution of the discrete algebraic Riccati equation given by (Dorato and Levis, 1971; Lewis, 1986)
P~ - Q + A T P~[I + SP~]-IA = Q + A r P ~ A - ATprB[R + BTp~B]-tBrP~A
(80)
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
21
The Hamiltonian form of (76) and (77) can be written as the forward recursion (Lewis, 1986) +
with
1)
Hr _ [A + BR-'~BTA-TQ, _A_TQ
-BR-1BTA -T] A_ T
(82)
where Hr is the symplectic matrix which has the property that the eigenvalues of n r can be grouped into two disjoint subsets P1 and r2, such that for every A~ E 1"1 there exists Ad E P2, which satisfies A~ • Aa = 1, and we can choose either P1 or P2 to contain only the stable eigenvalues (Salgado et al., 1988). For the weakly coupled discrete systems, corresponding matrices in (79)-(82) are given by
cA2] cA3 A4 ' B -
A-[A1
eB2 ] 1BT _ [ $1 6S2] eB3 B4 , S - BReS~2 $3
[B1
(83) The optimal open-loop control problem is a two-point boundary value problem with the associated state-costate equations forming the Hamiltonian matrix. For weakly coupled discrete systems, the Hamiltonian matrix retains the weakly coupled form by interchanging some state and costate variables so that it can be block diagonalized via the nonsingular transformation introduced in (Gajic and Shen, 1989), see also (Gajic and Shen, 1993). In the following, we show how to get the solution of the discrete-time algebraic Riccati equation of weakly coupled systems exactly in terms of the solutions of two reduced-order continuous-time algebraic Riccati equations.
R "~
Partitioning vector A(k) such that A(k) - [AT(k ) A~(k)] T with Al(k) E and A2(k) E R "~, we get
x2(k + 1) ~l(k + 1) A2(k + 1)
--
H~
x2(k) ~l(k) A2(k)
(84)
It has been shown in (Gajic and Shen, 1993; in the chapter on the open-loop control, page 181) that the Hamiltonian matrix (82) has the following form
H~-
"Air eA2~ Sir
ES2~
fA3r A4r
~4r
~3r
0~r eQ2~ AT~ eAzT~ eQ3r Q4r EAT2r AT2~
(85)
22
XUEMIN SHEN ET AL.
Note that in the following there is no need for the analytical expressions for matrices with a bar. These matrices have to be formed by the computer in the process of calculations, which can be done easily. Interchanging second and third rows in (85) yields
I
xl(k 1) 1 Al(k ++ 1) x2(k + 1) )~2(k + 1)
_
A2r S rlIXl 'I
I Air ATllr eQ2r eATlr Qlr 6A3r eS3r A4r ~ 603r eATI2r Q4r A T 2 r
(86)
eT3r T4r where
Air
Tit' T3r
Ax(k) _ x2(k) A2(k)
)~2x21k)k)
S:~r
S~r
(87)
A3r
---
AT2r'
~
AT22r
Introducing the notation
v(k)- ~(k)
v(k)-
(88)
we have the weakly coupled discrete system under new notation U(k + 1) = V(k + 1) =
TlrU(k) -~-F.T2rV(]r ,T3~U(k) + T~rV(k)
(89)
Applying the transformation from (Gajic and Shen, 1989, 1993), defined by
T~
_
[I-e2LrHr
6Hr
l
--eLr] I
~(k/
_a
'
Tr I _
I
-eH~
'Lr ]
I - e2ttr Lr
(90)
v(k/l
to (89), produces two completely decoupled subsystems [ 771(k +772(k + 1)1)] - 77(k + 1) - (Tlr -e2T2~H~)~7(k)
(91)
[
(92)
~i(k + 1) ] ~2(k + 1) - ~(k + 1) - (T4r + E2H,-T2r)~(k)
DISCRETE-TIMEWEAKLYCOUPLEDSYSTEMS
23
where Lr and Hr satisfy (93)
Hr Txr -- T4r Hr + T3r --~.2HrT2rHr -- 0
L~ (T,~ + , ~ z ~ T ~ ) - ( T .
-,~T~rZ~)Lr - ~
- o
(94)
The unique solutions of (93) and (94) exist under condition that matrices TI~ and -T4~ have no eigenvalues in common, (Gajic and Shen, 1989). The algebraic equations (93) and (94) can be solved by using the Newton method (Gajic and Shen, 1989), which converges quadratically in the neighborhood of the sought solution. The good initial guess required in the Newton recursive scheme is easily obtained, with the accuracy of O (e2), by setting e -- 0 in those equations, which requires the solution of linear algebraic Lyapunov equations. The rearrangement and modification of variables in (86) is done by using the permutation matrix E of the form
il(k,1
o o ollxl(k1 E[~(k)
~(k) ]
(95)
From (88), (90)-(92), and (95), we obtain the relationship between the original coordinates and the new ones
~,(k) _ ETT, EFx(k)]
_ii~[x(k)
-
[Hlr n2r]Ix(k)] II3r
II4~
,~(k)
(96)
Since A(k) = P,.x(k), where Pr satisfies the discrete algebraic Riccati equation (80), it follows from (96) that
[ ~l(k) ]
(97)
In the original coordinates, the required optimal solution has a closed-loop nature. We have the same characteristic for the new systems (91) and (92), that is,
,72(k)
[~(~)] -
0
0]
P~b
~(k)
(98)
24
XUEMINSHENET AL.
Then (97) and (98) yield [P~o
P~bO] __ (II3r + II4rPr)(IIlr + II2~P~) - t
(99)
it can be shown from (21) that II~ = I + O ( e ) ~ IIt~ = I + O ( e ) , II2~ = O(e), which implies that the matrix inversion defined in (99) exists for sufficiently small e. Following the same logic, we can find P~ reversely by introducing [f~tr
E T T'~ z E - ft,. -
~2r]
(100)
f~3,. f~4,.
and it yields Pr -
([
f~3r + f~4r era 0
0 P~
])(
~Zr + fl2r
0
o])-,
Prb
(101)
The required matrix in (101) is invertible for small values of e since from (100) we have f~r = I + O(e) =~ f~tr = I + O(e), f~2~ = O(e). Partitioning (91) and (92) as Laar [~t(k +
I)
:::
r/2(k) ]
(k)
-
(k)
(102)
1o3,
and using (98) yield to two reduced-order nonsymmetric algebraic Riccati equations Praazr - a 4 r P r a - a 3 r + Praa2rPra = 0 (104) P~bbt~ -b4~ P ~ b - b3~ + P~bb2~P~b = 0
(105)
It is very interesting that the algebraic Riccati equation of weakly coupled discrete-time control systems is completely and exactly decomposed into two reduced-order nonsymmetric continuous-time algebraic Riccati equations (104)(105). The latter ones are much easier to solve. It can be shown that O(E 2) perturbations of (104) and (105) lead to the symmetric reduced-order discrete-time algebraic Riccati equations obtained in (Shen and Gajic, 1990b). The solutions of these equations can be used as very good initial guesses for the Newton method for solving the obtained nonsymmetric algebraic Riccati equations (104) and (105). Another way to
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
25
find the initial guesses which are O(e 2) close to the exact solutions is simply to perturb the coefficients in (104)-(105) by O(e2), which leads to the reducedorder nonsymmetric algebraic Riccati equations
p(O)-~r,~ ,r - A~lrP (~ - Olr + p(O)~-~-p(O)r,, -r r,, - 0 T D( 0 pr(O)-y-.(0)-5--.(0) _ 0
(106)
Note that the nonsymmetric algebraic Riccati equations have been studied in (Medanic, 1982). An efficient algorithm for solving the general nonsymmetric algebraic Riccati equation is derived in (Avramovic et al., 1980). The Newton algorithm for (104) is given by p(i+l) (air + a2rP (/)) - ( a 4 r - P ( i ) a 2 r ) p ( i + l ) _ aar + P(ia)a2rP(ia) i - 0, 1, 2, ... (107) The Newton algorithm for (105) is similarly obtained as
b
(blr+b2r
-
- rb b2r) p(i+rb l,"
(lo8)
i - 0, 1, 2, ... The proposed method is very suitable for parallel computations since it allows complete parallelism. In addition, due to complete and exact decomposition of the discrete algebraic Riccati equation, the optimal control at steady state can be performed independently and in parallel. The reduced-order subsystems in the new coordinates are given by (109) ~l(k + 1) = (blr + b2rPrb)~l(k)
(110)
In summary, the optimal strategy and the optimal performance value are obtained by using the following algorithm. Algorithm 3: 1) 2) 3) 4) 5)
Solve decoupling equations (93)-(94). Find coefficients air, bit, i -- 1, 2,3, 4, by using (102)-(103). Solve the reduced-order algebraic Riccati equations (104)-(105), which leads to Ira and Prb. Find the global solution of Riccati equation in terms of Pra and Prb by using ( 101). Find the optimal regulator gain from (79) and the optimal performance criterion as Jopt = 0.5xT(to)Prx(tO).
26
XUEMIN SHEN ET AL.
Example 1: In order to demonstrate the efficiency of the proposed method, a discrete model (Katzberg, 1977) is considered. The problem matrices are given by A-
I 0.964 -0.342 0.016 0.144
0.180 0.802 0.019 0.179
Q = 0.1/4,
0.017 0.162 0.983 -0.163
R =/2,
0.0191 0.179 0.181' 0.820
B-
e = 0.329,
0.019 0.180 0.005 -0.054
nl = 2,
0.0011 0.019 0.019 0.181
n2 = 2
The optimal global solution of the discrete algebraic Riccati equation is obtained as
Pexact
-
1.3998 -0.1261 0.1939 -0.4848
-0.1261 1.2748 0.5978 1.3318
0.1939 0.5978 1.1856 0.7781
-0.4848] 1.3318 0.7781 1.9994
I
Solutions of the reduced-order algebraic Riccati equations obtained from (104) and (105) are
[0.7664 oos,o] b_[12042 0140]
P~a- 0.2287 0.3184 '
1.1374 2.4557
By using the formula of (101), the obtained solution for Pr is found to be identical to Pexact and the error between the solution of the proposed method and the exact one which is obtained by. using the classical global method for solving algebraic Riccati equation is given by P e x , c t - Pr - O(10 -13)
Assuming the initialconditions as :ca'(to) = [1 m a n c e value is
Jopt =
1 1
1] the optimal perfor-
0.sxT(0)P,-z(0) = 5.2206.
B. N E W F I L T E R I N G M E T H O D F O R W E A K L Y C O U P L E D LINEAR DISCRETE SYSTEMS The continuous-time filtering problem of weakly coupled linear stochastic systems has been studied by (Shen and Gajic, 1990c). In this section, we solve the filtering problem of linear discrete-time weakly coupled systems using the problem formulation from (Shen and Gajic, 1990b). The new method is based on the exact decomposition of the global weakly coupled discrete algebraic Riccati equation into reduced-order local algebraic Riccati equations. The
DISCRETE-TIMEWEAKLYCOUPLEDSYSTEMS
27
optimal filter gain will be completely determined in terms of the exact reducedorder continuous-time algebraic Riccati equations, based on the duality property between the optimal filter and regulator. Even more, we have obtained the exact expressions for the optimal reduced-order local filters both driven by the system measurements. This is an important advantage over the results of (Shen and Gajic, 1990b; 1990c) where the local filters are driven by the innovation process so that the additional communication channels have to be used in order to construct the innovation process. Consider the linear discrete-time invariant weakly coupled stochastic system x l ( k + 1) = Alxl(k) + eA2x2(k) + Glwl(k) + eGzw2 (k)
x2(k + 1) = eAaxl(k) + A4x2(k) + eGawl(k) + G4w2(k) ~(0) = ~0, ~ ( 0 ) = ~0
(111)
with the corresponding measurements
y ( k ) - [ y l [(yk2) (] k- )
ecaC1 eC2]c4[ x+lx( 2k () ]k )
[vl(k)]v2(k)
(112)
where Xi E Pt, ni are state vectors, wi C lZrl and vi E lZII are zero-mean stationary white Gaussian noise stochastic processes with intensities Wi > 0, V~ > 0, respectively, and yi E R II, i = 1, 2, are the system measurements. In the following Ai, Gi, (7/, i - 1, 2, 3, 4, are constant matrices. The optimal Kalman filter, driven by the innovation process, is given by (Kwakernaak and Sivan, 1972) :~(k + 1) = A~(k) + K[v(k) - C:~(k)]
(113)
where A
[A1 A2] eAa
A4 ' C -
[cl
eC3
C4 ' K -
[K1 K2] eK3
K4
(114)
0]
(115)
The filter gain K is obtained from
K - AP/CT ( v + cP/CT) -1 '
v
0
V2
where P! is the positive semidefinite stabilizing solution of the discrete-time filter algebraic Riccati equation given by
t'I - APIA T
APIC T(V + C P / C T ) - I c P I A T + GWG T
(116)
28
XUEMIN SHEN ET AL.
where G -
6G3
G4
W -
0
W2
(117)
Due to the weakly coupled structure of the problem matrices the required solution P! has the form
el--
e]l
6Pf2 ] PIa
(118)
Partitioning the discrete-time filter Riccati equation given by (116), in the sense of weak coupling methodology, will produce a lot of terms and make the corresponding problem numerically inefficient, even though the problem orderreduction is achieved. Using the decomposition procedure proposed from the previous section and the duality property between the optimal filter and regulator, we propose a new decomposition scheme such that the subsystem filters of the weakly coupled discrete systems are completely decoupled and both of them are driven by the system measurements. The new method is based on the exact decomposition technique, which is proposed in the previous section, for solving the regulator algebraic Riccati equation of weakly coupled discrete systems. The results of interest which can be deduced from Section IV.A are summarized in the form of the following lemma. Lemma 1 Consider the optimal closed-loop linear weakly coupled discrete system z l ( k + 1) -
(A1 - B1F1 - e 2 B 2 F 3 ) z , l ( k ) + ~.(A2 - B1F2 - B 2 F 4 ) z 2 ( k )
x2(k + 1) - e(A3 - B3F1 - B4F3)xl(k) + (A4 - B2F2 - e 2 B 3 F 2 ) x 2 ( k ) (119) then there exists a nonsingular transformation Tr
(~2o) such that
q" 1) -- (al,. + a2rPra)~l(k) (2(k + 1) : (bl,. + b2,.P,.b)(2(k)
~'l(k
(121)
where Pra and Prb are the unique solutions of the exact reduced-order completely decoupled continuous-time algebraic Riccati equations P~al~ - a 4 ~ P ~ -a3~ + P~a2~P~ = 0 Prbblr - b4r Prb -- b3r q- Prbb2r Prb -- 0
(122)
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
29
Matrices air, bit, i = 1, 2, 3, 4, can be found from (102)-(103). The nonsingular transformation Tr is given by
T,= (II1r+ II2rPr)
(123)
Even more, the global solution Pr can be obtained from the reduced-order algebraic Riccati equations, that is
Pr -- (~3r + ~4r[ PraO
0
Known matrices f~i,., i = 1, 2, 3, 4 and II1,., II2r are given in terms of the solutions of the decoupling equations defined in Section IV.A.
The desired decomposition of the Kalman filter (113) will be obtained by using duality between the optimal filter and regulator, and the decomposition method developed in Section IV.A. Consider the optimal closed-loop Kalman filter (113) driven by the system measurements, that is
:~l(k + 1) -- (A1 - KIC1 - e2K2Ca)~l(k) + e(A2 - KIC2 - K2C4)~,z(k) + g l y l ( k ) + eg2y2(k)
+ ,g3vl(k) + g,v,~(k)
(125)
By using (111)-(112) and duality between the optimal filter and regulator, that is A ---. A T, Q ~ G W G T, B ---, C T B R _ I B T ---, c T v _ I c
(126)
the filter "state-costate equation" can be defined as
[
x(k + 1)
z(k)
where Hf --
AT + CTv-1CA-1GWGT _A_IGWGT
-cTv-tCA-I
A_ 1
]
(128)
Partitioning A(k)as A(k) - [AT(k) AT(k)]w with Al(k) E R n' and A2(k) E R n2, (127) can be rewritten as following I xxl2((kk++ l1)1 ) _
Al(k + 1)
)12(~ -~- 1)
Sl,
A~! eA2T! ~Sas- &s Qlf ~Q2! All! eA12! eQaI Q4I eA21y A22!
:~2(k)
1
Al(k) A2(k)
(129)
30
XUEMIN SHENET AL.
Interchanging the second and third rows yields
ixlkl,1 xz(k + 1)
QI! eATI
~2(]g -+- 1)
"Q3f
)~i(k+ 1)
_
sl,
All! eS3!
eQ2! eA-12! ATI $4! 6A21f Q4f A22f
__ [ TI]
6T2.f]
)~i(k)
z2(k) )i2(k)
(130)
Al(k)
ET3s T4S
z2(k)
where
T3I-
Q~s Aiif ' At21
Q2I
Q,3I A21] '
Q,4I A22]
A12]
(131)
These matrices comprise the system matrix of a standard weakly coupled discrete system, so that the reduced-order decomposition can be achieved by applying the decoupling transformation from Section IV.A to (130), which yields two completely decoupled subsystems r/l(k+ 1) r/2(k + 1)
al! a2! a3! a4!
_
[
~l(k + 1) ,~2(k + 1)
_
[
r/l(k) r/2(k)
_
[TI! -e2T2IHI] r/2(k)
bi! b2] b4l ] [ ~l(k)~2(k) -- [T4/ + E2I'IfT2~] [ ~(k)
l b31
(132)
(133)
Note that the decoupling transformation has the form of (90) with H I and L.t matrices obtained from (93)-(94) with T/I's taken from (131). By duality and Lemma 1 the following reduced-order nonsymmetric algebraic Riccati equations exist (134) Pl.al! - a4] Pla - a3I + Plaa2! PI~ = 0
Pybbl] - b4I Pfb -- b3y + Pybb2] Pyb = 0
(135)
By using the permutation matrix
ixl,k,1 9~(k) ~(k)
-E
~(k) ~(k)
1 i,.1 o o -
0 0
Z,,~ 0
0 0
o o
~(k) ~(k)
(136)
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
31
we can define HI_[II~/ IIaf
H2f]_ET[I-e2L!H ! ell!
II4/
-eLf] I E
(137)
Then, the desired transformation is given by (138)
T f = (IIl! -J- II2fP.f)
The transformation Tr applied to the filter variables (125) as 1)1
produces
T
~2]
(139)
'/)l(k 1) ] _ [~2(k + 1) +
[A1-K1CI-e2K2C3 e(A2-K1C2-K2C4)] [~)1(k) ] TfT [e(A3- gac~ - K4C3) A 4 - K4C4- e2K3C2 TT 7)2(k) +TfW[ K1
eK3
eK2 ]
K4 y(k) (140)
such that the complete closed-loop decomposition is achieved, that is ?)l(]g "]- 1) -- ( a l / -~- a2fPfa)T~l(k) 'Jl-K l y ( k ) ?02(k q- 1) - (bl! + b2yPfb)T~]2(k)-Jr- K 2 y ( k )
[K,]
where
Kz
- TfTK
(141)
(142)
It is important to point out that the matrix P/ in (138) can be obtained in terms of Pla and Plb by using (124), with f~lf, f~2f, f~af, ~41 obtained from
~3f
~4f
-eHf
[ -- e2Hf Lf
A lemma dual to Lemma 1 can be now formulated as follows. Lemma 2 Given the closed-loop optimal Kalman filter (125) of a linear
discrete weakly coupled system. Then there exists a nonsingular transformation matrix (138), which completely decouples (125) into reduced-order localfilters
32
XUEMINSHENETAL.
(141) both driven by the system measurements. Even more, the decoupling transformation (138) and the filter coefficients given in (132)-(133) can be obtained in terms of the exact reduced-order completely decoupled continuous-time Riccati equations (134) and (135). It should be noted that the new filtering method allows complete decomposition and parallelism between local filters. The complete solution to our problem can be summarized in the form of the following algorithm. Algorithm 4: 1) 2) 3) 4) 5) 6) 7)
Find Ttl, T21, 7"3I, and 7"4I from (131). Calculate I,! and H I from (93)-(94) with T/l's obtained from (131). Find ail, bil, for i = 1, 2,3,4 from (132)-(133). Solve for PI,, and Plb from (134) and (135). Find Tr from (138) with P! obtained from (124). Calculate KI and K2 from (142). Find the local filter system matrices by using (141).
C. LINEAR-QUADRATIC GAUSSIAN OPTIMAL CONTROL PROBLEM This section presents a new approach in the study of the LQG control problem of weakly coupled discrete systems when the performance index is defined on an infinite-time period. The discrete-time LQG problem of weakly coupled systems has been studied in (Shen and Gajic, 1990b). We will solve the LQG problem by using the results obtained in previous sections. That is, the discrete algebraic Riccati equation is completely and exactly decomposed into two reduced-order continuous-time algebraic Riccati equations. In addition, the local filters will be driven by the system measurements, on the contrary to the work of (Shen and Gajic, 1990b) where the local filters are driven by the innovation process. Consider the weakly coupled discrete-time linear stochastic control system represented by (Shen and Gajic, 1990b)
zl(k + 1) = Alzi(k) + EA2z2(k) + Btut(k) + EB2u2(k) z2(k + 1) = eA3:vt(k) + A4:v2(k) + eB3ux(k) + B4u2(k)
(144)
DISCRETE-TIMEWEAKLYCOUPLEDSYSTEMS
33
with the performance criterion
1
~ [~(k)~(k) + ~(k)R~(k)]
J--~E
,
k=o
R > 0
(145)
where zi E R nl, i = 1,2, comprise state vectors, u E R 'hi, i = 1, 2, are the control inputs, y E R z~, i -- 1, 2, are the observed outputs, wi E R "i, i = 1, 2, and vi E R t~, i = 1, 2 are independent zero-mean stationary Gaussian mutually uncorrelated white noise processes with intensities Wi > 0 and V / > 0, i = 1, 2, respectively, and z E R '~, i = 1, 2 are the controlled outputs given by (146)
z(k) - Dlxx(k) + D2:e2(k)
All matrices are of appropriate dimensions and assumed to be constant. The optimal control law of the system (144) with performance criterion (145) is given by (Kwakernaak and Sivan, 1972)
u(k) = -F~,(k)
(147)
with the time-invariant filter ~(k + 1) = As
+ Bu(k) + K[y(k) - C&(k)]
(148)
where
A-
cA3
A4'
B-
C-
eC3
C4
, K-
eB3
B4
eK3
(149)
K4
The regulator gain F and filter gain K are obtained from
F - (R + B T P r B ) - I B TPrA
(150)
K - A P I C T (V + CPyCT) -1
(151)
where Pr and P! are positive semidefinite stabilizing solutions of the discretetime algebraic regulator and filter Riccati equations, respectively given by
Pr -- D T D + A T p r A -- A T p r B ( R + B T p r B ) - I B T p r A
(152)
Pj - A P j A ~ - A P j C ~ (V + C P j C ~) -~ C P j A ~ + V W V ~
(153)
34
XUEMIN SHEN ET AL.
where eD3
D4'
eG3
G4
The required solutions P,. and 1:'I have the forms
[~rl P"-
eP T
esx ,PI ]
~Pr2] Pr3
' P!-
eP 2 Pya
(155)
In obtaining the required solutions of (152) and (153) in terms of the reduced-order problems, (Shen and Gajic, 1990b) have used a bilinear transformation technique introduced in (Kondo and Furuta, 1986) to transform the discrete-time algebraic Riccati equation into the continuous-time algebraic Riccati equation. In our case, the exact decomposition method of the discrete algebraic regulator and filter Riccati equations produces two sets of two reducedorder nonsymmetric algebraic Riccati equations, that is for the regulator Praalr - a 4 r P r a --aar -+" Praa2rPra -- 0
Prbblr -b4rPrb-
bz~ +
Prbb2rPrb = 0
(156)
(157)
and for the filter P/aal! - a4! P1a - a3! + Plaaz! P/a = 0
P]bbl] -- b4y P/b
- bay +
Pybb2]P]b = 0
(158)
(159)
where the unknown coefficients are obtained from previous sections. The Newton algorithm can be used efficiently in solving the reduced-order nonsymmetric Riccati equations (156)-(159). It has shown in the previous section that the optimal global Kalman filter, based on the exact decomposition technique, is decomposed into reduced-order local optimal filters both driven by th.e system measurements. These local filters can be implemented independently and they are given by 7)1(k + 1) - ( a l /
+ az1P1a)rCll(k)+ Kly(k) + Bxu(k)
~)2(k + 1) - ( b l / + b2/Pfb)T~2(k) -t- K , y ( k ) + B , u ( k ) where
3
BI| _ T_TB_ B2 J
(n 1 + II2/ ns )-TB
(160)
(161)
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
35
The optimal control in the new coordinates can be obtained as
u(k)- -F~,(k)-
-FTrT [7)1(k)] -
[7)l(k) ]
(162)
where F1 and F2 are obtained from [Vx
F2 ] - F T ~ - (R + B w P~B)-IB r P~A(III! + II2!PI) T
(163)
The optimal value of J is given by the very well-known form (Kwakernaak and Sivan, 1972)
1
Jopt - ~tr[D TDP! + P~K(CPIC T + V ) K T]
(164)
where F, K, Pr, and Pr are obtained from (150)-(153). 1. CASE STUDY: DISTILLATION COLUMN In order to demonstrate the efficiency of the proposed method, we consider a real world control system - - a fifth-order discrete model of a distillation column (Kautsky et al., 1985). This model is discretized with the sampling rate of AT -- 0.1 in (Shen and Gajic, 1990b; see also, Gajic and Shen, 1993, page 153). The system matrices are given by
A = 10 -3
989.50 117.25 8.7680 0.9108 0.0179
5.6382 814,50 123.87 17.991 0.3172
0.2589 76.038 750.20 183.81 1.6974
6.0733 -0.6192
B T _ 10_3 [ 0.0192 [ -0.0013
0.0125 5.5526 107.96 668.34 13.298
8.2911 -13.339
0.0006 0.3700 11.245 150.78 985.19 9.1965 -18.442
0.7025 ] -1.4252
]
and the other matrices are chosen as C-
1 0
1 0
0 1
0 1
0] D T D - I s , 1 '
R-I2
It is assumed that G - I~ and that the white noise processes are independent and have intensities
W--Is,
V - - 0.1 • I2
36
XUEMINSHENETAL.
It is easy to see that this model possesses the weakly coupled structure with ni = 2, 2 = 3, ande = 8.2911/13.3~9= 0.62. The obtained solutions for the LQG control problem are summarized as following. The completely decoupled filters driven by measurements y are given by ~l(k+l)-
[0.3495 -0.2126
-0.5914] [0.6426 0.4551 ~l(k)+ 0.3277
+
7)2(k + 1) -
0.6607 -0.0833 -0.6043
[0.0163 -0.0478 -0.1302
0.0065 0.0027
0.0298 0.4020 -0.6039
0.0670] 0.0403] y(k)
-0.012] u(k ) 0.0054
-0.1464 ] -0.1255 ,~2(k) 0.3915
0.1037] [0.064-0.0080] 0.2649 / y(k)+ 0.0085 -0.0165 0.57991 0.0013-0.0103
u(k)
The feedback control in the new coordinates is obtained as u(k) -
[0.37320.4318] [_0.0739_0.17550.4207] -0.4752
-0.5981 ~x(k)+
0.0961
0.2230
-0.6128 ~2(k)
The difference of the performance criterion between the optimal value, Jopt, and the one of the proposed method, J, is given by Jopt = 216.572 J - Jopt - 2.6489 x 10 -11
Simulation results are obtained by using the software package MATLAB (Hill, 1988).
V. REFERENCES 1.
Aganovic, Z. Optimal Control of Singularly Perturbed and Weakly Coupled Bilinear Systems, Ph. D. Dissertation, Rutgers University, 1993.
2.
Aganovic, Z. and Z. Gajic. Optimal control of weakly coupled bilinear systems. Automatica, 29(1993): 1591-1593. Aldhaheri, R. and H. Khalil. Aggregation of the policy iteration method for nearly completely decomposable Markov chains. IEEE Trans. Automatic Control, AC-36(1991): 178-187.
3.
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
4.
5.
6. 7. 8. 9.
10. 11. 12.
13. 14.
15.
16. 17. 18. 19.
20.
37
Arabacioglu, M., M. Sezer, and O. Oral. Overlapping decomposition of large scale systems into weakly coupled subsystems, 135-147, in Computational and Combinatorial Methods in System Theory, C. Byrnes and A. Lindquist, Eds., North Holland, Amsterdam, 1986. Avramovic, B., P. Kokotovic, J. Winkelman, and J. Chow. Area decomposition for electromechanical models of power systems. Automatica, 16(1980): 637--648. Delebecque, F. and J. Quadrat. Optimal control of Markov chains admitting strong and weak interconnections. Automatica, 17(1981): 281-296. Delacour, J., M. Darwish, and J. Fantin. Control strategies of large-scale power systems. Int. J. Control, 27(1978): 753-767. Dorato, P. and A. Levis. Optimal linear regulators: the discrete time case. IEEE Trans. Automatic Control, AC-16(1970): 613-620. Gajic, Z. Existence of a unique and bounded solution of the algebraic Riccati equation of the multimodel estimation and control problems. Systems & Control Letters, 10(1988): 185-190. Gajic, Z. and H. Khalil. Multimodel strategies under random disturbances and imperfect partial observations. Automatica, 22(1986): 121-125. Gajic, Z. and X. Shen. Decoupling transformation for weakly coupled linear systems. Int. J. Control, 50(1989): 1517-1523. Gajic, Z., D. Petkovski, and X. Shen. Singularly Perturbed and Weakly Coupled Linear Control Systems ~ A Recursive Approach. Springer-Verlag, Lecture Notes in Control and Information Sciences, 140, New York, 1990. Gajic, Z. and X. Shen. Parallel Algorithms for Optimal Control of Large Scale Linear Systems, Springer-Verlag, London, 1993. Gomathi, K., S. Prabhu, and M. Pai. A suboptimal controller for minimum sensitivity of closed-loop eigenvalues to parameter variations. IEEE Trans. Automatic Control, AC-25(1980): 587-588. Harkara, N., D. Petkovski and Z. Gajic. The recursive algorithm for optimal output feedback control problem of linear weakly coupled systems. Int. J. Control, 5t)(1989): 1-11. Hill, D. Experiments in Computational Matrix Algebra, Random House, New York, 1988. Ikeda, M. and D. Siljak. Overlapping decompositions expansions and contractions of dynamic systems. Large Scale Systems, 1(1980): 29-38. Ishimatsu, T., A. Mohri, and M. Takata. Optimization of weakly coupled systems by a two-level method. Int. J. Control, 22(1975): 877-882. Kando, H., T. Iwazumi, and H. Ukai. Singular perturbation modeling of large-scale systems with multi-time scale property. Int. J. Control, 48( 1988): 2361-2387. Kaszkurewicz, E., A. Bhaya, and D. Siljak. On the convergence of parallel asynchronous block-iterative computations. Linear Algebra and Its Applications, 131(1990): 139-160.
38
XUEMIN SHEN ET AL.
21. Katzberg, J. Structural feedback control of discrete linear stochastic systems with quadratic cost. IEEE Trans. Automatic Control, AC-22(1977): 232-236. 22. Kautsky, J., N. Nichols, and P. Van Douren. Robust pole assignment in linear state feedback. Int. J. Control, 41 (1985): 1129-1155. 23. Khalil, H. Multi-model design of a Nash strategy. J. Optimization Theory and Applications, 31( 1980): 553-564. 24. Khalil, H. and P. Kokotovic. Control strategies for decision makers using different models of the same system. IEEE Trans. Automatic Control, AC-23(1978): 289-298. 25. Khalil, H. and Z. Gajic. Near-optimum regulators for stochastic linear singularly perturbed systems. IEEE Trans. Automatic Control, AC-29(1984): 531-541. 26. Kokotovic, P., W. Perkins, J. Cruz, and G. D'Ans. e---coupling approach for near-optimum design of large scale linear systems. Proc. IEE, Part D, 116(1969): 889-892. 27. Kokotovic, P. and G. Singh. Optimization of coupled nonlinear systems. Int. J. Control, 14(1971): 51-64. 28. Kondo, R. and K. Furuta. On the bilinear transformation of Riccati equations. IEEE Trans. Automatic Control, AC-31(1986): 50-54. 29. Kwakernaak, H. and R. Sivan, Linear Optimal Control Systems, Wiley Interscience, New York, 1972. 30. Lewis, F. Optimal Control, Wiley, New York, 1986. 31. Mahmoud, M. A quantitative comparison between two decentralized control approaches. Int. J. Control, 28(1978): 261-275. 32. Medanic, J. Geometric properties and invariant manifolds of the Riccati equation. IEEE Trans. Automatic Control, AC-27(1982): 670--677. 33. Medanic, J. and B. Avramovic. Solution of load-flow problems in power stability by E---coupling method. Proc. lEE, Part D, 122(1975): 801-805. 34. Ohta, Y. and D. Siljak. Overlapping block diagonal dominance and existence of Lyapunov functions. J. Math. Anal. Appl., 112(1985): 396-410. 35. Ozguner, U. and W. Perkins. A series solution to the Nash strategies for large scale interconnected systems. Automatica 13(1977): 313-315. 36. Ozguner, U. Near-optimal control of composite systems: the multi timescale approach. IEEE Trans. Automatic Control, AC-24(1979): 652-655. 37. Petkovski, D. and M. Rakic. A series solution of feedback gains for output constrained regulators. Int. J. Control, 29(1979): 661-669. 38. Petrovic, B. and Z. Gajic. Recursive solution of linear-quadratic Nash games for weakly interconnected systems. J. Optimization Theory and Applications, 56(1988): 463--477. 39. Power, H. Equivalence of Lyapunov matrix equations for continuous and discrete systems. Electronic Letters, 3(1967): 83.
DISCRETE-TIME WEAKLY COUPLED SYSTEMS
39
40. Qureshi, M. Parallel Algorithms for Discrete Singularly Perturbed and Weakly Coupled Filtering and Control Problems, Ph. D. Dissertation, Rutgers University, 1992. 41. Riedel, K. Block diagonally dominant positive definite approximate filters and smoothers. Automatica, 29(1993): 779-783. 42. Saksena, V. and J. Cruz. A multimodel approach to stochastic Nash games. Automatica, 17(1981a): 295-305. 43. Saksena, V. and J. Cruz. Nash strategies in decentralized control of multiparameter singularly perturbed large scale systems. Large Scale Systems, 2(1981b): 219-234. 44. Saksena, V. and T. Basar. A multimodel approach to stochastic team problems. Automatica, 18(1982): 713-720. 45. Salgado, M., R. Middleton, and G. Goodwin. Connection between continuous and discrete Riccati equation with applications to Kalman filtering. Proc. IEE, Part D, 135(1988): 28-34. 46. Sezer, M. and D. Siljak. Nested E---decomposition and clustering of complex systems. Automatica, 22(1986): 321-331. 47. Sezer, M. and D. Siljak. Nested epsilon decomposition of linear systems: Weakly coupled and overlapping blocks. SlAM J. Matrix Anal. Appl., 3(1991): 521-533. 48. Siljak, D. Decentralized Control of Complex Systems, Academic Press, Cambridge, MA, 1991. 49. Shen, X. Near-Optimum Reduced-Order Stochastic Control of Linear Discrete and Continuous Systems with Small Parameters, Ph. D. Dissertation, Rutgers University, 1990. 50. Shen, X. and Z. Gajic. Optimal reduced-order solution of the weakly coupleddiscrete Riccati equation. IEEE Trans. Automatic Control, AC35(1990a): 60(0602. 51. Shen, X. and Z. Gajic. Approximate parallel controllers for discrete weakly coupled linear stochastic systems. Optimal Control Appl. & Methods, 11(1990b): 345-354. 52. Shen, X. and Z. Gajic. Near-optimum steady state regulators for stochastic linear weakly coupled systems. Automatica, 26(1990c): 919-923. 53. Shen, X., Z. Gajic, and D. Petkovski. Parallel reduced-order algorithms for Lyapunov equations of large scale linear systems. Proc. IMACS Symp. MCTS, Lille, France, (1991): 697-702. 54. Skataric, D. Quasi Singularly Perturbed and Weakly Coupled Linear Control Systems, Ph.D. Dissertation, University of Novi Sad, 1993. 55. Skataric, D., Z. Gajic, and D. Petkovski. Reduced-order solution for a class of linear quadratic optimal control problems. Proc. Allerton Conf. on Communication, Control and Computing, Urbana, (1991): 440--447.
40
XUEMIN SHEN ET AL.
56. Skataric, D., Z. Gajic, and D. Arnautovic. Reduced-order design of optimal controller for quasi weakly coupled linear control systems. Control Theory and Advanced Technology, 9(1993): 481-490. 57. Srikant, R. and T. Basar. Optimal solutions of weakly coupled multiple decision maker Markov chains. Proc. Conf. on Decision and Control. Tampa, FL, (1989): 168-173. 58. Srikant, R. and T. Basar. Iterative computation of noncooperative equilibria in nonzero-sum differential games with weakly coupled players. J. Optimization Theory and Applications. 71(1991): 137-168. 59. Srikant, R. and T. Basar. Asymptotic solutions of weakly coupled stochastic teams with nonclassical information. IEEE Trans. Automatic Control. AC37(1992a): 163-173. 60. Srikant, R. and T. Basar. Sequential decomposition and policy iteration schemes for M-player games with partial weak coupling. Automatica. 28(1992b): 95-105. 61. Stewart, G. Introduction to Matrix Computations. Academic Press, New York, 1973. 62. Su, W. Contributions to the Open and Closed Loop Control Problems of Linear Weakly Coupled and Singularly Perturbed Systems. M.S. Thesis, Rutgers University, 1990. 63. Su, W. and Z. Gajic. Reduced-order solution to the finite time optimal control problems of linear weakly coupled systems. IEEE Trans. Automatic Control, AC-36(1991): 498-501. 64. Su, W. and Z. Gajic. Parallel algorithm for solving weakly coupled algebraic Riccati equation. AIAA J. Gudance, Dynamics and Control, 15( 1992): 536--538. 65. Washburn, H. and J. Mendel. Multistage estimation of dynamical weakly coupled in contiuuous-time linear systems. IEEE Trans. Automatic Control, AC-25(1980): 71-76. 66. Zhuang, J. and Z. Gajic. Stochastic multimodel strategy with perfect measurements. Control ~ Theory and Advanced Technology, 7(1991): 173-182.
Techniques in Stochastic System Identification with Noisy Input 8z Output System Measurements Jitendra
K. Tugnait
D e p a r t m e n t of Electrical Engineering A u b u r n University Auburn, A l a b a m a 36849
I. I N T R O D U C T I O N Parameter estimation and system identification for stochastic linear systems have been a topic of active research for over three decades now [7],[19],[20],[37],[45]. It is often assumed that the measurements of the system output are noisy but the measurements of the input to the system are perfect. The problem considered in this chapter is that of identification of stochastic linear systems when the input as well as the output measurements are noisy. An interesting example of system identification with noisy input may be found in [38] where the problem of (off-line) estimation of certain parameters associated with the dynamics of a submerged undersea vehicle is studied. The various control inputs (the rudder angle, the stern-plane angle, etc.) and the corresponding motion variables (pitch angle, yaw rate, etc.) can only be remotely sensed, hence, are contaminated with sensor noises. This is a typical example of situations where the model considered in this chapter should prove to be useful. In multivariate time series problems where one is interested in exploring the relationship (transfer function) between two groups of variables, it is CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
42
JITENDRAK. TUGNAIT
more logical to "symmetrically" model the system by allowing all measured variables to be noisy [29]. Such models are called errors-in-variables models in the econometrics literature [6]. In this chapter we consider a specific class of systems, namely, those where the input process is non-Gaussian and the measurement noise at the input as well as the output is Gaussian if the input process has symmetric probability density function (PDF). The noise processes are allowed to be non-Gaussian with symmetric PDF if the input process has asymmetric P DF. Clearly, this model may not be always appropriate but there are several situations of practical interest where such assumptions are valid. For instance, a pseudo-random binary sequence is often used to probe a control system for identification purposes [7], [19], [20]; such sequences are clearly non-Gaussian with nonvanishing trispectrum. We will now consider another such case in some detail. A signal processing application of considerable interest is that of differential time delay estimation in oceanic environments for passive sources. In this problem it is desired to estimate the time-difference of arrivals of a passive acoustic source signal at two sensors. There exist several approaches to accomplish this by exploitation of the second-order statistics (correlation and cross-correlation functions) of the data [10],[12]. The mathematical model underlying this approach is that of linear system identification with measurements at one of the sensors being modeled as the input to the system and the measurements at the other sensor being modeled as the system output. In a non-dispersive medium the linear system is just a delay (time delay to be estimated), whereas in a dispersive medium it is a general linear system [26]. A frequently made assumption is that the noises at the two sensors are uncorrelated Gaussian processes. This is clearly true for receiver induced noises. But it is not true for ambient noise. Under certain conditions, the ambient noise has been found to be a colored Gaussian process that is correlated both temporally as well as spatially [10],[11],[28]. The model considered in this chapter is quite appropriate for this problem. Past approaches to the problem of stochastic linear system identification may be divided into two classes: those that exploit only the second order statistics, and those that use higher (higher than second) order cumulant statistics. A good survey of the work done prior to about 1980 is given in [2]. For later work, see [3]-[9], [16]-[18], [29] and [32]. Higher order statistics have been exploited in [5], [6], [12]-[14], [31], [33]-[35], [39], [40] and [44]. Shderstrhm [1],[2] allows only white additive noise at the input, and furthermore, the input and the output noises are assumed to be mutually uncorrelated. Most of the early work in this area has been done in econometrics where such models have been called errors-in-variables models. When only second order statistics are exploited, it is known that, in general, there does not exist a unique solution [3]-[6],[9]. Therefore, atten-
STOCHASTIC SYSTEMIDENTIFICATION
43
tion has been focused on characterization of the class of transfer functions which fit the data. Fairly complete results are available for scalar dynamic case [3]-[6] but the multivariable case remains incomplete[9]. The use of higher order cumulant statistics ([23],[24])can, in principle, yield consistent parameter estimates. Deistler [5] (see also [14]) has shown how to estimate the transfer function of an SISO system in the frequency domain by use of the higher order cumulant spectrum of the output and the higher order cumulant cross-spectrum of the input-output record. That is, [5] presents a non-parametric approach. In [12] a special case (FIR filters) has been considered where output third-order cumulants and input-output third-order cross-cumulants have been used to derive linear parameter estimators. Implicit in [12] is the use of a (higher order) persistency of excitation condition. No general results are currently available to indicate when such a condition would hold. Consistency can be shown for the case of fourth-order cumulants [44]. Instrumental variable type approaches have been presented in [31] using third-order cumulants and in [35] using fourth-order cumulants of the inputoutput record. The consistency results of [31] and [35] have been proven only for i.i.d. (independent and identically distributed) inputs. Otherwise one needs to check the invertibility of a m a t r i x which depends upon the unknown parameters to be estimated; therefore, the invertibility condition check is not practical. The consistency results of the approach proposed in this chapter hold under far more general conditions. On the other hand we confine our attention to scalar models whereas [31] and [35] deal with multivariable models. In [34] a novel cost function involving the third-order cumulants of the input-output data has been proposed and it has been shown to be proportional to a conventional mean-square error criterion based upon noiseless data. Its extension to the fourth-order cumulant case has not been provided in [34]. Note that when the system input has a symmetric PDF, its thirdorder statistics will vanish rendering the approaches based upon the thirdorder statistics useless. One example is that of a pseudo-random binary sequence as the system input. Consistency of the approach of [34] has been established under several restrictive conditions such as the system input u(t) is a linear process given by u(~) - G(q-1)e(~) (see Sec. II for notation) where {e(t)} is a zero-mean, i.i.d, process with nonzero third cumulant at zero lag and G(q) represents a stable, linear model such that G(1) ~= 0. It should be noted that unlike the second-order statistics case, one can not, in general, model a stationary random process with a given higher-order cumulant spectrum as having been generated by driving a linear system with an i.i.d, sequence [27]. In this chapter we do not require any such constraint. Moreover, our approaches also apply to fourth-order statistics case unlike [34]. Also [34] requires that the noise processes, if non-Gaussian
44
JITENDRA K. TUGNAIT
with symmetric PDF, should be linear processes. We do not need such an assumption in this chapter. In [40] several linear/iterative approaches using the auto-and/or cross- third-order cumulants of the input-output processes have been presented. Conditions under which the proposed approaches will yield consistent parameter estimators have not been provided in [40]. Indeed we have given examples in [44] where the cross-cumulant method of [40],[12] may yield biased results. The simulation results presented in [40] show that the cross-cumulants-based approach of [40] outperforms the approach of [34] by a wide margin for the presented example. In [28] and [39] the square root of the magnitude of the fourth cumulant of a generalized error signal is proposed as a performance criterion for parameter estimation. Both single-input single-output and multiple-input multiple-output models have been considered in [39]. Strong consistency of the proposed parameter estimator has been established for linear inputs in [39] for Gaussian noise processes. The approach of [281,[39] results in a nonlinear estimator that requires a good initial guess for convergence; unfortunately, no method for reliable initialization was provided in [39]. The approach of [28],[39] is briefly discussed in Sec. II. In this chapter two new classes of parametric frequency domain approaches are proposed for estimation of the parameters of scalar, linear errors-in-variables models. One of the proposed approaches is a linear estimator where using the bispectrum of the input and the cross-bispectrum of the input-output, the system transfer function is first estimated at a number of frequencies exceeding one-half the number of unknown parameters. The estimated transfer function is used to estimate the unknown parameters using an overdetermined linear system of equations. In the second approach a quadratic transfer function matching criterion is optimized by using the linear estimator as an initial guess. Both the parameter estimators are shown to be consistent in any measurement noise that has symmetric PDF. The input to the system need not be a linear process but must have nonvanishing bispectrum. These two classes of approaches can be modified to exploit integrated polyspectrum, either bispectrum or trispectrum. The integrated polyspectrum is defined as a cross-spectrum between the process and a nonlinear function of the process; see Sec. III for further details. As discussed in Sec. III, integrated polyspectrum (bispectrum or trispectrum) is computed as a cross-spectrum; hence, it is computationally cheaper than the corresponding polyspectrum particularly in the case of the fourth-order cumulant spectrum. If the non-Gaussian input to the linear system has a symmetric P DF, its bispectrum and the integrated bispectrum will vanish whereas its trispectrum and the integrated trispectrum will not, provided that the fourth cumulant of the input 0'4= is nonzero. Extension of the bispectrum-based approaches to trispectrum-based approaches is computationally complex and the resulting estimators are likely to have poor statis-
STOCHASTIC S Y S T E M IDENTIFICATION
45
tical performance because of the high variance of the trispectrum estimators [24]. Herein lies the significance of the integrated polyspectrum-based approaches which apply with almost equal computational and programming ease to both cases, those involving integrated bispectrum as well as those concerned with integrated trispectrum. Compared to [39] we do not need the input to be a linear process for consistency to hold true. Compared to [5] we use parametric models. The chapter is organized as follows. The approach of [28],[39] is briefly discussed in Sec. II. In Sec. III a more precise statement of parameter estimation problem under consideration is provided along with a definition and some analysis of the integrated polyspectrum of interest in this chapter. The bispectrum-based approaches are described in Sec. IV. Sec. IV includes estimation of the auto- and cross-bispectrum, their large sample properties, a linear parameter estimator and a nonlinear parameter estimator which is asymptotically equivalent to a negative log-likelihood function. The integrated polyspectrum (bispectrum and trispectrum) based approaches are described in Sec. V which includes estimation of the integrated bispectrum and integrated trispectrum, their large sample properties, two linear parameter estimators and two nonlinear parameter estimators which are asymptotically equivalent to some negative log-likelihood functions. Consistency of the proposed parameter estimators is established is Sec. VI under some mild sufficient conditions. Finally, two simulation examples are presented in Sec. VI to illustrate the proposed approaches and to compare them with several existing approaches. Certain technical details may be found in the Appendix.
II. A T I M E - D O M A I N T E C H N I Q U E II.A. M O D E L A S S U M P T I O N S To keep the notation and the basic ideas simple, we initially consider only a causal SISO model; later, we generalize to MIMO models. Let u(t) and s(t) denote the "true" input and output, respectively, at (discrete) time t. The two are assumed to be related via a finite-dimensional ARMA(na, nb) model
A(q- 1)s(t) : B(q- 1)u(t)
(ILl)
A(q -1) "- 1 + ~ aiq -i
(II-2)
where
/-----1
46
JITENDRA K. TUGNAIT
B(q- 1) ._ ~
biq-i
(11-3)
i=1
and q-1 is the backward shift operator, i.e., q-is(t) - s ( t - 1), etc. As in [1] and [7], for example, we have assumed for convenience only that no term bou(t) is present, i.e., that there is at least one delay in the system. Explicitly, we may write (II-1) as 'r~ffi
nb
s(t) + E ais(t - i) - E biu(t - i). i=1
(11-4)
i=1
It is assumed that the input u(t) is generated by driving an asymptotically stable linear filter H(q -1) by another process e(t)
u ( t ) - H(q-1)e(t)
(II-5)
H(q -1) -- C(q-1)/D(q-1),
(11-6)
where
and C(q- 1) and D(q- ~) are finite-dimensional. The processes {u(t)} and {s(t)} are not available for measurement. But we can measure noise-contaminated input and output
9(t) - u ( t ) + vi(t)
(11-7)
y(t) - s(t) + vo(t)
(11-8)
The following conditions on the above model are assumed to hold.
( H I ) A(z) ~ 0 for [z[ _< 1 where z is a complex variable. Moreover, A(z) and B(z) are coprime (they have no common factor).
(H2)
The process {e(t)} is a zero-mean, i.i.d, non-Gaussi=r~ sequence with 74~ := E{e4(t)} - 3[E{e2(t)}] 2 76 0 and E{e8(t)} < oo.
(H3) The noise processes {vi(t)} and {v0(t)} are stationary, zero mean, jointly a=ussi=~ and independent of {e(t)}. Moreover, there exist 0<M
Icov{vi(t), vi(tl)}l <_MB It-ill for all t and tl [cov{vo(t), vi(tl)}] <_ MB It-ill for all t and tl [cov(vo(t), vo(tl)}[ _< M/3 I'-t~l
for all t and tl.
STOCHASTICSYSTEMIDENTIFICATION
47
( H 4 ) D(z) r 0 for I~1 ~ 1 so that the filter H(q -1) is asymptotically stable. Moreover, H(q -1) ~ O. R e m a r k 1. Condition (H1) is a standard assumption needed to ensure asymptotic stability of the underlying system; similar remarks apply to (H4). It is a standing assumption in the rest of the paper that the system (II-1) has been in operation for a long time so that under (H1), (H2) and (H4), the output {s(t)} for t > 1 is "fourth order wide-sense stationary," i.e., its cumulants of order up to four are invariant to any time shifts. If {vi(t)} and {Vo(t)} are generated by driving asymptotically stable finite-dimensional linear systems by white Gaussian sequences, then the restrictions on the covariances required in (H3) are satisfied. [] Define the vector of the unknown parameters in the system model as
O(na, rib) "-- (al, a2, " " , an., bl, b2, " " , bn,).
(II-9)
Let 0o, n~o and nbo denote the true values of the respective vector a n d / o r parameters. Also define the following set | "-- {/91 (HI) holds }. The objective is to estimate the transfer function B ( q - 1 ) / A ( q -1) (equivalently, 0 ) given a data record {a(i), y(i), 1 < i <_ N}. It is assumed that the intended application is such that estimates of the noise statistics are not needed; see, for example, the applications such as time delay estimation and system parameter estimation discussed in [26] and [38], respectively, and in Section I. If estimates of the noise statistics are needed, they can be obtained either in a second step after identifying the system parameters, or simultaneously by a combined second and fourth order statistics based method. C u m u l a n t S t a t i s t i c s : In the sequel we will use higher order cumulant statistics of an error signal. It is therefore useful to recall certain results from [23] and [24] that we will use later. The (joint) fourth cumulant of the zero-mean random variables f ( t l ) , f(t2), f(t3) and f ( t 4 ) i s given by cum4{f(tl), f(t2), f(t3), f(t4)} - E { f ( t l ) f ( t 2 ) f ( t 3 ) f ( t 4 ) }
-E{f(tl)f(t2)}E{f(ta)y(t4)}-
-E( f(t ) f(t,
E{f(tl)f(ta)}E{f(t2)y(t4)}
f(t )}
(II-lO)
provided the appropriate moments exist. If a group of zero-mean random variables, say f ( t l ) and f(t4), are independent of the remaining zero-mean random variables ( f ( t 2 ) and f(t3), for example), then cum4{Y(tl), f(t2), f(t3), f(t4)} -- 0.
(11-11)
48
J I T E N D R A K. T U G N A I T
Let {g(i)} be the impulse response function of an SISO linear time-invariant system that is driven by a random sequence {e(t)} that obeys (H2). Let { f ( t ) } denote the corresponding output sequence. That is, we have oo
f(t) -
~
g(i)e(t - i).
(II-12)
i=-oo
Then oo
CUMa{f(t)} "- c u m , { f ( t ) , f(t), f(t), f(t)} - 74e E
g4(i)
(II-13)
i=-oo
if Ig(i)l < Mf~lil for some 0 < f~ < 0 and 0 < M < oo. Finally, given an arbitrary Gaussian process {v(t)}, we have
Cum,{,(t~),
,(t~), ,(t~), ,(t,)}
II.B. A FOURTH
-- O.
CUMULANT
(II-14)
CRITERION
Here we propose a novel identification criterion for estimation of 8 given the input-output record. A very widely used criterion for system identification is the mean-square error (or, least-squares) where the error may be the output error, or input error, or generali~.ed error [19]. We propose to use a function of the fourth cumulant of the error as our performance measure. Since the third and higher order cumulants of Gaussian random variables and processes are identically ~.ero, our proposed criterion has the useful property of being insensitive to (i.e., unaffected by) the Gaussian noise processes (at least asymptotically) whether at the system input or the system output. Define a "predictor" !)(t; 8) of y(t) given the input-output record {z(i),y(i), l <_ i <_ t} as 9(t; e) - - - ~
a,(e)y(t - i) +
i--1
b~(e)=(t - i)
(II-15)
i=1
where the notation hi(8) denotes the parameter ai parametrized by 8. Now define the error signal as
((t; e)
.-
y(t) - 9(t; e) ne
7~b
i:1
i=1
y(t) + E a~(O)y(t- i ) - E bi(O)z(t- i).
(II-16)
STOCHASTIC SYSTEM IDENTIFICATION
49
Further define the criterion N
.- N
o) - 3
t=l
N
[N
t=l
o) ]=.
(Ii- 7)
Let Oc C | be any compact set. We propose the following optimization criteria for selection of 8
min
0~v " - a r g { 0 G d e
X/I
(I1-18)
In (II-18) we are proposing to minimize the square root of the magnitude of the sample fourth cumulant of the error signal. [In fact, any monotone function of IJ4N(8)l will yield the same global extremum point.] Since the fourth cumulants of Gaussian processes are identically zero, it dearly is unaffected by the input-output noises, at least for large samples. We show in Sec. II.C that the above criterion yields a strongly consistent parameter estimator under certain (mild) sufficient conditions. R e m a r k 2. We show later that IJ N(e)I converges w.p.1 to I ~ 1 E % _ ~ g (i; e) where the sequence g(i; 8) is such that g(i; 80) 0 (see Lemma 1). This implies that in the vicinity of the desired solution, all the derivatives up to the third order of IJ4N(O)I w.r.t, g(i; O) are zero for large N; in turn, this implies that the derivatives up to the third order w.r.t. 8 are also zero. Therefore, the function IJ4N(O)l is relatively insensitive to small perturbations in the parameters in the vicinity of the true solution. This fact has two consequences. First, it is likely to translate into a relatively high variance parameter estimate (poor asymptotic efficiency). Second, since the Hessian of ]J4N(8)I w.r.t. O is asymptotically a null matrix, any optimization based on a Newton-Raphson method will be ill-conditioned (for large N), or will reduce to a gradient descent method if some sort of regutarization is issued. As a consequence the convergence of the numerical method of estimate computation will be quite slow. It is not clear how to tackle the first drawback in the framework of the proposed method. To remedy the second drawback, we propose the criterion function (II18). It is not too difficult to verify that the second order derivatives of X/IJ4N(O)I w.r.t, g(i; O)in the vicinity of the desired solution now yield a positive-definite diagonal Hessian, for large N. Therefore, the numerical optimization aspect now becomes well-conditioned. [] R e m a r k 3. We note that a weighted fourth cumulant (moment) criterion has been discussed and thoroughly analyzed for time series problems (input not measured) by Donoho [48]. Also, for FIR system identification problems, the sample fourth moment of the error signal has been proposed as a cost function by Walach and Widrow [49]. []
50
JITENDRAK. TUGNAIT
As discussed in [19] there are several ways to define the error signal for system identification purposes. We have chosen what [19] calls the generalized error, the other possible error signals being input error and output error. Our technique is equally applicable to these other two error signals. The main advantage of using the generalized error is that it is obtained by FIR (finite impulse response) filtering of input-output records, hence it does not pose any stability problems. On the other hand, the other two error signals are obtained, in general, by IIR (infinite impulse response) filtering of either input or output records. In [19] the generalized error is preferred because it leads to a linear parameter estimator. In our case however there is no such luck: the proposed parameter estimator is nonlinear. The criterion (II-18) is nonlinear in 8. Therefore, we have to resort to iterative optimization by use of some sort of gradient-type method. We can use the Newton-Raphson method for parameter optimization [7, Section 7.6], [15]. The method requires computation of the gradients and the Hessian of the cost function. In the current problem analytical expressions for the derivatives of the criterion turn out to be quite straightforward. The details are omitted and the reader is referred to [39]. II.C. ASYMPTOTIC
PROPERTIES
In this section we investigate the convergence points of the proposed parameter estimators as the data record length tend to infinity (N --, oo). A main result is T h e o r e m 1. Consider the parameter estimator defined by (II-18). Assume that the assumptions (H1)-(H4) hold for 0o - 0o(n~o, nbo). Then as N ---, oo, 0N converges to a set Do w.p.1 if 80 6 @c where
Do-{O[
B(q-1; 0o) B(q-1;O) A(q-1;O) = 0o) } "
R e m a r k 4. In the above the convergence to the set Do is to be interpreted in the sense of Ljung [20],[46]. That is, lim N--.oo
inf 06D0
IION-0[]
~.p.1 0
[]
R e m a r k 5. Implicit in the assumption 80 6 | is the assumption that na >_ nao and nb >_nbO. [] The following theorem follows from Theorem 1 and some standard results (see, e.g., [7, Section 6.3]). The additional condition imposed in Theorem 2 ensures that the set Do consists of a single element 8o.
STOCHASTIC SYSTEM IDENTIFICATION
51
T h e o r e m 2. Consider the parameter estimator defined by (II-18). Assume that the assumptions (H1)-(H4) hold for 00 - 00(n=0, nbo). Then limN--.oo 0N -- 00 w.p.1 if m i n ( n ~ - n=0, nb -- rib0) -- 0 and 0o E @o. 9 II.D. MIMO MODELS We now turn to MIMO (multiple input multiple output) models. Although the discussion will be couched in terms of a vector difference equation ( full polynomial form of matrix fraction description [7, Chapter 6], [47]), the basic results carry over to other canonical (or pseudo-canonical) forms[37] too. We caution the reader that we use the same notation as in the previous sections even though now we have vector processes. It is hoped that this does not cause undue confusion. II.D.1. Model Assumptions The "true" processes, m-vector input u(~)and p-vector output s(~) related via the full polynomial form model (ILl) where
are
nn.
A(q -1) "- I + Z
Aiq-i
p x p matrix polynomial
(11-19)
i=1 nb
B(q -1) "- ~
Biq -~
p x m matrix polynomial
(II-20)
i--1
In general, all the elements of the matrix coefficients {Ai, Bj} are treated as unknowns. It is assumed that the input u(~) is generated by driving an asymptotically stable m x n linear filter H(q -1) by another n-vector process
u ( t ) - H ( q - 1 ) e ( t ) - IV(q-l)] - 1 C ( q - 1 ) e ( t ) ,
(II-21)
where C(q -1) and D(q -1) are finite-dimensional matrix polynomials. We measure noise-contaminated input and output as in (II-7) and (II-8). The following conditions on the above model are assumed to hold. det A(z) :/: 0 for ]z[_ 1 where z is a complex variable. Moreover, A(z) and B(z) are left coprime (they have no common factor).
(H1M)
( H 2 M ) The process {e(t)} is a zero-mean, i.i.d, non-Gaussian sequence with "r4,i "- E{e~(t)} - 3[Z{e~(t)}] 2 r 0 and E{e~(t)} < oo where ei(r denotes the i-th component of e(t). Moreover, ei(r is independent of ej(t) for i :/= j. Furthermore, either ")'4,~ > 0 for every l<_i_nor-),4ei<0forevery l_i
52
J I T E N D R A K. T U G N A I T
(HAM) The noise processes {vi(t)} and {v0(t)} are stationary, zero mean, jointly Gaussian and independent of {e(t)} such that their variancescovariances decay exponentially as in (H3). ( H 4 M ) det D(z) # 0 for Izl < 1 so that the filter H(q -1) is asymptotically stable. Moreover, we assume that H(q -1) ~ O. R e m a r k 6. (H2M) is much stronger than the SISO counterpart (H2). Not only do we require {e(t)} to be non-Gaussian but we also demand that its components must all have either 74ei > 0 or "Y4~i < 0. It is not yet clear how to relax this condition. Furthermore, (H2M) imposes both "spatial" as well as temporal independence condition on {e(t)}. It is not clear if any arbitrary i.i.d, vector sequence {e(t)} can be decomposed into {F~(t)} where {d(t)} is i.i.d, with all its components mutually independent and the matrix F is such that {e(t)} and {F~(t)} have identical second and fourth order cumulants. Existence of such a matrix F is an open problem. [] Let Ai(i, j) denote the ij-th element of the matrix Ai, and similarly for Bi. Define the vector of the unknown parameters in the system model as
8(ha, rib) "-- ( A k ( i , j ) , B r ( s , t ) , 1 < i,j,s
1 < k < ha, 1 < r < rib,
1
(II-22)
Let 80, n~0 and rib0 denote the true values of the respective vector and/or parameters. Also define the following set O "- {8[ (H1M) holds }. II.D.2. F o u r t h C u m u l a n t C r i t e r i o n Mimicking the developments in Section II.B, define a "predictor" ~(t; 6) of y(t) given the input-output record {z(i), y(i), 1 < i < t} as n6
~(t; 8 ) " - - E
~b
Ai(8)y(t - i) + E
i--1
Bi(8)z(t - i),
(II-23)
i--1
and the weighted error signal as
~(t; e)
--
h [y(t) - #(t; e)]
=
A[y(~) + ~
71.,
nb
A,(e)y(t
i:1
- i) - ~
B,(e)~(~
-
O]
(II-24)
i:1
where A is an arbitrary constant nonsingular matrix. Finally, with ei(t; 8) denoting the i-th element of the p-vector e(t; 8), we define the (scalar) criterion p
J4N(8) "-- E i=l
N
[N-1E t=l
N
ei (t; e) - 3 [ n -~ E t=l
ei~(t; e)
]~ ].
(II-25)
STOCHASTIC SYSTEM IDENTIFICATION
53
Let Oc C O be any compact set. We propose the following optimization criterion for selection of 8 min V/[j4N(8) [}. 8N "-- arg{ 0 E Oc
(II-26)
II.D.3. Consistency The counterpart to Theorem 1 is T h e o r e m 3. Consider the parameter estimator defined by (II-26). Suppose that (H1M)-(H4M) hold for 8o - 8(n,~o, nbo). If 80 e Oc, then as N ---, c~, 8N converges to a set Do w.p.1 where Do - { 0 [ A-l(q-1; O)B(q-1; O) - A-l(q-1; Oo)B(q-X; 0o) } * The proof of the above result is along the lines of the proof of Theorem
POLYSPECTRAL
III. MODEL ASSUMPTIONS SPECTRA
TECHNIQUES
AND CUMULANT
Let u(t) and s(t) denote the "true" input and output, respectively, at (discrete) time t. The two are assumed to be related via a finite-dimensional ARMA(n~, nb) model
A(q- ~)~(t) - B(q- ~)~,(t)
(III-1)
where n~
A(q -1) "- 1 + E a,q-'
(III-2)
i=l
B(q- 1) ._ ~ b,q-i
(III-3)
i=1
and q-1 is the backward shift operator, i.e., q-ira(t) - m ( t - 1), etc. As in [1] and [T], for example, we have assumed for convenience only that no
54
JITENDRAK. TUGNAIT
term bou(t) is present, i.e., that there is at least one delay in the system. Explicitly, we may write (III-1) as 'n,~.
'/'1 b
s(t) + E a , s ( t - i) - E biu(t- i). i=1
(III-4)
i=1
The processes {u(t)} and {s(t)} are not available for measurement. But we can measure noise-contaminated input and output
z(t) - u(t) + v,(t)
(111-5)
y(t) -- s(t) + vo(t)
(111-6)
The model (111-1) is assumed to be exponentially stable and minimal, i.e., the following condition is assumed to be true. (HIP)
A(z) # 0 for [z[ < 1 where z is a complex variable. Moreover, A(z) B(z) are coprime (they have no common factor).
and
It is also assumed that all of the processes involved (i.e., z(t), y(t), v,(t), and vo(t)) are zero-mean and jointly stationary. Furthermore, the noise sequences {v,(t)} and {vo(t)} are independent of {u(t)}, hence of {s(t)}. Define the vector of the unknown parameters in the system model as 8(n~, nb) "-- (hi, a 2 , - ' - , a,~., bl, b2,'' ", b,~).
(111-7)
Let 8o, n~o and nbo denote the true values of the respective vector and/or parameters. Also define the following set
0 "- {8] A(z; 8) -# 0 for Izl <_ 1}, where A(z; 8) denotes the polynomial (III-2) parametrized by 8. The objective is to estimate the transfer function B(q-1)/A(q-1)(equivalently, 8) given a data record {z(i), y(i), 1 <_i <_ N}. Consider the triple correlation function (or the third-order cumulant function) C=~y(i, k) defined as
C~y(i, k) "- E{x(t + i)z(t + k)y(t)}.
(III-8)
Denote the cross-bispectrum of input/output (two-dimensional discrete Fourier transform of C==~(i,k)) by B=~y(wl,w2), i.e., oo
B~u(wl,w2) -- E i=-oo
oo
E k=-oo
C~v(i, k)exp{-j(wli + w2k)}.
(III-9)
STOCHASTIC SYSTEM IDENTIFICATION
55
Similarly, let B===(Wl,W2) Bs,,s(Wl,W2) denote the bispectra of the processes {a:(t)} and {s(t)}, respectively. From the above definitions it is easy to see that k)
-
lff
(2~r)2
~ Bsss(wl, w2)exp{j(wli + w2k)} dwl dw2.
w
(III-lO)
Define
w(t) "-- s2(t)-E{s2(t)} and @ ( t ) " - s2(t).
(III-11)
Then both the cross-spectrum between the process {@(t)} and {s(t)} and the cross-spectrum between the process {w(t)} and {s(t)} are given by OO
S~,(w) "-
E{w(t)s(t + k)}exp{-jwk}
E / c - - - oo 00
(III-12) k=-oo
It then follows that (III-13) W
Compare (III-lO) with (III-13) to deduce that
"/r
if
-
(111-14)
W
Notice that the cross-spectrum between the signal s(t) and its square can be interpreted as an integrated bispectrum of s(t). This integrated bispectrum will form a basis (along with the integrated trispectrum, to be defined later) for unknown parameter estimation. It is easy to see that since bispectrum of a Gaussian process is identically zero, so is its integrated bispectrum. Therefore, the integrated bispectrum of {z(t)} equals the integrated bispectrum of {s(t)}. In the sequel it will be easier to work with the centered (zero-mean) s2(t), i.e., w(t).
56
JITENDRA K. TUGNAIT
Turning to the trispectrum, it is defined as [23],[24] Tssss(Wl~ W2~ w3) OO
OO
"--
OO
E E E
i = - c o k = - o o 1=-oo
Cssss(i, k, l)exp{--j(Wli +
w2k +
w31)}
(111-15)
where
c,,,,(i, k, z)
.
E{,(t),(t + i),(t + A,),(t + 0}-
-
E(,(t),(t + ~)}E(,(t + l),(t + i ) } E(,(t),(t + i)}E{,(t + k),(t + Z)}E(,(t),(t + Z)}E(,(t + k),(t + i)}
(111-16)
is the fourth-order cumulant function of the process {s(t)}. From the above definition it is easy to see that
o.,..(~, k, 0 -
l fff
Tssss(Wl,W2,w3)~(wxi+w"k+w"l)dwld w 2 d w 3 . (111-17)
g(t) "-- s3(t)-
3s(t)E{s2(t)}and r(t)
Define
-- g ( t ) - E{g(t)}.
(111-18)
Then both the ~o~-~pr br162162the p~or162{F(t)} and {,,(~)} and the cross-spectrum between the process {r(t)} and {s(t)} are given by OO
E{,-(0,(~ + ~,)}~p{-j,,,k}
s , , (,,,) - k=-oo iX)
o....(0, 0, k ) ~ p { - j ~ k }
- s,.(~).
(111-19)
k=-oo
It then follows that
c,,,,(0, 0, k) -
1/:
~
w
s ~ , ( ~ ) ~ p g ~ k } d~.
(111-20)
STOCHASTIC SYSTEM IDENTIFICATION
Compare
(III-17)with (III-20) to
57
deduce that
1F; T,,,, (~, ~ , ~ ) d ~ d ~ = S,, (~).
(III-21)
Notice that the cross-spectrum between the signal s(t) and a function of its cube can be interpreted as an integrated trispectrum of s(t). Given the above definitions we are now ready to state the remaining model assumptions. A s s u m p t i o n Set I. We assume that the bispectra of the noise processes are zero and that the noise processes are statistically independent of the process {u(t)}, hence of {s(t)}. It is also assumed that B,,,~,~(Wl,w2) ~ 0 and that (H1P) holds. Assume that all moments of the various processes involved, viz., s(t), u(t) etc., exist. Also assume that the third and lower order cumulant/cross-cumulant sequences of the various processes involved, viz., Cs,,=(rl, v2) etc., satisfy the following summability conditions OO
[1 +
I~ I] Io,~.~...~ (~,
9" , ~ - ~ ) 1
< oo,
"F1 ~--. ~ - r k _ 1 - - - - - O0
for j = 1, 2 , . . . , k - 1 and k = 2 and 3 where
~,(t) e {s(t), ~(t), ~(t), yCt), v~(t), Vo(O}. The above summability conditions are sufficient conditions for the corresponding bispectrum to exist [24],[42],[43]. They are also sufficient for the asymptotic results concerning the bispectrum estimator discussed in Sec. IV.A.1 and concerning the integrated bispectrum discussed in Sec. V.A.1 to hold. [] A s s u m p t i o n Set II. In addition to those stated in Assumption Set I, assume that {u(t)} is such that at least one of the following conditions hold true: ( A S I ) E{u3(t)} =: 73,, r 0. ( A S 2 ) {u(t)} is a finite-dimensional, linear non-Gaussian process generated by driving a bounded-input bounded-output linear filter by a zero-mean, independent and identically distributed (i.i.d.) sequence {e(t)} (with E{eS(t)} # - 0 ) : OO
u(t)
--
~ i-----oo
g(i)e(t- i),
(III-22)
58
JITENDRA K. TUGNAIT
A s s u m p t i o n Set III. We assume that the trispectra of the noise processes are zero and that the noise processes are statistically independent of the process {u(t)}, hence of {s(t)}. It is also assumed that T,~,,,,,.,(Wl, w2, w3) 0. It is assumed that (H1P) holds and that all moments of the various processes involved, viz., s(t), u(t) etc., exist. Also assume that the fourth and lower order cumulant/cross-cumulant sequences of the various processes involved, viz., Cs~=y(rl, T2, r3) etc., satisfy the following summability conditions OO
E T 1 ~""" I T h - - 1 =
[1 + Irj[] [C,,z,...z~(rl,'' ", rk-1)l < oo, --
O0
for j = I, 2,..., k - 1 and k = 2, 3 and 4 where e
y(t),
The above summability conditions are sufficient conditions for the corresponding trispectrum (Or cross-trispectrum) to exist [24],[42],[43]. They are also sufficient for the asymptotic results concerning the integrated trispectrum discussed in Sec. V.B.1 to hold. Assume also that {u(t)} is such that at least one of the following conditions hold true: ( A S h ) 74= := E{u4(t)} - 3[E{u2(t)}] 2 # 0. ( A S 4 ) {u(t)} is a finite-dimensional, linear non-Gaussian process as in (AS2) except that now 0'4~ # 0. [] Suppose that the assumption (AS1) is true. Then we can not have S,~,~(w) - 0 because then 73u = 0 contrary to the assumption. Suppose that the assumption (AS2) is true. Then using (III-22) it can be shown that[23],[24] (111-23)
Su2u(w) = 7 a , G 2 ( - w ) G ( w ) , where
= oo
G(w)- ~ g(k)exp{-./,,,k}
(111-24)
k=-oo
and IX)
G2(w)- ~ g2(k)exp{-jwk}. It then follows that unless G(w)
(111-25)
- O, we have S~2~(w) ~ O. Thus, under
either (AS1) or (AS2) (or both), the integrated bispectrum of u(t) is nonvanishing. Turning to the integrated trispectrum, it easily follows that under
STOCHASTICSYSTEMIDENTIFICATION
59
the assumption (AS3) (i.e., ")'4u ~= 0), we must have S~u(w) ~ 0 (where r,,(t) = u3(t) - 3u(t)E{u2(t)} ), else we have a contradiction. Under (AS4), 0'4~, = 74, E L g4(k) ~: 0 unless a(w) = 0 or "r4, = 0.
IV. B I S P E C T R A L
APPROACHES
In this section two new parametric frequency domain approaches are proposed for estimation of the parameters of linear errors-in-variables models. One of the proposed approaches is a linear estimator where using the bispectrum of the input and the cross-bispectrum of the input-output, the system transfer function is first estimated at a number of frequencies exceeding one-half the number of unknown parameters. The estimated transfer function is used to estimate the unknown parameters using an overdetermined linear system of equations. In the second approach a quadratic transfer function matching criterion is optimized by using the linear estimator as an initial guess. The estimators are later analyzed in Sec. VI under Assumption Set I. IV.A. LINEAR
ESTIMATOR
It follows from (III-4)-(III-6), (III-8) and (III-9) that
Bx~y(Wl,W2)- H*(eJ(~'+~2))B~x~(wl,w2), where
and H* is the complex conjugate of H. Therefore, if Bx~x (wl, w2) -Y: 0 then H,(ed(O~+~) ) _
Bxxv(wl,w2)
(IV-l)
The above relation occurs in [5] and [41]. Our new linear approach is to first rewrite (IV-l) as
H(eJ~) _ S;x~(wl, w - wl)
(IV-2)
Thus, for a given w several estimates of H(e j~) can be obtained from (IV-2) by using different values of wl.
60
J I T E N D R A K. T U G N A I T
IV.A.1. E s t i m a t i o n of B i s p e c t r u m a n d C r o s s - B i s p e c t r u m : We now discuss estimation of the bispectra given the data {$(t), y(t), 1 < t < N} and some large sample properties of the estimators. The desired bispectra can be computed by smoothing the sample bicovariance [21], by smoothing the sample bispectrum in the bifrequency domain [25], or by dividing the sample into segments and averaging the segmentwise sample bispectrum and then smoothing in the bifrequency domain [22],[30]. In this paper we follow the approach of [22],[30]. Suppose that the given sample sequence of length N is divided into K nonoverlapping segments each of size LB samples so that N - K LB. Let X(~)(w) denote the discrete Fourier transform (DFT) of the i-th block { ~ ( t + ( i - - 1 ) L B ) , l <_t <_Ls} ( i - l , 2 , . . . , K ) given by Ls-1
xCi)(wk) --
z(t + 1 + ( i - llLs)exp(--jwkt)
E
(IV-3)
t=0
where wk -
2~"
~--~Bk,
k-0,1,.--,LB-
1.
(IV-4)
Similarly define Y(i)(w~). Then the biperiodogram for the i-th block of data is given by
x(')
)x(')
+
(IV-5)
where X* denotes the complex conjugate of X. The cross-biperiodogram for the i-th block of data is similarly given by
(k, z) --
1 X(i)(w})X({)(w,)[y(i)(wk + w,)]* L--;
(IV-6)
Finally the bispectrum estimate/}~,(m, n) is given by averaging over K blocks and spatially smoothing over a rectangular window of size M x M where M - 2/17/+ 1 is an odd integer: M
13~(rn, n)-- M -2 E
E
r = -
lVI 8 = -
K
1 [KE/}(~i2"(m+r'n+s)]'
IVI
(IV-7)
i = l
and similarly we have /3..~ (rn, n) -- M -2
E
E
r = - A~r a = - / ~ r
I
[gE/~(:2~ (rn+r'n+s)]; i=1
(IV-8)
STOCHASTICSYSTEMIDENTIFICATION
61
details may be found in [22],[25],[30] and references therein. The principal domain of ~) is the triangular grid 10
0
2
+I
(IV-9)
The principal domain of ~.~yfi(i)(k, l) is a larger triangular grid [41] -
-
2'
2
(iv- o)
In (IV-7) and (IV-8) all the pairs (m -4- r, n-4- s) are kept strictly inside the principal d o m a i n s / ) ~ and /)~,u, respectively. This is accomplished by appropriate selection of the pairs (m, n). Such (m, n)'s are given by
Dx~.~ - {(m, n) [ m l
M k . - M, n LB ~-~-l,
M l - .~I,
(m,n) E D ~ }
(IV-11)
for the auto-bispectrum and by D~y - D(.~) U D(2)
(IV-12)
for the cross-bispectrum, where D z(t) - {(m,n) l m - M k - . ~ / I , zy
LB
n-Ml-M, e
and D(~)y - {(m, n)[ ( m , - n ) 6 D(1)u }. Remark. Later we will let N and I.,s become large and we will then discuss and exploit some large sample properties of the bispectrum and cross-bispectrum estimates. As N and LB become large, m and n do not remain fixed; what remains fixed is the bifrequency (27rm/LB, 27rn/LB) -(win, wr,). Hereinafter, all convergences and large sample values are assumed to involve such fixed bifrequencies even though explicitly we may still use the notation B~z~:(m, n), etc. [] The large sample properties of Bxz~(m, n) for (m, n) 6 D ~ and of /3~xu(m, n) for (m, n) 6 D z ~ have been discussed in [22], [25], [30] and [41] following the asymptotic theory presented in [24, Chapter VI], [42, Chapter 4] and [43]. Define A N
:-=
M LB
MK
___= ~ . N
62
JITENDRAK. TUGNAIT
The quantity AN is a measure of the "bandwidth" of the bispectrum estimate. Under the cumulant summability condition stated in Assumption Set I, it follows from [24, Chapter VI], [42, Chapter 4] and [43] that E{/~==(m,n)} -
B===(m,n) + O(AN),
1 var{Re{/3xxz(rn, n))} - 2 ~--~"~TvSx(m)Sx(n)Sz(m + n) + O( = var{Im{/}x== (m, n)}},
(IV-13) 1 NAN (IV-14)
where S~(rn) denotes the power spectral density of {z(t)} at frequency w,~ = 21tin~LB. Note that NA~r -- KM2/LB. Therefore, as N + oo such that AN ---, 0 and NA~r -+ oo, we obtain an unbiased and mean-square consistent estimator. Moreover, under the above asymptotic conditions, the estimate/~===(m, n) (m, n # 0), is complex Gaussian, and independent of [ ~ ( m ' , n ' ) (m',n' # 0 ) i f m' # m and/or n' ~- n, such that both (m, n) and (m', n') are in the interior of D===. Similar results holds for cross-bispectrum estimates leading to E{/3==y(rn, n)} -
B==u(m, n) + O(AN),
1 var{Re{/3==y(m, n)}} - 2NA~ v S=(m)S,(n)Sy(m + n) + O( = var{Im{/3==~ (rn, n)}}.
(IV-15) 1
NAN (IV-16)
The estimate/3,,~ (rn, n) (m, n :/= 0), is complex Gaussian, and independent of/3,,~ (m', n') (m', n' r 0) if m' r m and/or n' ~= n, such that both (m, n) and (m', n') are in the interior of D,,~. Selection of AN (i.e., of K, M and LB) for a given N is usually a compromise between the bias and the variance of the bispectrum estimator. The bias of .B,,,(m, n) is O(AN) whereas its variance is inversely proportional to NAUt. In order for the bispectrum estimator to be consistent, one must have limN--.oo AN -- 0 as well as limN--.oo NAVy -- oo. One way to accomplish this is to choose AN -- N r176 with 0 < c < 0.5. In terms of K, M and LB, one may pick K = 1, LB = N and M = N r176 with 0 < c < 0.5. Alternatively, one may choose M = 1, LB = N ~ and K = N ~162 with 0 < c < 0.5. The smallest bias results with c --, 0 whereas the smallest variance occurs for c---, 0.5 .
STOCHASTIC
SYSTEM IDENTIFICATION
63
IV.A.2. O v e r d e t e r m i n e d Linear S y s t e m of E q u a t i o n s : Returning to (IV-2) and using the estimates of the bispectrum and cross-bispectrum, we have
H(~;~)
--
B^* x~(rn,
for (m, n - m) 6 D**y
m)
n -
andn-lM-_M,
1<1< -
-
LB
2M
L e t / t B ( e J ~ ) denote the least-squares estimate of H(e jW~) given Ar(w,~;w,~) for a fixed n with m ranging over (m, n - m) E D ~ y . Then we have
_ E(~,._~)~...
~(~.)
&~(m,.
- r n ) B^* xx~(m, n-
m)
(IV-17)
E ( ~ , ~ _ ~ ) ~ . . I & ~ (m, ~ - m)I ~ for n - l M - M, 1 < l <_ ( L s / ( 2 M ) ) - 1. From the definition of H(e j~) we have - E
aiH(eJ'~")e-J'~"i + E
i--1
bie-J'z"i = H(eJ'Z'~)
(IV-18)
i=1
for any w,~. Noting that ai's and bi's are real and H(e j'') is, in general, complex-valued, we rewrite (IV-18)after replacing H(eJ"") with its leastsquare estimate ~rB(eJ~), as ?'ta.
~rtb
- ~ ~a~{~(~ ~ 1 ~ - ~ } i=1
+ ~ b,R~{~- j ~ ) i=1
= R~{~rB(~J~-)}
(w-19)
and ~a,
- E i----1
~b
ailm'[Hs(eJ~")e-Jw'~i} + E
bilm{e-J~"i)
i=1
= Im{HB (eJ~")}.
(IV-20)
We solve (IV-19) and (IV-20) for ai's and bi's (via singular value decomposition techniques, e.g.) with n - l M - IVI, 1 < < 2Ls m 1m M _ 1.
64
JITENDRA K. TUGNAIT
IV.B. NONLINEAR
ESTIMATOR
We follow a quadratic transfer function matching approach. Let H(efW[8) denote the transfer function of (III-4) with the system parameters specified by the parameter vector 8 as defined in (III-7). Define (IV-21)
_
B:..(m,n)
The following result is from [24],[431. (Recall also the Remark of Sec. IV.A.1). A proof of L e m m a 1 also follows from the mean-square convergence implied by (IV-13)-(IV-16). L e m m a 1. Under the Assumption Set I, we have limN--.oo/~,,, (m, n) limN~/3,,y(m,n) Lemma
2.
B , . , (m, n) i.p. (in probability),
-
B . . ~ ( m , n ) i.p.
*
[36],[7]
(i) Let {a:lv} and {yN} be two sequences of random variables such that a=lv --4 a: i.p. and yN --4 y i.p. as N ~ oo. Then ~ r + yN ~ a: 4-y i.p. and zNYN ~ zy i.p. (ii) Let f be a continuous function on the real line. If a:N --* 9 i.p. then N
-*
oo.
(iii) Let {XN} be a sequence of random vectors that converges in distribution to Af(0, P), a Gaussian distribution function with zero mean and covariance matrix P. Let {AN} be a sequence of random square matrices that converges i.p. to a nonsingular matrix A. Define YN = AN XN. Then YN converges in distribution to A/'(0, APAT).
Proof: Part (i) of L e m m a 2 is Exercise 2 in [36, Sec. 4.1, p. 66] and part (ii) of L e m m a 2 is Exercise 10 in [36, Sec. 4.1, p. 66]. Part (iii)of L e m m a 2 is [7, Corollary to L e m m a B.4, p. 551]. n L e m m a 3. Under the Assumption Set I as N --* oo, the following results are true for any fixed (w,~, w,~) and (w,~,, w,~,) in the interior of the set D ~ . (A) A N V ~ ( / ~ ( e J ( ~ , ~ + ~ = ) ) - H(eJ(~,~+~=)[8o))converges in distribution where to the complex normal distribution Aft(0, a,~,~) 2 =
'~'~
+
[ B , . , ( m , n)12
9
S T O C H A S T I C S Y S T E M IDENTIFICATION
65
(B) H(eJ('~'+~")) and ~(eJC~..,+~.,)) are statistically independent for
(.,,,~) # (m',,r
9
Proof." It follows from the asymptotic results (IV-11)-(IV-16) discussed in Sec. IV.I.1, Lemma 1 and Lemma 2. [3 Lemma 3 motivates the following cost criterion which is the negative log-likelihood (up to some constants which do not depend upon 0) of the asymptotically complex Gaussian vector {~r(eJ(~-'+'~)), (re, n ) C D ~ 0 is the interior of D~zu. Choose 0 to minimize the cost where D~u
Z
~;.,(m,,,) B...(m,n)
2
_ H (d (,,,,,,.+,.,,,,.) iO)
2 /,:,,,,,,.
(IV-22)
In the above we have assumed that cr~,~ 3/: 0. If such is not the case then we delete that particular value of (m, n) from the summation. In practice, S~(m) is unknown; therefore, we replace it by its consistent estimate obtained by block averaging and/or frequency-domain smoothing as in (IV5)-(IV-8). Thus we have the following practical cost criterion
j(d)(o)
2 -
^:r
B..z(m,n)
_ H(d(,~-+,~,,)
10 )
,,2 /crm, ~
(IV-23)
where
,:,-,.,,,,"~ - &(,-,',.)& (-.).-r (,~ + -.)/I.b.,,, (~,,.,.)1 ~.
(IV-24)
Mimicking Lemmas 1 and 2 it is easy to show that crm, ^ 2 ~ tends to cr,~ n2 i.p. as N ---, oo provided S ~ ( m , n) :fi O. Minimization of (IV-23) w.r.t. 0 is a nonlinear optimization problem that is initialized by the linear estimator obtained by solving (IV-19)-(IV-20).
V. I N T E G R A T E D P O L Y S P E C T R A L T E C H N I Q U E S In this section, the frequency domain approaches presented in Sec. IV are discussed using integrated polyspectra. As discussed in Sec. III, integrated polyspectrum (bispectrum or trispectrum) is computed as a crossspectrum; hence, it is computationally cheaper than the corresponding polyspectrum particularly in the case of the fourth-order cumulant spectrum. If the non-Gaussian input to the linear system has a symmetric PDF, its bispectrum and the integrated bispectrum will vanish whereas its trispectrum and the integrated trispectrum will not, provided that the
66
JITENDRA K. TUGNAIT
fourth cumulant of the input 74~ is nonzero. Extension of the approaches of Sec. IV to trispectrum-based approaches is computationally complex and the resulting estimators are likely to have poor statistical performance because of the high variance of the trispectrum estimators [24]. Herein lies the significance of the integrated polyspectrum-based approaches which apply with almost equal computational and programming ease to both cases, those involving integrated bispectrum as well as those concerned with integrated trispectrum. V.A. I N T E G R A T E D
BISPECTRAL
TECHNIQUES
V . A . 1 . Linear E s t i m a t o r
Under the Assumption Set II, we have S=~y(w) = S ~ , ( w )
= D F T{E {z2 ( t ) y ( t + k)}}
where DFT stands for discrete Fourier transform. It follows from (III-4)(III-6) that under the Assumption Set II S=~(w) = H*(eJ")S=~=(w) where H(z) is as in Sec. IV, S=~=(w) = S,~,~(w), and S==u(w) = S~=s(w). Therefore, if S===(w) r 0 then
H*(e j'') -- S = 2 y ( w ) [ S = = , ( w ) ] -~.
(V-!)
E s t i m a t i o n of I n t e g r a t e d A u t o - a n d C r o s s - B i s p e c t r u m We now discuss estimation of the integrated bispectra given the data {re(t), y(t), 1 _< t _< N} and some large sample properties of the estimators. Suppose that the given sample sequence of length N is divided into K nonoverlapping segments each of size LB samples so that N - K L s . Let X(i)(w) be as in (IV-3). Let X~i) (w) denote the discrete Fourier transform (DFT) of the i-th block {m2(t + ( i - 1)Ls), 1 _< t < L s } ( i - 1, 2,-.-, K) given by Ls-1
X~i)(wk) --
E
m2(1 + t + ( i - 1)LB)exp(--jwkt),
(V-2)
t=0
27r wk. -- --~Bk,
k -- O, 1,..., LB -- 1.
(V-3)
In order to set the mean of {m2(t)} in the i-th block to zero, simply set X~~)(0) - 0. Then the cross-periodogram for the i-th block of data is given by
~(i) (k) "-===
1 X~i)(wk)[X(,)(w~)], LB
(V-4)
STOCHASTIC SYSTEM IDENTIFICATION
67
where X* denotes the complex conjugate of X. The cross-periodogram ~u(m) is defined similarly. Finally the estimate S ~ ( m ) i s given by averaging over K blocks and spatially smoothing over a rectangular window of size M where M - 2M + 1 is an odd integer: K
1 r =-/l~r
~(i) (m + r)], i=1
LB
wherem-Ml-l~l,
(V-5)
1 < l < -~
- l,
and similarly we have /~r
K
1 r=-/~r
where m -
MI-
M,
~(') (m + ~)],
(v-6)
i=1
LB
1
-
2M
The large sample properties of ..~=y(m) and s follow from the asymptotic theory presented in [24, Chapter VI], [42, Chapter 4] and [43], just as in Sec. IV.A.1. Under the cumulant summability condition stated in Assumption Set I, it follows from [24, Chapter VII, [42, Chapter 4] and [43] that for large N, the estimate S~y(m) (m ~: 0), is complex Gaussian, and independent of S~2~(n) (n ~= 0) for m # n such that
(v-~) 1
~S~(m)S~(m) = var{Im{;~x2~ (m)}}.
+ O(N -~) (v-s)
In the above, as in Sec. IV.A.1, AN -- M / L B is the bandwidth of the crossspectrum estimates. Moreover, as in Sec. IV.I.1, we must have AN ~ 0 and NAN --4 oo as N ~ oo in order to obtain an unbiased and mean-square consistent estimator. Similar results hold for S ~ ( m ) . O v e r d e t e r m i n e d L i n e a r S y s t e m of
E q u a t i o n s :
Returning to (V-l) we set
s~(m)
(m)]
(v-9)
68
J I T E N D R A K. T U G N A I T
Mimicking Sec. III.A.2, we solve (IV-19)-(IV-20) for ai's and bi's (via singular value decomposition techniques, e.g.) with n - IM-ll/I, 1 -<- I -<- -~2M - 1 and using the above estimate of I-IiB(eJ~"). V.A.2. Nonlinear Estimator We follow a quadratic transfer function H(eJ~[8) terparts of Lemmas 1 and L e m m a 4. Under the
transfer function matching approach. Let the be as in Sec. IV.B. Lemmas 4 and 5 are coun3, respectively. Assumption Set II, we have
-
-
9
L e m m a 5. Under the Assumption Set II as N - . oo, the following results are true for any fixed 0 < win,win, < 7r. (A) v/NAN[[-IzB(e j'''') --H(eJ~'~[80)] converges in distribution to the 2 complex normal distribution Aft(0, a,~) where _
-=
(B) Hzs(eJ~'~) and Hxs(eJ~,~') are statistically independent for m r m'.
Proof: Mimic proof of Lemma 3.
[] As in Sec. IV.B, Lemma 5 motivates the following cost criterion which is the negative log-likelihood (up to some constants which do not depend upon 0) of the asymptotically complex Gaussian vector {Hzs(e j~'~), m E D=2= } where
D=2= "- { m l m -
lM-1VI,
l < l <__( L s / ( 2 M ) ) -
l}.
Choose 8 to minimize the cost 2
^* (m) _ H(e~.,10) reED=2=
/a~.
(V-10)
z2=
2 In the above we have assumed that a,~ ~: 0. If such is not the case then we delete that particular value of m from the summation. In practice, the various power spectral densities are unknown; therefore, we replace them by their consistent estimates obtained by block averaging and/or frequencydomain smoothing as in (V-4)-(V-6). Thus we have the following practical cost criterion
"Nr(zB)(e)-- ~ reED=2=
^,
2
$,S;=~'(m)(m) - //(#""le) /~'~ ~2=
(V-11)
STOCHASTIC SYSTEM IDENTIFICATION
69
where ~^= _
~====(m)~(m)/l~===(m)l
(v-12)
~"
Using Lemmas 2 and 4 it is easy to show that a,~^2 tends to cr,~2 i.p. as N ~ oo provided S,2=(m) =/: O. Minimization of (V-11) w.r.t. O is a nonlinear optimization problem that is initialized by the linear estimator of Sec. V.A.1. V.B. I N T E G R A T E D
TRISPECTRAL
TECHNIQUES
First define r,(t) -
z a ( t ) - 3 z ( t ) E { z 2 ( t ) } - E{$a(t)}.
Replacing z2(Q with r=(t) in Sec. V.1 and using the Assumption Set III yields the integrated trispectrum-based approaches. Replacing z(t) with u(Q in the above equation yields r~(t) which is identical to r(t) in (III-18). V . B . 1 . Linear E s t i m a t o r
Under the Assumption Set III, we have
S,.~(w) - S,~,(w) - DFT{E{r,,(t)y(t + k)}}. It follows from (III-4)-(III-6) that s,.~(~) where
-
H*(e~)S,.,(~)
H(z) is as in Sec. IV, S,.,(w) - S,,,u(w), and S,.~(w) - S,,,,(w).
Therefore, if S,.=(w) r 0 then H * ( g ~) -
s,.~(~)[s,.,(~)] -1.
(v-~3)
E s t i m a t i o n of I n t e g r a t e d A u t o - and C r o s s - T r l s p e c t r u m
We now discuss estimation of the integrated trispectra given the data {z(t), y(Q, 1 _ t < N} and some large sample properties of the estimators. Let X' (w~) be as in (IV-3). Let R(~)(w~) LB--1
E t=O
-
[z3(t+l+(i-1)LB)-3z(t+l+(i-1)Ls)f~2"
]exp(-jwkt),(V-14)
70
JITENDRA K. TUGNAIT
where N
1 /~2, -- ~ E
a~2(t)"
(V-15)
t--1
In order to set the mean of { a 3 ( t ) - 3a(t)E{a2(t)}} in the i-th block to zero, simply set R(~)(0) - 0. Then the estimate S~.,N(rn) is given by -
M-'
{ ~1 / ~ 1 [ ~ 1 RC')(w,~+, ) [X (i, (wm+,)]* ] }.
i l=-/~r
(V-16 )
"-
estimate --r is defined similarly. As in Sec. V.A, for large N (such that both LB and K become large) we have The
E{i.:N(m)}
-
S,.~(m)
+
O(AN),
(V-17)
and 1 ~S,.r.(rn)S~(rn) 2NAN = var{Im{..r
+ O ( N -1)
(m)}}.
(V-18)
Similar results hold for K.. (m)}. O v e r d e t e r m i n e d L i n e a r S y s t e m of E q u a t i o n s : Returning to (V-13) we set S~. ~(m)[S;=,(rn)]
(V-19)
.
Mimicking Sec. IV.A.2, we solve (IV-19)-(IV-20) for ai's and bi's (via singular value decomposition techniques, e.g.) with n - l M - M 1 < l < ~2M - 1 and using the above estimate o f t-IiT(eJw'*). '
- -
- -
V.B.2. N o n l i n e a r E s t i m a t o r We follow a quadratic transfer function matching approach as in Sec. V.A.2. Let H(eJ~lS) be as in Sec. IV.B. Lemmas 6 and 7 are counterparts of Lemmas 4 and 5, respectively. L e m m a 6. Under the Assumption Set III, we have limN~ooL.,(m) -
S,.,(m)i.p.,
limN-,oo Sr,,y (m) -- Sr,,y (m) i.p.
9
L e m m a 7. Under the Assumption Set III as N --. oo, the following results are true for any fixed 0 < w,,, w,~, < It.
STOCHASTIC SYSTEM IDENTIFICATION
/ 1
(A) ~/N/XN[~zT(eJ~ ) --H(d~le0)]
converges in distribution to the 2 complex normal distribution A/'c(O, o'm) where
o-,,,,~ -
,s',..,.. (,,,,)s'~,,, (,,,.)is',.. = (,,,,)1-2.
(B) HzT(e j~'~) and HIT(e d~,~' ) are statistically independent for m -fi m'. @
Proof: Mimic proof of L e m m a 3.
[] As in Secs. IV.B and V.A.2, L e m m a 7 motivates the following practical cost criterion 2
7(ZT) m 6 D . ~ ,~
- H(#~-~ 10)
&.,(-,)
/~i
(v-20)
where
o-,,, "~ -
.~,.:,.:(r,,,)D,,,,(m)/ID,.::
(m)l ~.
(v-21)
As before it is easy to show that cr,~ ^ 2 tends to ~rm2 i.p. as N --~ oo provided S~.=(m) =fi 0. Minimization of (V-20) w.r.t. 0 is a nonlinear optimization problem that is initialized by the linear estimator of Sec. V.B.1.
VI. CONSISTENCY In this section we discuss the (weak) consistency of the proposed approaches (as N ~ c~). Let H(ed~;80) denote the transfer function (IV-l) of the true model (III-4) with n= - n=o, n - n~o and 0 - 0o. Consider the system of equations (IV-18) rewritten as follows after replacing H(e j~) with H(eJ~;80) (1 _<1 _ L)"
- ~ i=l
a,H(eJW'; Oo)e-jw'' + ~
b,e -j~'' = H(eJW'; 00).
(VI-1)
i=l
Note that in (VI-1) wz represents an arbitrary frequency, not necessarily equal to 27rl/LB. The following result is central to the consistency results. L e m m a 8. Given the transfer function H(w; 80) of model (III-4) at frequencies 0 < w 1 < o32 < " ' " < 03 L < 71" such that n : + n b <__ 2L and min(n= - n~o, n b - nbo) >_ O. Then the set of solutions to (VI-1) is such that H(eJ~; O) - H(eJ~; 0o). *
72
J I T E N D R A K. T U G N A I T
Proof: If (VI-1) is satisfied for some 0 (defined in (111-7)), then H(eJ~'; O) = H(eJo';Oo) for 1 _< I _< L. Define 'rT, a, 0
ACz; Oo)B(z; O) - [ E ai(O~
A(z; 0, 0o) -
T~,b
[ E bi(O)z-i]
i=0
-
~
i=1
(VI-2)
~(0,0o)~-',
i=1
where
ao(Oo)"-- 1 so that 0.1(0, 0 o ) - bl(O). Also define lq~a
~(~;o.Oo1
-
A(~; 0)B(~; 0o1 -
[~ ~,(01.-'1[~ i=O
rt~+ttbo
--
E
[~,(0,Oo)z-',
i=1
where ao(O) "-- 1 so that bx(O, Oo) -- bl(Oo). H(eJo';Oo) for some l, then
;~(d~,; o, Oo)
-
lq~bO
~(d~'; o, Oo)
b,(0o)~-']
i=l
(VI-3) Clearly if H(eJ~'; O)
=
(VI-4)
since H(eJo;O) B(eJo;O)A-l(eJo;O) for any w and 0. Using (VI-2) and (VI-3) this in turn implies that -
-
~[~,(o. Oo)
-
~,(o, Oo)]e -j~'' = 0
(VI-5)
i--1
where ~. - max(nao+nb, na+nbo) and we define 8i(0, 0o) - 0 for n=o+nb + 1 _< i <_ ~. and bi(O, 0o) - 0 for na + nbO + 1 <_ i <_ ~. Since the coefficients 8,(0, 0o)'s and bi(0, 0o)'s are real-valued, it also follows from (VI-5) that
~--~[a, Ce, Oo) -
~,(o, Oo)]d ~'' -
o.
(vI-6)
i=1
By the hypothesis of the lemma, (VI-5) and (VI-6) are true for 1 < l < L. This implies that the ( ~ , - 1)-degree polynomial equation
E[Czi+l(O, Oo) - [~i+l(O,Oo) ]z i - 0 i=0
(VI-7)
STOCHASTIC SYSTEM IDENTIFICATION
73
has 2L roots. Under the hypothesis of the lemma, 2L > n~ + rib > ~, because m i n ( n ~ - n~o, nb -- rib0) _ 0. Now (VI-7) can have only K - 1 < 2L roots; hence, we must have a~+~(0,0o)
-
b~+~(0,0o) fo~ 0 < i _< n -
leading to H(d~;O) - H(d~;Oo). VI.A. BISPECTRAL
1,
(VI-8)
n
APPROACHES
D e f i n i t i o n . A stationary input {u(t)} is persistently exciting of order L w.r.t, the bispectral statistics if its bispectrum B,~,,(w,~,,w,.,) is nonzero at L distinct bifrequencies in the interior of its principal domain. 9 T h e o r e m 4. The linear parameter estimator discussed in Sec. IV.A is (weakly) consistent under the Assumption Set I if (IV-19) and (IV-20) are solved at L frequencies 0 < wl < ~02 < ..- < wL < ~r such that na +rib < 2L, min(na - nao, nb -- rtbO) = 0, and Bu==(m, n - m) =/=0 for all bifrequencies used in (IV-17). 9 Proof: Eqns. (IV-19) and (IV-20) can be expressed as a linear system of equations
FNO = fN
(VI-9)
where 0 is the (rt= + nb)-column vector of unknown parameters, the matrix FN is 2L x (n= + rib) consisting of appropriate parts of the left-side of (IV19) and (IV-20), and fN is an 2L-column vector consisting of the right-side of (IV-19) and (IV-20). By (IV-17) and Lemmas 1 and 2,
limN~ooI-IB(e j~
-- H(eJ'~
i.p.
(VI-10)
Let Foo and foo denote FN and fN, respectively, when the transfer function estimates are replaced with their true values. Then by Lemma 2 we have limN--.ooFN = Foo i.p. and limN--.oofN = foo i.p.
(VI-11)
Since min(n= - nao, nb -- nbO) -- 0, by Lemma 8, ^(IV-19) and (IV-20) have a unique solution given by 0 = O0 provided that HB(e j~ = H(eJ~ 0o) in (IV-19) and (IV-20). Hence, Foo has full rank so that for large N, we have ON - - [ F ~ c F N ] - I F T f N
00
--
[FgF ] -
Foofoo i.p. as N ---, co.
(VI-12)
This proves the desired result. Define inf 0N "-- art{ 0 6 0 C
j~B)(o)}.
(vi-13)
74
JITENDRA K. TUGNAIT
where J(B)(O)is given by (IV-23) and Oc C 0 is a compact set. Convergence of this nonlinear estimator is discussed next. T h e o r e m 5. Suppose that Assumption Set I holds and J(B)(O) utilizes at least L distinct bifrequencies (w,~,w,~) in the interior of the principal domain of B~,~,~(w,~,w,~) such that n , + nb _< 2L, and B ~ ( m , n) 7/=0 for all bifrequencies used in (IV-23). If 0o E @c, then 0N defined in (VI-13) converges i.p. to a set Do as N ~ oo where
D o - { O I B(q-t;O) O) =
B(q-t;O~ 0o) } "
Proof:
See the Appendix. [] The following theorem follows from Theorem 5 and some standard results (see, e.g., [7, Section 6.3]). The additional condition imposed in Theorem 6 ensures that the set Do consists of a single element 0o. T h e o r e m 6. Consider the parameter estimator defined by (VI13). Suppose that Assumption Set I holds and J(B)(O) utilizes at least L distinct bifrequencies (w,~, w,~) in the interior of the principal domain of B~,~,~,(w,~,w,~) such that n~ + n b < 2L, m i n ( n ~ - n~o, nb - nbo) -- 0, and B~,u~,(m, n) =/=0 for all bifrequencies used in (IV-23). If 00 E | then 0N converges i.p. to 00 as N ---, oo. 9 VI.B. INTEGRATED
POLYSPECTRAL
APPROACHES
The following results mimic the corresponding results of Sec. VI.A with obvious modifications. Therefore, we will simply state the main results without any proofs. Definition. A stationary input {u(t)} is persistently exciting of order L w.r.t, the integrated bispectral statistics if its integrated bispectrum S~,~,~(w) is nonzero at L distinct frequencies in the interval (0, ~r).. Definition. A stationary input {u(t)} is persistently exciting of order L w.r.t, the integrated trispectral statistics if its integrated trispectrum S,~(w) is nonzero at L distinct frequencies in the interval (0, ~r).. T h e o r e m 7. The integrated bispectrum-based linear parameter estimator discussed in Sec. V.A.I is (weakly) consistent under the Assumption Set II if (IV-19) and (IV-20) are solved (after replacing ~B(eJ~) with ~iB(eJ~) ) at L frequencies 0 < wl < w2 < "- < WL < 7r such that n~ + nb _< 2L, min(n~- n~0, nb -- rib0) -- 0, and S,~,~(rn) ~ 0 for all frequencies used in
(Iv- 9)
(IV-20).
9
T h e o r e m 8. The integrated trispectrum-based linear parameter estimator discussed in Sec. V.B.I is (weakly) consistent under the Assumption Set Ill if (IV-19) and (IV-20) are solved (after replacing ~rB(ej~) with ~XT(eJ'~") ) at L frequencies 0 < wl < w2 < "" < WL < 7r such that n~ + n b _< 2L,
STOCHASTIC SYSTEM IDENTIFICATION
75
m i n ( n a - n~0, n b - n~0) = 0, and S,.,.,(m) ~: 0 for all frequencies used in (IV-19) and (IV-20). 9 T h e o r e m 9. Consider the nonlinear parameter estimator defined by
rUB)
inf ON "-- arg{ 8 6 |
"N
(8)}
where | C | is a compact set. Suppose that Assumption Set II holds r(XB) and "N (8) utilizes at least L distinct frequencies w,~ in the interval (0, 7r) such that n~ + n b < 2L, m i n ( n ~ - n~0, n b - rib0) - 0, and S,,=,.,(m, n) =/=0 for all frequencies used in (V-11). If 8o 6 Oc, then 0N converges i.p. to 8o aS N
--+ oo.
Theorem
9
10.
O~ . -
Consider the nonlinear parameter estimator defined by
~rg{ o
1.(IT)
inf c eo
"~
(0)}
where | C | is a compact set. Suppose that Assumption Set III holds and rUT) o n (8) utilizes at least L distinct frequencies w.~ in the interval (0, ~) such that n~ + n b < 2L, min(n~ - n~o, nb - n~o) = 0, and S,.,~(m, n) 5/=0 for all frequencies used in (V-20). If 0o 6 Oc, then 0N converges i.p. to Oo as N ~ oo. 9
VII. S I M U L A T I O N E X A M P L E S We now present two simulation examples to illustrate the proposed approaches. Example 1 is from [39] and Example 2 is from [40],[34]. In the both the examples the proposed approaches were applied using a block length LB=64 and a smoothing window M = I (/l~r - 0, i.e., no smoothing). The software package NL2SOL [15] was used for nonlinear optimization using the results of the linear approaches as initial guesses. E x a m p l e 1 [39]. The system to be identified is given by
~(~) -
1.5~(t-1)
-
0.7~(t-2)
+ ~(t-1)
-
0.5~(t-2).
(vii-l)
Thus, we have bl - 1.0, b2 - -0.5, al - -1.5, and as - 0.7. The input {u(/)} is generated as
~(t) -
0.3~(t-1)
+ ~(~)
(vii-2)
where {e(t)} is an i.i.d, binary sequence with values 1 a n d - 1 , each with probability 0.5. Thus, we have E{e2(t)} - 1., 73~ - 0. and 74~ - - 2 . The noisy input-output observations are given by y(~) -
~(~)+
~o(~),
76
JITENDRA
~(t) = ~ ( t ) +
K. TUGNAIT
v~(~).
The mutually uncorrelated colored Gaussian noises {vi(t)} and { V o ( t ) } are generated as v,(t)
= 0.8vi(t-1)
+ fi(t)
(VII-3)
and ~o(t) =
- o . o ~o(t - 1) + Yo(t)
(vii-4)
where { f i ( t ) } and { f o ( t ) } are mutually independent i.i.d., zero-mean Gaus~i~n ~ . q u ~ . ~ . The ~ i ~ . ~ of f,(t) ~.d fo(t) ~ . ~ele~t.d to yield signal to noise power ratio of 20dB at the input sensor and the output sensor, respectively. Thirty independent realizations of 4000 input-output data pairs were generated. The methods proposed in Sec. V.B of the chapter were applied along with several existing approaches. The approaches of Sec. V.B were applied using a block length LB=64 and a smoothing window M = I (M = 0, i.e., no smoothing) 9 For the linear estimator, we used 1 < I< b_z__ 1 - 30 ' __ m 2 i.e., 30 frequency points in the interval (0, ~r). The nonlinear estimator minimizing -rUT u )(0) given by (V-20) was initialized by the linear estimator. The frequency points used in (V-20) were identical to that for the linear estimator. Table I displays the arithmetic means, standard deviations and root mean-square (RMS) errors of the results of the various parameter estimators. The approach of [39] is based upon the fourth cumulant of a generalized error signal at zero lag, and the approach of [35] uses the fourth-order cross- and auto-cumulants of the input-output data at various lags. The results pertaining to the approach of [35] have been taken from [35]. For comparison the results of a least-squares criterion (labeled MSE in Table I) are also shown. It is seen that the MSE criterion relying only on the second-order statistics of the input-output record, produces a substantial bias (and low variance) for this example whereas the proposed frequencydomain approaches proposed in this chapter yield very good results: the variance is comparable to the MSE criterion and the bias is much reduced. The proposed approaches perform much better than the approaches of [39] and [35]. E x a m p l e 2 [40],[34]. The system to be identified is given by s(t)
= 1.5 s(t - 1) -- 0.T s(t -- 2) + u ( t ) + 0.5 u ( t - 1).
(VII-5)
Thus, we have b 0 - 1.0, bl - 0.5, al - -1.5, and a2 - 0.7. The input {u(t)} is obtained as u(t)
= e(t)
m 0 . 2 e ( t - - 1 ) + 0.3e(t--2)
(VII-6)
STOCHASTIC SYSTEM IDENTIFICATION
77
T A B L E I. Parameter Estimates: Example 1, 30 Monte Carlo runs, SNR = 20 dB, N - 4000 - input-output data record length in each run, or= one standard deviation. [ IT: Integrated trispectrum based approaches of Sec. V.B.; MSE: a meansquare error criterion minimizing the prediction errors, see criterion (76) in [39]. ] Parameters True Values Approach 1
IT: (Linear) (Sec. V.B.1)
i[
(Nonlinear) (Sec. V.B.2)
mean O"
RMS mean O"
RMS mean o"
[351
RMS mean O"
MSE (from [39])
RMS mean O"
RMS
al 82 -1.500 0.700 estimate statistics: N -1.4257 0.6364 • • (0.0782) (0.0664) -1.4754 0.6774 • • (0.0257) (0.0232) 0.6i89 -1.3363 4-0.2366 • (0.2877) (0.2153) -1.5631 0.7494 4-0.2585 • (0.2661) (0.2194) O.2587 -0.9490 • • (o.5511) (0.4413)
bl 1.000 - 4000 0.9832 • (0.0173) 0.9839 • (0.0182) 0.9562 • (0.2138) 1.0142 4-0.0113 (0.0182) 0.9902 • (0.0145)
b2 -0.500
04391j]
• (0.0688) -0.4845 • (0.0215) -0.4537 • (0.2349) -0.5761 • (0.2754) 0.0782 • (0.5732) ......
78
JITENDRAK. TUGNAIT
where {e(t)} is an i.i.d., zero-mean, unit variance (one-sided) exponential sequence. Thus, we have E{e2(t)} = 1. and "y3~ = 2. The noisy inputoutput observations are given by
y(t) = ~(t) + ,o(t), ~(t) = = ( t ) +
,~(t).
The mutually correlated colored Gaussian noises {vi(t)} and generated as vi(t) = fi(t) - 2.33fi(t - 1) +
0.75fi(t -
{vo(t)}
are
2) + 0.5fi(t - 3)
+0.3fi(t - 4) - 1.4fi(t - 5)
(VII-7)
and Vo(t)
=
,,(t)
+ 0 . 2 v , ( t - 1) - o . 3 " , ( t - 2) + 0 . 4 " , ( t - 3),
(VII-8)
where {fi(t)} is an i.i.d., zero-mean Gaussian sequence 9 The variances of vi(t) and vo(t) are selected to yield a signal to noise variance ratio of 5 dS at the input sensor and the output sensor, respectively. One hundred independent realizations of 4000 input-output data pairs were generated. The methods proposed in Secs. iV.B and V.A.2 of the chapter were applied along with several existing approaches. The proposed approaches were applied using a block length LB=64 and a smoothing window M = I (/17/ = 0, i.e., no smoothing). The nonlinear estimator mini ( I S ) (0) given by (V-11) was initialized by the linear estimator of imizing--N Sec V.A.1 for which we used 1 < l < L s _ 1 -- 30, i.e. 30 frequency points in the interval (0, v). The frequency points used in (V-11) were identical to that for the linear estimator. A similar procedure was followed for minimization of J(B)(O) where we selected all the frequency points in the interior of D ~ . Table II displays the arithmetic means, standard deviations and root mean-square (RMS) errors of the results of the various parameter estimators. The approach of [31] uses the third-order cross- and auto-cumulants of the input-output data at various lags; so do the approaches of [40] and [34]. The results pertaining to the approaches of [34] and [40] have been quoted from [34]. It is seen from Table II that the frequency-domain approaches proposed in this chapter yield the best results and the approaches of [31] and [34] the worst.
STOCHASTIC SYSTEM IDENTIFICATION
79
T A B L E II. Parameter Estimates: Example 2, 100 Monte Carlo runs, SNR - 5 dB, N - 4000 - input-output data record length in each run, dr- one standard deviation. [ IB: Integrated bispectrum based approach of Sec. V.A.]
Parameters True Values Approach IB: (Nonlinear) (Sec. V.A.2)
mean
Bisp.: (Nonlinear) (Sec. IV.B)
[31]
[34]
[40] (Cross-cum.)
at a2 b0 -1.500 0.700 1.000 estimate statistics: N - 4000
bl 0.500
0.6818
1.0333
•
+o.oo31
:o.ol02
0.4419 4-0.0366
RMS
(0.0191)
(0.0185)
(0.0384)
(0.0686)
mean dr
-1.4555 4-0.0245
1.0647 4-0.0435
0.4173 4-0.0487
RMS
(0.0508)
0.6556 4-0.0239 (0.0504)
(0.0779)
(0.0060)
mean dr
-1.8394 4-0.4095
1.0351 4-0.3834
1.0084 4-0.0330
RMS
(0.5319)
(0.5092)
(0.0341)
0.1561 4-0.4232 (0.5453)
mean dr
1.6200 4-1.1680
0.8430 4-1.4300
RMS
(1.1740)
0.3690 4-0.9710 (0.9798) ,I
mean dr
-1.4444 4-0.0472
1.0596 4-0.1278
0.5377 4-0.1485
RMS
(0.0729)
(1.4371) 0.6496 4-0.0406 (0.0647)
0.9700 4-0.2450 (0.2627)
(0.1410)
(0.1532).,
dr
-1.4815
,
80
JITENDRAK. TUGNAIT
VIII. CONCLUSIONS Two new classes of parametric frequency domain approaches were presented for estimation of the parameters of scalar, linear errors-in-variables models. One of the proposed classes of approaches consists of linear estimators where the bispectrum or the integrated polyspectrum (bispectrum or trispectrum) of the input and the cross-bispectrum or the integrated cross-polyspectrum, respectively, of the input-output are exploited. Based on these polyspectra the system transfer function is first estimated at a number of frequencies exceeding one-half the number of unknown parameters. The estimated transfer function is then used to estimate the unknown parameters using an overdetermined linear system of equations. In the second class of approaches, quadratic transfer function matching criteria are optimized by using the results of the linear estimators as initial guesses. Both classes of the parameter estimators are shown to be consistent in any measurement noise that has symmetric probability density function when the bispectral approaches are used. The proposed parameter estimators are shown to be consistent in Gaussian measurement noise when trispectral approaches are used. The input to the system need not be a linear process but must have nonvanishing bispectrum or trispectrum. We emphasize that unlike the second-order statistics case, one can not, in general, model a stationary random process with a given higher-order cumulant spectrum as having been generated by driving a linear system with an i.i.d, sequence [27]. Computer simulation results were presented in support of the proposed approaches. Performance comparisons with several existing approaches show that the proposed approaches outperform them. On the theoretical side, the proposed estimators are consistent under sufficient conditions that are more general than that for existing approaches. Moreover they apply with almost equal computational and programming ease to both cases, those involving third-order statistics as well as those concerned with the fourth-order statistics. We considered only single-input single-output models. There are no essential difficulties in extending them to multiple-input multiple-output models although details remain to be worked out.
ACKNOWLEDGMENTS This work was supported, in part, by the National Science Foundation under Grant MIP-9101457. Part of this chapter represents work done jointly with my graduate student Yisong Ye.
STOCHASTICSYSTEMIDENTIFICATION
81
REFERENCES [1] T. SSderstrSm, "Spectral decomposition with application to identification," in Numerical Techniques for Stochastic Systems, F. Archetti and M. Cugiani (eds.), North-Holland: Amsterdam, 1980. [2] T. SSderstrSm, "Identification of stochastic linear systems in presence of input noise," Automatica, vol. 17, pp. 713-725, Sept. 1981. [31 B.D.O. Anderson and M. Deistler, "Identifiability in dynamic errorsin-variables models," J. Time Series Analysis, Vol. 5, pp. 1-13, 1984. [4] B.D.O. Anderson, "Identification of scalar errors-in-variables models with dynamics," Automatica, vol. 21, pp. 709-716, 1985. [5] M. Deistler, "Linear errors-in-variables models," in S. Sittanti, (ed.), Time Series and Linear Systems, Lecture Notes in Control and Information Sciences, Vol. 86, pp. 37-86, 1986. [6] M. Deistler and B.D.O. Anderson, "Linear dynamic errors-in-variables models: Some structure theory," J. Econometrics, vol. 41, pp. 39-63, 1989. [7] T. SSderstrSm and P. Stoica, System Identification. Prentice Hall Intern.: London, 1989. [8] P. Stoica and A. Nehorai, "On the uniqueness of prediction error models for systems with noisy input-output data," A utomatica, vol. 23, pp. 541-543, 1987. [9] M. Green and B.D.O. Anderson, "Identification of multivariable errorsin-variables models with dynamics," IEEE Trans Automatic Control, vol. AC-31, pp. 467-471, 1986. [10] A.G. Piersol, "Time delay estimation using phase data," IEEE Trans. Acoustics, Speech, Signal Processing, vol. ASSP-29, pp. 471-477, June 1981. [11] W.S. Burdic, Underwater Acoustic System Analysis. Prentice-Hall: Englewood Cliffs, N.:I., 1984. [12] C.L. Nikias and R. Pan, "Time delay estimation in unknown Gaussian spatially correlated noise," IEEE Trans. Acoustics, Speech, Signal Processing, vol. ASSP-36, pp. 1706-1714, Nov. 1988. [13] :].K. Tugnait, "Time delay estimation in unknown spatially correlated Gaussian noise using higher-order statistics," in Proc. ~3rd Asilomar Conf. Signals, Systems, Computers, pp. 211-215, Pacific Grove, CA, Oct. 30- Nov. 1, 1989.
82
JITENDRAK. TUGNAIT
[14] H. Akaike, "On the use of non-Gaussian process in the identification of a linear dynamic system," Annals of the Institute of Statistical Mathematics, vol, 18, pp. 269-276, 1966. [15] :I.E. Dennis, Jr. and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall: Englewood Cliffs, NJ, 1983. [16] M. Deistler, "Linear dynamic errors-in-variables models," in :I. Gani and M.B. Priestley, Essays In Time Series And Allied Processes, Journal of Applied Probability, Special volume 23A, pp. 23-40, 1986. [17] V. Solo, "Identifiability of time series models with errors in variables," in :I. Gani and M.B. Priestley, Essays In Time Series And Allied Processes, Journal of Applied Probability, Special volume 23A, pp. 63-74, 1986. [18] R.E. Kalman, "Identifiability and modeling in econometrics," in P.R. Krishnaiah, Editor, Developments in Statistics, vol. 4, pp. 97-136, New York" Academic, 1983. [19] K.:I./~strSm and P. Eykhoff, "System identification- A survey," A utomatica, vol. 7, pp. 123-162, 1971. [20] L. Ljung, System Identification: Theory for the User. Prentice-Hall: Englewood Cliffs, N.J., 1987. [21] T. Subba Rao and M.M. Gabr, An Introduction to Bispectral Analysis and Bilinear Time Series Models. New York: Springer-Verlag, 1984. [22] K.S. Lii and M. Rosenblatt, "Deconvolution and estimation of transfer function phase and coefficients for non-gaussian linear processes," Ann. Statistics, vol. 10, pp. 1195-1208, 1982. [23] D.R. Brillinger, "An introduction to polyspectra," Annals Math. Statistics, vol. 36, pp. 1351-1374, 1965. [24] M. Rosenblatt, Stationary Sequences and Random Fields. Birkhs Boston, 1985. [25] M.:I. Hinich, "Testing for Gaussianity and linearity of a stationary time series," J. Time Series Analysis, vol. 3, no. 3, pp. 169-176, 1982. [26] B.Y. Hamon and E.:I. Hannah, "Spectral estimation of time delay for dispersive and non-dispersive systems," J. Royal Statis. Soc. Set. C (Applied Statistics), vol. 23, pp. 134-142, 1974.
STOCHASTICSYSTEMIDENTIFICATION
83
[27] A.M. Tekalp and A.T. Erdem, "Higher-order spectrum factorization in one and two dimensions with applications in signal modeling and non-minimum phase system identification," IEEE Trans. Acoustics, Speech, Signal Processing, vol. ASSP-37, pp. 1537-1549, Oct. 1989.
[28]
J.K. Tugnait, "Stochastic system identification with noisy input using cumulant statistics," in Proc. 29th IEEE Conf. Decision FA Control, pp. 1080-1085, Honolulu, Hawaii, Dec. 5-7, 1990.
[29]
M. Deistler, "Symmetric modeling in system identification," in H. Nijmeijer and J.M. Schumacher (Eds.), Three Decades of Mathematical System Theory, Lecture Notes in Control & Information Sciences, Springer, 1989.
[30]
K.S. Lii and K.N. Helland, "Cross-bispectrum computation and variance estimation," A CM Trans. Math. Software, vol. 7, pp. 284-294, Sept. 1981.
[31]
Y. Inouye and H. Tsuchiya, "Identification of linear systems using input-output cumulants," Intern. J. Control, vol. 53, pp. 1431-1448, 1991.
[32]
J.A. Cadzow and O.M. Solomon, Jr., "Algebraic approach to system identification," IEEE Trans. Acoustics, Speech, Signal Processing, vol. ASSP-34, pp. 462-469, June 1986.
[33]
J.K. Tugnait and Y. Ye, "Stochastic system identification with noisy input-output measurements," in Proc. 26th Annual Asilomar Conf. Signals Systems Computers, pp. 741-745, Pacific Grove, CA, Oct. 26-28, 1992.
[34]
A. Delopoulos and G.B. Giannakis, "Strongly consistent output only and input/output identification in the presence of Gaussian noise," in Proc. 1991 ICASSP, pp. 3521-3524, Toronto, Canada, May 14-17, 1991.
[35]
Y. Inouye and Y. Suga, "Identification of linear systems with noisy input using input-output cumulants," Intern. J. Control, to appear. [Also in Proc. IEEE Signal Proc. Workshop on Higher Order Statistics, pp. 9-13, South Lake Tahoe, CA, June 7-9, 1993.] K.L. Chung, A Course In Probability Theory. New York: Harcourt, Brace, World Inc., 1968. E.:I. Hannan and M. Deistler, The Statistical Theory of Linear Systems. Wiley: New York, 1988.
84
JITENDRA K. TUGNAIT
[38]
G.J. Dobeck, V.K. Jain, K.W. Watkinson, and D.E. Humphreys, "Identification of submerged vehicle dynamics through a generalized least squares method," in Proc. 1976 IEEE Conf. Decision and Control, Clearwater , FL, Dec. 1976, pp. 78-83.
[39]
J.K. Tugnait, "Stochastic system identification with noisy input using cumulant statistics," IEEE Transactions on Automatic Control, vol. AC-37, pp. 476-485, April 1992.
[40]
J.M.M. Anderson and G.B. Giannakis, "Noisy input/output system identification using cumulants and the Steiglitz-McBride algorithm," in Proc. ~5th Asilomar Conf. Signals, Systems, Computers, Pacific Grove, CA, pp. 608-612, Nov. 1991.
[41]
M.3. Hinich and G.R. Wilson, "Time delay estimation using the cross bispectrum," IEEE Trans. Signal Proc., vol. ASSP-40, pp. 106-113, Jan. 1992.
[42]
D.R. Brillinger, Time Series Data Analysis and Theory. New York" Holt, Rinehart and Winston, 1975.
[43] D.R. Brillinger and M. Rosenblatt, "Asymptotic theory of estimates of kth order spectra," in Spectral Analysis of Time Series, B. Harris, Ed. New York: Wiley, 1967, pp. 153-188.
[44]
J.K. Tugnait, "On time delay estimation with unknown spatially correlated Gaussian noise using fourth order cumulants and cross cumulants," IEEE Transactions on Signal Processing, vol. SP-39, pp. 1258-1267, June 1991.
[45] G.C. Goodwin and K.S. Sin, Adaptive Filtering, Prediction And Control. Prentice-Hall: Englewood Cliffs, N.J., 1984. [46] L. Ljung, "Convergence analysis of parametric identification methods," IEEE Trans. Automatic Control, vol. AC-23, pp. 770-783, 1978.
[47]
P.H.M. Janssen, "General results on the McMillan degree and the Kronecker indices of ARMA and MFD models," Intern. J. Control, vol. 48, pp. 591-608, 1988.
[48] D. Donoho, "On minimum entropy deconvolution," in Applied Time Series Analysis, II, D.F. Findley (Ed.), Academic: New York, 1981. [49] E. Walach and B. Widrow, "The least mean fourth (LMF) adaptive algorithm and its family," IEEE Trans. Information Theory, vol. IT-30, pp. 275-283, March 1984.
STOCHASTIC SYSTEM IDENTIFICATION
85
APPENDIX In this appendix we provide a proof of Theorem 5. We need some auxiliary results to accomplish this. L e m m a 9. Under the Assumption Set I, it follows that limN_,ooJ(B)(O)=
~
j(s)(O ) I H(eS(~'~+~)10~ -
H(eY(""'+"")IO)12
(A-l)
i.p. uniformly in 0 for 0 6 | where Oc is an arbitrary compact set consistent with the model assumptions, SL is a fixed collection of L bifrequencies and
~,: (Oo) _ s . ( , . ) s . ( . ) s ~ ( , . + .) I B . . . ( m . . ) l ~Proof:
By (III-29) and Lemmas 1 and 2, we have
limN_, oo5~,~ = cr~,~(00) i.p.
(A- 2)
By (III-26), (A-2) and Lemmas 1 and 2, we have limN__,oo/~ (ey (~.~+ ~ ) ) ~^-1 . -
H(~(~-+~)leo)~.-~
(eo) i...
(A-3)
Consider
D,.,.,,(O) "-I~r(~c~-+~)) _
-_
-
H(e~('...+'..)l o) I~ O'mn ^-~
-
i H(e~(,,,,.,+,,,,,)lOo)
_
i ~(eS(~.+~))i z
H(e~(~.+~)10) 12 O^-z 'rn n
O.mn _--2
Re{H(eY("...+'~..)lO)[H(e~(,.,...+,~..)[Oo)
I H( ej('~
sup sup ~,~,w~ e [-Tr, 7r] 0 e Oc
O. '-m2n
- ~r (eY(~...+,~..))]* ~^-z , , , }.
By compactness of @c and continuity of and w,~, we have
wm
(A-4)
H(ei("''.+~")lO)
I H(eJ(""'+"")lO )
+ (A-~)
in 0 as well as in
I< M < oo
(A-6)
By (A-2), (A-3) and L e m m a 9.,given any e and 6 > 0 there exists an integer
N(e, 6) such
that
P{I [l~(ej(~..+~))12 _
IH(eS(,,,.,,+,,,,.)lOo)l~]~.,..-2 I<
' } > 1 -- ~
(A-7)
86
J I T E N D R A K. T U G N A I T
and P{[ [B'(ej(''''+''')) - H(eJ("+"")leo)]&ma~
I< ' ) > 1 - 6,
for every N > N(e, 6). Hence it follows that given any el (= > 0 there exists an integer N ( e l , 6 ) such that P{[Dm~(O)l < el} > 1 - 6 V0 E |
VN > N(el,~).
e+
(A-8)
2Me) and (A-9)
That is, convergence of D,.,.,,.,(O) is uniform in /9. The desired result now follows by using the uniform convergence of D,~,~ (0), Lemma 2(i) and noting that
IJ(oo')(0)-
ID.~.(0)I <_ Lel
<_
(A-10)
( m , n ) 6 S t , C Do~
if [Dm.(e)[ < e~. [] L e m m a 10. Under the Assumption Set I, we have J(ooB)(o) -- O for any O E |
r
OED0
9
P r o o f : From (A- 1)
&oo(O) - o => H(e~("'+"")lOo) - H(e~('~
)
for (m, n) ~ SL c D ~o :=> H(eY'[Oo ) =r
H(e~'lO)
Vw 6 (-Tr, 7r], by Lemma 8
06Do.
Conversely, 8 C Do
:::> H(eJ('~"+"")]8o)
-
H(ei(''+~")]8)
for (m, n) C SL C D ~o y ::~ J (off) ( 8 ) - O.
This completes the proof. [] P r o o f of T h e o r e m 5. Define a set |
"-- |
- S(Do, p)
where S(Do,p)
-- { 8 1 8 E 0 ,
inf 0 E Do
II e - e II< p }.
STOCHASTICSYSTEMIDENTIFICATION
87
Since S(Do, p) is open, O(p) is a closed subset of a compact set @c C (9, hence compact. B y continuity of j(S)(O)in O and compactness of O(p), there exists some O* 6 | such that
o einf e(p) j(ooB)(o)
J(ooB)(o*)
-
-
.
(A-11)
u(.) > o - &)(Oo)
where we have used Lemma 10 in deducing that #(p) > 0. By Lemma 9, given any e 0 there exists an integer gl(e, ,5) such that V0 6 @(p) and VN _> N1 (e, 6)
P{J(B)(O) > J(s)(o)
-
e >_
I~(P) e} > 1 - ,5.
(A-12)
The above equation may be rewritten as inf
P{ o ~ e(o)
j(s )
(o) > ~ ( p ) - ~} > 1 - ~
VN _> N~(~, 6).
(A-~3)
Similarly, by Lemma 9, given any e < #(p)/3 and 6 > 0 there exists an integer N2(e, ,5) such that V0 E Oc and VN > N2(e, 6) we have
P{J~B)(8)< ff(off)(O)+ e} >
1
-
6.
(A-14)
In particular, we have
P{J(s)(oo) < J(off)(Oo)+ e - e} >
1-6
VN > N~(E, 6).
(A-~S)
However, since
~f j~B~(0) _<s~(Oo), 06(90 it follows from (A-14) and (A-15) that
J(s)(o)
P{ O6|
VN _> N 2 ( e ,
< e _ 1 - - 6
6).
(A-16)
Consider 0N as defined in (V-13). Then we have inf 0COo
P{0N 6 S ( D o , p ) } -- P
>P
inf
OEec
J(B)(o) < e -<
~,(p) - ,
<
inf
j(NB)(0) <
inf 0 6 e(p)
#(p)/3
o e e(p)
'
j(B)
} (0)
j(B) (0) }
88
JITENDRAK. TUGNAIT
> I -
inf
P
060c
J(B)(O)>/~(p)/3 i~f
-
> I - 26
P
8 6 6)(p)
VN
> N(~,6)
>_
c}
Jc.~)(e) < .(p)-,}
= m~x{N~(~.~). N~(..6)}.
-
(A-17)
where we used (A-13) and (A-16) to obtain (A-17). This proves the desired result. []
Robust
Stability
Randomly
of Discrete-Time
Perturbed
Systems
Edwin Engin Yaz Electrical Engineering Department University of Arkansas Fayetteville, AR 72701
I. I N T R O D U C T I O N Maintaining a control system's stability in the presence of parameter perturbations is of prime importance in system design and therefore much research effort has been spent in this area for improving sufficient conditions for stability robustness. In the present work, our main focus will be on state space perturbation results. These can be classified either according to the structural information about the perturbations as nonlinear, linear unstructured, correlated and highly structured etc. or the type of analysis method used like Lyapunov-based methods, Gronwall-Bellman inequality methods, Gershgorin circle results etc. We will not try to give a list of works in the area of deterministic perturbations which is left to the surveys like [1]-[3], but we will give a brief account of the existing approaches to random parameter robustness and provide some new results. In this type of stochastic analysis, perturbations are allowed to be arbitrarily large for some sample paths in contrast to the deterministic theory which assumes unknown but bounded perturbations. The most common CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
89
90
EDWIN ENGINYAZ
type of random parameter model is the white noise (timewise uncorrelated) with zero mean. Such models have been used in satellite attitute control [4], in macroeconomic theory [5], population dynamics [6],[7],vibrations analysis [8], to model roundoff errors in digital control computations [9], chemical reactor stability [10], etc. In modelling systems with large changes in operation region in state space, jump Markov parameters are preferred over white random sequences. Many applications can be cited [11]-[14] involving economic models with sudden jumps in parameter values, failure occurence in power systems and aircraft controls, tracking of targets performing evasive maneuvers, steam temperature control for solar receivers, etc. In this work, we will present a survey of existing results and introduce some new ones on the robust stability of discrete-time systems with random perturbations. We will start with perturbation models and stochastic stability concepts needed in this work. Next, white noise parameter perturbations will be considered and a survey of results together with some new ones will be provided for nonlinear, unstructured and structured linear perturbations. Then, we analyze what happens when a colored noise perturbation is treated as white and develop an approach to stability assessment in such situations. Also, new robustness results will be given for systems with j u m p Markov type perturbations, and mixed jump Markov and white noise perturbations. Finally, a discussion of how these robustness results can be used in control problems will follow.
The following notation is used throughout the manuscript. Af denotes the set of natural numbers. For an n-dimensional real vector x E l:t n, the general vector norm is II Xllr = ( ~ = 1 IXilr) 1/" In this work if the subscript of the norm is missing, then it means that any vector norm can be used in
STABILITYOF RANDOMLYPERTURBEDSYSTEMS
91
that expression. For an n x n dimensional matrix A E R n•
with real ele-
ments,
A T,
]A],
p(A),
AI(A)(An(A)),
al(A)(an(A)),
A >_ 0(A > 0), tr[A],
respectively denote the transpose of A, the matrix formed by replacing every element of A by its absolute value, the spectral radius of A, the maximum (minimum) eigenvalue of a symmetric matrix (A), the maximum (minimum) singular value of A, nonnegative(positive) definite symmetric matrix A and trace of A. A | B is the Kronecker product of A and B, A [2] is the Kronecker square or A | A, and st[A] is the stack (vector) operator applied on A. Most frequently used properties involving the Kronecker products and stack operators used in this work are st[ABC] = (cT|
A)st[B], tr[AB] = stW (A r) st (A) where stW(A T) = [st(AZ)] r, and
st(A + B) = st(A) + st(B), w C ~2 where (f2,/E, 7)) is a probability space,
E{x}
= 2 both denote expectations and E{x}$c} conditional expectation
for ~ E K~ a a-algebra of subsets of ~2. II. M O D E L S
AND MATHEMATICAL
PRELIMINARIES
Let us consider the nonlinearly perturbed system
Xk+l
--
Axk + f k(xk, w)
(1)
where xk E R n is the state, A is a constant matrix representing the known nominal part of the plant, and fk is a state-dependent random sequence that represents the perturbation. The nominal part will be assumed to possess a stability margin represented by
p(A)
< (1 + c~)-1/2 for some a > 0
so that the degree of robustness of the nominal system to destabilizing influences of the perturbation will be determined in terms of this stability degree. Because of this stability property, there exists a unique solution
p=pT>oto P = (1 +
a)ArPA + Q
(2)
92
EDWIN ENGINYAZ
for any matrix Q = QT > 0. Two kinds of nonlinear perturbations will be considered: i) Nonlinear function of xk dependent on a white noise perturbation, the only information about whom is a bound on the trace of its second moment E{II fk(xk,
w)ll 2} <_ l,=E{ll xkll2}, ~ ~ (0, ~ )
(3)
This model can effectively be used for cone-bounded nonlinear vector functions of state (saturation, dead-zone, piecewise linear, limiter, etc.) multiplied by white noise as in fk(xk, w) = Ck(xk)~(w) with al(r
<
II ~k II where v~(w) is a standard (with zero mean and identity covariance matrix) white noise vector. This is the stochastic counterpart of the model in [15]. The problem here is of determining the largest bound l' on the perturbation which can be tolerated by the system without causing instability. ii) Perturbation dependent on a nonlinear function of the state and a homogeneous Markov chain rk with a finite number of states rk = i, i = 1, ...,N with transition probabilities 7rij = 7){rk+l = j, rk = i} which form the elements of the matrix [//]ij = 7rij and initial probabilities 79{r0 = i} = 7ri. Corresponding to perturbation bound (3), in this case we have
II A(~k, r~ = ~3 I1_
(4)
The unstructured linear counterparts of these perturbation models will be
xk+l = (A + Dk(w))xk
(5)
where Dk(w) is a matrix whose elements are either white, second order (with bounded first two moments) random sequences or dependent on states of a finite Markov chain.
In either case, the stochastic parameters will
STABILITYOF RANDOMLYPERTURBEDSYSTEMS
93
be independent of the initial state. Bounds will be found on the second moment of the perturbation matrix to maintain stability. In the structured linear case, we will use the perturbation model m
Dk(w) = E dkj (w)DJ
(6)
j=l
where the unknown part is d~j (w), real valued scalar random sequences and D j are known constant matrices. Bounds on the noise variances of dkj will be derived for the preservation of stability. The following stability types, which are the most important in practice, will be used in this work: The trivial solution of equation (1) is called mean square exponentially stable(m.s.e.s.) if for all [] x0 I1< c~, there exist/3 > 0, a C (0, 1) such that
E{[[ ~kll2} ~/3~ k,
for all k E Af
(7)
The trivial solution of equation (1) is called almost surely (with probability one or sample path) exponentially stable (a.s.e.s.) if for all II x0 [[< c~, there exist fl > 0 and c~ E (0, 1), such that Hxk ]]_
(8)
It is shown for both types of random perturbations that the m.s.e.s. property implies a.s.e.s. [16]-[17]. The Lyapunov (positive super-Martingale) method [18]-[19] of stability assessment that will be used requires the existence of a candidate Lyapunov function
V(x, r)
for x E R n, r -
i, i = 1, ...,N with
b~ II xll 2 ~ V(~, ~) ~ 52 II xll ~, bx, 62 > 0
(9)
94
EDWIN ENGIN YAZ
t h a t satisfies
z
r)
II
(10)
> 0
in which case both m.s.e.s, and a.s.e.s, properties are valid. s 9R n • Af R is defined for white noise perturbations as
s t~. = s V(xk) := E{ V(Axk + fk)lxk} -- V(xk)
(lla)
where N - 1 and for j u m p Markov perturbations as
s ~ i = s V(xk, i) := E { V(Axk + Ik, r~+l)lxk, r~ = i} - V(xk, i)
(116)
Similar to the Lyapunov theory for deterministic systems, q u a d r a t i c functions are used in stochastic p a r a m e t e r linear system stability analysis problems.
So, for white p e r t u r b a t i o n s in system (5), we can use K. =
xk TPxk, P = p T > 0 and obtain s K. = xkT[ATPA + A T P D + D T P A + D T P D - P]xk
(12)
so t h a t the negative definiteness of the matrix in the square brackets will guarantee stability in b o t h senses.
Therefore, based on the exponential
stability characteristics of the known part A T P A -
P in this expression,
bounds on the mean D and the weighted second m o m e n t D T p D of the p e r t u r b a t i o n matrix need to be determined to maintain stability. For Markov type parameters, we assume
V(x, i) = x r p i x , pi = piT > 0 and obtain N
Z K.i= xkT[(A + D,)T E j--1
PJniy(A + D,) - Pilxk
(13)
STABILITY OF RANDOMLY PERTURBED SYSTEMS
for
95
i,j = 1, ..., N. In this equation, there are N matrix expressions to give
negative definite results for both types of stability. For linear systems of the form (5), it is also possible to directly write the second moment evolution equations as
Pk+l = APkA T + APkD T + DPkA T + DP~.DT, Pk - xkxk T
(14)
and N
PJ k+l = E
(A + D,)Pk 'Trij (A + D,) r
(15)
i--1
respectively for white and Markov perturbations. In this case, it is easy to see that the necessary and sufficient conditions for m.s.e.s, are respectively
p(A|174
<1
for white noise perturbations and p(r
< 1
...
~rNI(A + DN). | (A + DN) I
(17 )
where
I ~n (A + DI ) | (A + D~) "
o~
rlN(A + 01) | (A + D1) "..
( Tb)
7rNN(A + DN) | (A + DN)
for Markov type perturbations. The above conditions are obtained by applying stack operator to both sides of (14) and (15) and then using the elementary properties of Kronecker products as mentioned in section I. See [20] for details.
These conditions are also sufficient for a.s.e.s,
mented above. III. W H I T E N O I S E P E R T U R B A T I O N S A. N o n l i n e a r P e r t u r b a t i o n s
as com-
96
EDWIN ENGIN YAZ
Let us consider the perturbed system xk+i
=
(i)
Axk + Ik(xk, w)
where the nominal system parameter has the property
p(A) < (1 + a)-~/2
(18)
for some a > 0. This stability margin will be used to find estimates of # which is the bound on the second moment of the perturbation vector in (3). We first present the robustness results given in [21]: Theorem 1. Let (18) hold. The perturbed system (1) is m.s.e.s, and a.s.e.s. if ~, < ~,~ = [ ~ . ( Q ) ( t
+ ~)-~A~-~(P)] ~/~
(19)
where P, Q are given by equation (2). Proof. The quadratic Lyapunov function candidate ~ -- xkTpxk, where P > 0 satisfies (2) due to (18) and the inequality
(a.i/2Axj: - oFV2f k) rP(at/2Axk - oFV2f ~:) >_0 used in the form
xkTATpfk lead to
+
fkTpAxk <_axkTATpAxk
+
a-lfkTPfk
1:.V(xk) <_xkr[(1 + a ) A T p A - P]xk + (1 +
O~-~)E{.fkTP.hlXk}
<_xkT[--Q + (t + a-~),~(P)it2]xk < -[,~,~(Q) - (t + o~-l),~(P)l~2l
II ~k II~
where we substituted from (2) and (3). This inequality interpreted in light
of e q ~ t i o ~
(9), (10), (11~) will Sve the r~ult.
STABILITYOF RANDOMLYPERTURBEDSYSTEMS
97
Two important points are brought out in the work [21]. First, the stochastic perturbation in (1) and a deterministic sector nonlinearity in [15] result in the same bound in (19). This allows us to use the result in [15] that the choice Q = I will yield the maximum bound over all Q = QT > O. Second, when the stability radius (1 + a) -1/2 of the nominal part is found, the maximum bound is not given by the corresponding c~ma~ over all the possible a values in [0, amax] but by a value in this interval. For normal A matrices, the optimal a value is obtained analytically as s ~ - p-2(A) - 1
(20)
that yields the largest tt value It ~ = 1 - p ( A )
(21)
When the A matrix is not normal, an analytical solution for the maximum bound does not seem possible, so a numerical technique can be used as in [15] to find the largest bound possible by this technique. Theorem 2. For the same system as in theorem 1, the same results hold if it < tt2 = - a l ( A ) +
{a21(A) + [c~A~(P)+ An(Q)](1 + a ) - l A l - l ( P ) } l / 2 (22)
To prove this theorem, Schwarz inequality is used. Other than that the proof is entirely similar. Theorem 3. For the same system as in theorem 1, the same results hold if l, <
= [o.5
Z-2(1 +
(23)
where/3 > 0 is such that a l ( A k) < fl(1 + a)-k/2
(24)
98
EDWIN ENGIN YAZ
This result is proved by the use of the discrete Belhnan-Gronwall lemma in [21]. Corollary 1. If A is diagonalizable and t' < It4 = [0.5c~3"-2(1 + O~)-111/2
holds where 7 = ffl(T)(71(T--l)
(25)
with T being the similarity transformation
that diagonalizes A, then the perturbed system (1) is m.s.e.s, and a.s.e.s. If A is a normal matrix, then 3' = 1. The proof is similar to that of theorem 3.
B. U n s t r u c t u r e d Linear P e r t u r b a t i o n s Let us start with bounds given for unstructured linear perturbations in [211. Corollary 2. System (5) with linear perturbation, where Dk(w) is a matrix whose elements are second order white noise sequences, is m.s.e.s,
and
a.s.e.s, if the following holds" sup AI(E{DkrDk}) < max {]t12,]t22,]t32,]t42}
(26)
k
where Itl,tt2,tta, and ~4 are respectively given by (19),(22),(23), and (25). The above results directly follow from Theorems 1-3 and Corollary 1. Let us continue with a new result for the same class of perturbations but this time assume that the perturbation is weakly stationary and t h a t the available bound on Dk(w) is on its second moment
E{DD T} < M
(27)
where M -- M T>_ O. The subscript is dropped based on the weak stationarity property.
STABILITYOF RANDOMLYPERTURBEDSYSTEMS
99
Theorem 4. The perturbed system (5) under all perturbations given by (27) remains m.s.e.s, and a.s.e.s, if p(A | A + a-lst[M]stT[I]) < (1 + a) -1
(28)
for some a > 0. Proof. Consider X = x T> 0 that satisfies X = E{(A + D ) X ( A + D)r} + V
(29)
for some V = VT > 0. The existence of such an X is necessary and sufficient for the m.s.e.s, and sufficient for the a.s,e.s, of the perturbed system by the results in section II. It follows that X = AXA T + AXD T + DXA T + DXD T + V
<(1+ a)AXA T + a-IDxDT + DXD T + V __<(1 + c~)AXA T + (I + a - 1 ) D X D T + V <(1 + a ) A X A T + (1 + a - 1 ) t r [ X ] M + V where the last step uses (27). Assume t h a t there exists an Xm -- Xm T > 0 that satisfies X,~ = (1 + a ) A X m A T + (1 + a-1)tr[X,~]M + V
(a0)
which is true if (28) holds because V > 0 [22]. This result is derived based on the use of Kronecker matrix and stack operation properties. Since from the above development, it follows that Xm >__E f (A + D ) X m ( A + D)T} + V
or equivalently Xm = E { ( A + D ) X m ( A + D) T} + V + Vm
for some Vm = VmT >__0, A + Dk(w) is m.s.e.s. The a.s.e.s, follows for this class of systems as mentioned in section II.
1O0
E D W I N E N G I N YAZ
C. S t r u c t u r e d L i n e a r P e r t u r b a t i o n s Consider the perturbation model (6) where dkj (w) are zero-mean and mutually uncorrelated random sequences. Equation (16) results in M
P(A[2] + Z
vjDj[2]) <
1, vj -- EI(dkJ) 2}
(32)
3"=1
as the necessary and sufficient condition for m.s.e.s, and sufficient condition for a.s.e.s. The following results that are given in [23] are derived by reformulating the stochastic stability problem through the use of (32) as a deterministic stability problem and then using the existing results for unknown but bounded parameter robustness results to find upper bounds on different norms of the vector of noise variances.
The four theorems
given below use deterministic Lyapunov theorems for the nominal part A [21 where
p(A)
< (1 + c~)-1/2 and highly structured correlated perturbations
E M 1 viDj [2]. Theorem 5. The perturbed system (5) with perturbation (6) where (18) is valid and d~1 (w) is a zero-mean and mutually uncorrelated random sequence is m.s.e.s, and a.s.e.s, if any one of the following holds 11 viii < [max [1 vii ~ <
ai2(Di)]-lf(a,A, Q,P)
ai-l (De)f (a, A, Q,P)
(33) (34)
tn
II vll~ < ax-a(~ IDfil)f (,~, A, Q, P)
(35)
/=1
where
. = [~, ~,..., ~mlr Dc = [D1 [2], D2 [2], ..., Dm [2]] and
f(a,A, Q,P) = - a l 2 ( A )
(36)
+ {a14(A) + al-~(P)(1 + a) -2
x [(2 + ,~),~o,,,(P) + o.,(Q)])~/~
(37)
STABILITY OF RANDOMLY PERTURBED SYSTEMS
101
with P being found as the unique positive definite solution to (1 +
c~)2A[2lrpA [21- P = - Q
(a8)
for some Q E R "2 such that Q = Q r > 0. Proof. Consider the equivalent deterministic stability robustness problem posed in equation (32). We use the Lyapunov function candidate 14. = xkrPxk where P is given by (38) and conduct a deterministic Lyapunov analysis. For any Q > 0, there is a unique P > 0 that solves (38) because p2(A[21) = p2(A) < ( l + c 0 -1 by property (18).Equation (38) is used together with the singular value inequalities m
m
al(y~ viDi [21) < ~ vial(Di [21) -<11 r II1 [max a12(Di)], i=1
i=1
m
viDi N) -- al([D1, ..., Dml[vlI,
al(~
..., vmI] r)
i=1 m
_ O'l(De)(~
vi2) 1/2 -- al(De)II v I1=
i=1 m GI(~ i=1
m
m
viPi) ~ ]miax viiGl(~-~ IPi[) -11 v I!oo G I ( ~ i=1
IPil)
i=1
to obtain a sufficient condition guaranteeing a negative definite difference for 14.. Theorem 6. The results of theorem 5 hold if any one of the following conditions holds:
II viii
< [m.ax a12(DOl-lg(o6
II vll:
l
<
Q, P)
al-l(D~)g(c~, Q,P)
(39)
(40)
m II Vllco `Q O ' l - l ( X ~ i--1
ID~[211)g(0~,Q,P)
(41)
102
EDWIN ENGIN YAZ
where g(a,
Q,P) = (1 + a)-l[a,~(Q)a(2 + a)/al(P)] V2
(42)
These results and the ones in the following theorem are obtained similar to the ones above by using deterministic Lyapunov theorems and are detailed in [23]. Theorem 7. The results of theorem 5 are valid if any one of the following conditions holds:
II .11~ < [max al(Pq)]-l/2h(a, Q)
(43)
II ",.,11~< [al(Pe)]-l/2h( c~, Q)
(44)
m
II vll~ < [Ol(y:~ IP,r /,3=1
Q)
(45)
where
h(~, Q) = (1 + ~)-~[o~(Q)~(2 + ~)1~/~
(46)
Pq = Di[2]rpDj [2], 1 < i, j _< m, and Pc = [PI~, P12, ..., Pq, ...,Prom] (47) Theorem 8. The results of theorem 5 are true if al(A) < 1 and one of the following holds: II vii1 < [max l
al2(Di)]-lJ(A)
II vii2 < al -I(De)j(A)
(48) (49)
m
II vlloo < a l - l ( E IDi[2]l)J(A)
(50)
i=-1
where
j(A) = 1 - a12(A) The above results are obtained by the use of ~ =1] xk Lyapunov function.
(51)
I!~ as the
STABILITY OF RANDOMLY PERTURBED SYSTEMS
103
We now present a result that can be found in [23] by using GronwallBellman Lemma: Theorem 9. The results of theorem 5 are true if A is diagonalizable and one of the following holds: l] vii1 < [m.ax a12(Di)]-lp(o~, 2)
II ~II~ <
al-l(De)P( a,
T)
(52)
(s3)
m
il ~LI~ < o ~ - ~ ( ~ IVPl[);(~, T)
(54)
i--1
where
p(~, T) = ~/[(~ + ~)o~(7)o~(T-~)]
(55)
with T being the similarity transformation that diagonalizes A. The next set of bounds makes use of frequency domain results in [3]" Theorem 10. The results of theorem 5 are valid if one of the following holds:
II vll~ < [max a12(D,)]-lq(A)
(56)
ii vl[2 < al-l(D~)q(A)
(57)
m
11 v[[o~ < a~-~(~--~
[D,[ei[)q(A)
(58)
i:1
where
q(A) =
[ sup
al(J~
A[2])-1] -1
(59)
O< O < ~r
Finally, the following stochastic robustness theorem makes use of the Kronecker product based deterministic perturbation bounds in [3]" Theorem 11. The results of theorem 5 are true if one of the following holds"
II ~1[~ < [m~x a12(Di)]-lr(A)
(60)
104
EDWIN ENGIN YAZ
II
< al-X(Dc)r(A)
(61)
fn
ID~[2]I)r(A)
II vlloo <
(62)
i=1
where
r(A) = min { a ~ ( A [2] - I), an~(A [2] 4- I ) , s ( A ) } and
s(A) = -a12(A) + [a14(A) + an,_x(A [4] - 1)11/2 IV. THE CASE OF COLORED
PERTURBATIONS
Let us consider the randomly perturbed system
xk+x = (A 4- dkD)xk
(63)
where A and D are constant matrices and p(A) < 1. We assume either for simplicity of the resulting analysis or because of erroneous modelling that dk is a scalar, zero mean, white, stationary noise and try to find a bound on its second moment to guarantee stability. If the whiteness assumption is not true but the Markov (colored noise or time-wise correlated) model for dk is given by dk+l ---- 4~dk 4- ~k
(64)
where I~l < 1 (see [24] for this assumption) and ~k is zero mean, stationary, and white, does our earlier bound on the second moment of dk based on our erroneous white noise assumption still guarantee stability of the system (63)? Based on our whiteness assumption, the existence of an X > 0 such that
X = A X A T 4- d2"-DXD r 4- V
(65)
for some V > 0, is necessary and sufficient for the m.s.e.s, and sufficient for a.s.e.s, of system (63). So, we have
p(A | A + d2D | D) < 1
(66)
STABILITY OF RANDOMLY PERTURBED SYSTEMS
105
as our stability condition. However, when dk is given by (64), then the steady state second moment (if it exists) expression for (63) yields
Y - - A YA T + ~loo(A YD r + D YA T) 4- doo2D Y D T
(67)
where doo - limk-~ooE{dk} - 0 due to the stability condition I~l < 1. From (64), we also have k-1
dk2 -~- ~2k'd024. E
(~)k-i-l~rr
(68)
' 1 _.~k dk2 -- ~r'2kd024- 1 - ~2 crr 2
(69)
/=0
where ar 2 is the variance of Ck or
for k E A/'. If we consider the limiting case using the monotonicity of the right side in the above expression, we obtain
doo 2 = lira E { d k 2 } . - - (1 - r 1 6 2 k--*oo
(70)
in (69). Therefore, comparing with (66), even if the noise is not white but has the model (64), the m.s.e.s, and a.s.e.s, of the system (63) will be preserved provided that d2 assumed for the white noise is such that > (1 -
or
in which case, X > Y will be obtained. In other words, in the case of an over-estimation of the steady state second moment of the colored noise by the variance of the erroneously assumed white noise, the condition (66) will be sufficient to preserve stochastic stability in the senses defined above.
106
EDWIN ENGIN YAZ
V. J U M P
MARKOV
PERTURBATIONS
Let equation (18) hold for some c~ > 0. This stability margin for the nominal system will allow us to accomodate perturbations of the form (4). Consider the equations N
(1 + a ) A r ( E
(71)
PJTrij)A- p i = Qi
j=l
for some Qi > 0, i = 1, ...,N. We will show that the stability margin of A (18) is sufficient to guarantee the existence of pi, i = 1, ...,N that satisfy (71) for some given H matrix. Let us observe that (71) can be obtained if a recursion like (15) converges to steady state values. The necessary and sufficient condition given in (17) for the convergence yields
p(
/
7ql(l+a)A r|
T
9
...
7rlu(l+a)A v|
...
7rNl(1 -+- ol)AT| A T
"'"
9 7rNN(1
+ ol)A T | AT
/
)<
i
(72)
which can be rewritten as
p(II | [A T | A 7]) < (1 + a ) - i from the definition of the Kronecker product.
(73)
But the spectral set of a
Kronecker product is composed of the products of eigenvalues of matrices in the product and the spectrum of A is equal to that of A T, so (73) is equivalent to
(m.ax [Ai(H) l) (max [/,j(A)[)(rn~x ],,k(A)]) < (1 + a)-i
(74)
where A,lt, and u are the eigenvalues of their matrix arguments. We know that for the stochastic (or Markov) matrix H, we have maxi I A i ( H ) I - 1 [25], so (18) is sufficient for (74) and therefore (72) to hold.
STABILITY OF RANDOMLY PERTURBED SYSTEMS
107
When the convergence to p i are satisfied, uniqueness of {p1, ...,pN} can easily be shown by assuming that there is another solution set
{Pa l, ..., PAN}, taking the difference of matrices in (71), and showing that p i = pa i, i = 1, ...,N, by using the previously presented stability arguments.
It is also easy to see that since Qi > 0, pi > 0 is true for all
i = 1, ..., N. So, we have proved the following result. Lemma 1. Let inequality (18) hold for some a > 0. Then, for given Qi > 0 and H, there exist matrices p i > 0 i - - 1, ..., N that satisfy (71). Consider the system (1) with jump nonlinear perturbation satisfying (4). We can state the following results: Theorem 12. Let equation (18) hold, then system (1) perturbed in the way defined by (4) remains m.s.e.s, and a.s.e.s, if N It i ,( ltl i -- [o~,,~n(Qi)(1 .+. o / ) - 1 ~ 1 - 1 ( E PJ 7rij)] 1/2 j=l
(75)
for i = 1, ...,N and where Qi and p i are given by (71). The proof of this result as well as the following two use lemma 1 together with techniques similar to the ones employed in finding perturbation bounds for white noise perturbations, so will not be repeated. Theorem 13. Let equation (18) hold. Then system ( 1 ) p e r t u r b e d in the sense of (4) remains m.s.e.s, and a.s.e.s, if
Iti < ,a2 i = - a l ( A ) + {a~2(A)
+ [a,\n(~ P3~O) + A.(Q')](1 + c~)-t,~(~ PJ 7r,j)} ~/2 j=l j=l
(76)
for i - - 1,...,N. We have the following result for unstructured linear perturbations given by (5):
108
E D W I N ENGIN YAZ
Corollary 3. Let (18) hold.
Then system (5) where Dk(w) is a matrix
random sequence with elements depending on states of a finite Markov chain is m.s.e.s, and a.s.e.s, if
a,(D,) < max {ltli,lt2 i}
(77)
for i = 1, ...,N. The proof of this result directly follows from the proofs of Theorems 12 and 13 and the use of (5) in (4). Another result for this class of perturbations is as follows" Theorem 14. Let (18) hold. The results of corollary 3 follow if
p{II T | A | A + a-lIIa | (st(I)stW(I))} < (1 + a) -1 where 7rllal2(Sl) H~
:'-"
9
...
(78)
7rNlal2(DN)I
o~
~
...
Proof.
From (15) and (17), it follows that the necessary and sufficient
condition for the m.s.e.s, and a.s.e.s, is the existence of X i > 0, 1 _< i _< N such that N
XJ = E (A + D,)riiXi(A + Di)T + Vj
i--1
for some ~ > 0. To find a sufficient condition for these types of stability in terms of bounds on the perturbations, we consider N
XJ = E (A + Di)rijX'(A + Di)T + Vj
i=l
N
_< (1 + a)A E i=1 N
< (1 + c~)AE /-=1
N 7rijXiAT + (1 + a -1) E DirqXiDiT i=1 N
7rijXiA r + (1 + ~- 1) E a12(D')Trijtr[Zi] i=1
STABILITY OF RANDOMLY PERTURBED SYSTEMS
So, if there exist X j
109
> 0, 1 < j < N, such that N
X d = (1 + c~)A~
N
7rijXiAr + (1 +
a12(Di)riitr[Xi]I
c~-1) ~
i=1
/=1
then this would be sufficient to guarantee stochastic stability in the senses mentioned above. The rest of the proof uses Kronecker product and stack operator properties to describe the stability criterion similar to equations (17a) and (17b). Structured linear perturbations of the type (6) can be treated similarly as follows: Corollary 4. Let (18) hold. The perturbed system (6) remains m.s.e.s, and a.s.e.s, if
Idol < An(Q~)(1 §
N o~-I)-I)~I-I(DT E
7rijPj)
j=l
for all 1 _< i _< N when
Qi and pi
are given by (71).
Proof. The proof follows from that of Theorem 12 by using the perturbation structure in (6). Corollary 5. Let (18) hold. Then the results of Corollary 4 are true if N
N
N
[di[ < [2al(DTE 7rijPJD)]-I{-al(ATE ~rijPJD+ DT E 7rijPJA) j=l N
j =1 N
+ [a12(ATE 7rijPJO+ or E 7rijPJA)+ 4 a l ( D r j=l
j=l
j =1 N
E
7rijPJD)
j=l
N
x An(Qi +
AT E 7rijPJA)]t/2} j=l
for 1 < i _< N with
Qi and pi
in(71).
Proof. The proof follows from that of Theorem (13) by taking the structure of the perturbations into account.
110
EDWIN ENGIN YAZ
VI. WHITE
AND JUMP
MARKOV
PERTURBATIONS
TOGETHER
Let us now consider the case where both types of perturbations exist in the system. This can happen e.g. in the following form [20],[45]" m
Xk+l
: (A -1-E dkl(W)D'i)xk
(79)
l=1
where
dkZ(w)
is a standard (zero mean and unit variance) scalar white
noise sequence independent of the elements of
Dl i,
i = 1, ..., N which are
dependent on a Markov chain with N states. Both random sequences are assumed to be independent of x0. For systems like (79), the second moment evolution can be written as N
m
N
(80)
SJk+l : A E pikTrijAT--[- Z E nlipkiTrijnliT i:1
l=l i:1
We have the following results: Theorem 15.
Let (18) hold.
Then the perturbed system (79) remains
m.s.e.s, and a.s.e.s, if m
N
D, irD, i)
AI(E
< (1 +
a)-lk.(Qi + aPi)kl-l(E 7rijPi)
l=l
(81)
j=l
for 1 _< i <__N where p i and
Qi are
given by (71).
Proof. Since (18) holds, it follows that there exist p i > 0 that solve (71) for some
Qi > o, 1 <_ i <__N.
Using the Lyapunov function I~. =
xkTpixk,
we obtain N
m
N
1214.'= xkr[Ar Z 7riiPiA + Z D"T(Z 7rijPJ)D'i- P'lxk j=l
I=1
j=l
by substitution from (79). Using (71) in the above equation yields CU i:
(1 -[-
C~)-lxkT[--Qi __ (y.pi [_(1
rn -~- o ~ ) E l=l
a)-lxkr[--.,kn(Qi-t - aP i) N m "4- C~),,~l( Z 7rijPi))~l (E D, ir D, i)lxk
<_ (1 4--[-(1
f=l
l=1
N
Dl i T E 7hjPJDI i] xk j=l
STABILITYOF RANDOMLYPERTURBEDSYSTEMS
111
Therefore, (81) is sufficient to guarantee negative-definiteness of Z314[.i. Theorem 16. Let (18) hold. Perturbed system (79) is m.s.e.s, and a.s.e.s. if
(82)
p { n r | A | A + Ha N (st(I)stT(I)) < 1 where / 7r11~1(E;n=1DllDl 1r)
...
7rN1A1(E/m_-1.DN1DI Nr) I
7rlu Al (F_ff=l D,1DI 1~)
"'"
7rNN,~~( ~ ~ ~ Di N Di N
~)
Proof. The proof is similar to that of Theorem 14 and applying the same linear algebraic techniques to tile steady state version of equation (80). Another possible way in which both types of perturbations may exist is the following [461, [47]"
(8a)
xk+l = (A + Dk ~ + Dkm)xk
where Dk w is a weakly stationary random matrix sequence with zero-mean and white elements independent of Dk r~ t h a t represents the Markov chain dependent parameters in the system. Both perturbations are independent of x0. We have the following results in this case: Theorem 17. Let (18) hold.
Then the perturbed system (83) remains
m.s.e.s, and a.s,e.s, if N
(1 + c~-l)crl(DimTDi m) -t- a,(E{D~rD~'}) < )~n(Qi)a,-l(Z 7rijPj) 3-=1
(84)
for all 1 _< i _< N where pi and Qi are as given by (71). Proof. Using the Lyapunov function
~i= xkTpixk
where pi satisfies (71)
112
EDWIN ENGIN YAZ
for some Qi > O, we obtain
N
s
xkT[E{(A + D ~ + Dim)r E 7rijPJ(A + D ~ + Dim)} - Pilxk j=l N
= ~ q ( A + D, m)~ ~
~
P' (A + D, m)
j=l N
+ E { D ~ r E 7rijPiD w} -
e'lxk
j=l N
N
_<xkr[(1 + ~)AT~_, lrijPJA + (1 + c~-l)D, mr ~ roPJDi m - P' j=l
j=l N
+ al(E{D~rD~})al(~_, 7ri~P3)]xk j=l <--I[ xk II2 {-,Xn(Q i) 4- [(14- ot-1)crl(OimrDi m)
N
+ al(E{D~rD~I)]aI(E 7rijPJ)} ~-=1
(as)
where we use the independence of perturbations and the zero-mean nature of white noise sequence in obtaining the second inequality.
The second
inequality uses Lemma 1 and equation (71) and a basic singular value inequality that is also used in arriving at inequality one. It follows that (85) is sufficient to obtain the result of the theorem. Theorem 18. Let (18) hold. Then the results of Theorem 17 hold if
N
N
j=l
j=l
Gr1
> al(E{D~~
(86)
STABILITY OF RANDOMLY PERTURBED SYSTEMS
113
and N o'I(D/)
N
7rijPJ){-Ul(E 7rqPJA)
< uI-I(E j =1
j =1
N
+ [a12(E
N
nijPiA) + al( E niiPj)
j--1
j--1 N
N
x {An(olATE 7rijA+ Qi) _ j=l
al(E
7rijPJ)al(E{DWrDW})}]t/2}
j=l
(sT) Proof. Starting the same way as in the proof of Theorem 17, we obtain N
s p~i <_XkT[ATE rijpjA j--1 N
N
+ 2al(Dim)al(E r
+ al2(Dim)al(E 7rijPJ)I
j=l
(ss)
j--1 N
+ al(E{owTow})al(E 7rijPJ)I- pi]x k 3"--1
Then (86) and (87) are sufficient to obtain the results. Theorem 19. Let (18) hold. Then the results of Theorem 17 hold if
p(IIT|174174
< ( l + a ) -1 (89)
where
HD -- (~-1
t rlla12(Dlm)st(I)stT(I) . ~IN a 12(O1 re)st (/)stW (i)
... ~Nla12(DNm)St(I)st (I) t ... . " rNN al 2(DN m)st (I) stW(I) (90)
Proof. The proof follows from the steady state conditional second moment
114
E D W I N E N G I N YAZ
equation N
PJ - E { Z
(A + D ~ + Dim)TrijPi(A + D w + Dim) T}
i=1 N
=~
N
(A + Dim)Tropi(A + Dim) T + E{D~ Z
i=1
~opiD ~r}
i=1 N
_< (1 + a)A Z
N
7toplAT + (1 + a -1) Z DimTriJpiDimr
i=1
i=1 N
N
+ E{D~D~} ~ r0tr[P ~] _< (1 + a)A ~ FroplAT i=1
/-=1
N
+ ~((1
+
o~-l)(~12(oim)I -t- E{D~D~r})Trijtr[P i]
i-=l
via the use of basic properties of Kronecker products and stack operators. VII. DISCUSSION
We have considered stability robustness of discrete-time systems with white noise and jump Markov type parametric perturbations. These perturbations are assumed to be nonlinear, unstructured linear and highly structured and correlated linear type. This paper has presented a concise survey of existing results scattered throughout several journals and conference proceedings. It has also contributed some new results especially on the robustness for colored perturbations modelled as white noise and jump Markov type parametric excitations. Numerical comparison of some of the results given here can be found in [21] and [23], however a comprehensive numerical study that includes the new bounds needs to be carried out. Because of the space limitation, we have not been able to cover all related results in this work. For example, there are results derived based on Furstenberg-Kesten theorem [26] which is about the product of ergodic random matrices like [27]~ The works [28] and [29] use Lie algebra together
STABILITY OF RANDOMLY PERTURBED SYSTEMS
with the discrete forms of Gronwall-Bellman lemma.
115
[30] uses Banach
space techniques developed for deterministic stability to derive mean square robustness bounds.
Another result using ergodic assumption for white
perturbations and guaranteeing almost sure stability can be found in [31]. Still, other non-Lyapunov results are in [32]. [33] contains an extension of the stochastic Lyapunov technique.
Finally, there are results on the
stochastic version of the total stability problem: [31], [34]- [37]. Although we have restricted the coverage to the analysis problem in this work, the application of similar results to the robust design of controllers based on the nominal deterministic system description for models with white noise type perturbations has been considered in [38]- [42]. The application of most of the perturbation bounds given in this work requires that the nominal system is exponentially stable with a known stability degree. This can easily be achieved in discrete time e.g. by regional pole placement techniques [43] or by linear quadratic regulator with prescribed degree of stability [44]. For instance, given the perturbed control system xk+l = Axk + Buk + f k(xk, w) where uk is the control vector and f~ satisfies (3), the state feedback controller designed based on the modified quadratic performance index OO
J = ~ (1 + ~ ) ~ ( ~ ~
+ (1 + ~) ~ n ~ )
k=0
for a > 0, (~, R > 0 and the nominal part of the system with f~(xk,w) - O, is uk = K~
= -(R + BTpB)-IBrPAxk
with P -- (1 + o~)ATPA - (1 + @ A T P B ( R + B : r P B ) - I B T P A +
= (i + ~ ) ( A + BK~ rP(A + B K ~ + (i + ~ ) K ~
~+
(91)
116
EDWINENGINYAZ
Assume that ((1 + c~)I/2A, B) is stabilizable, so with ~) > 0, it follows that there exists a unique positive definite solution P to the above Riccati equation (91). Using the Lyapunov function 1~. = xk r P x k , a perturbation bound It that maintains stochastic stability is e.g. given by Theorem 1 with P in (91) and Q = ~) + (1 + (~)K~
~ Obviously, other robustness
results can be utilized similarly in this context.
VIII. REFERENCES
1. P.Dorato, Robust Control, IEEE Press: New York (1987). 2. M.Mansour,"Robust stability of interval matrices,"Proc, of 28th CDC, Tampa, FL, pp. 46-51 (1989). 3. A.Weinmann, Uncertain Models and Robust Control, Springer- Verlag: New York (1991). 4. P.S.Sagirow,"The stability of a satellite with parametric excitation by the fluctuations of the geomagnetic field," in Stability of Stochastic Dynamical Systems, Lecture Notes in Mathematics, Springer- Verlag: New York,294, pp. 311-316 (1972). 5. M.Aoki, Optimal Control and System Theory in Dynamic Economic Analysis, Elsevier: New York (1976). 6. C.P.Tsokos and W.J.Padgett, Random Integral Equations with Applications to Life Sciences and Engineering, Academic Press: New York (1974). 7. R.R.Mohler and W.J.Kolodziej," On overview of stochastic bilinear control processes," IEEE Trans.
Syst. Man and Cybern., 10, pp.
913-91s (19s0). 8. R.A.Ibrahim, Parametric Random Vibration, Wiley: New York (1985).
STABILITYOF RANDOMLYPERTURBEDSYSTEMS
117
9. A.J.M.Van Wingerden and W.L.DeKoning," The influence of finite word length on digital optimal control," IEEE Trans. Aurora. Contr, 29, pp. 385-391 (1984). 10. T.J.A.Wagenaar and W.L.DeKoning," Stability and stabilizability of chemical reactors modeled with stochastic parameters," Int. J. Contr.,
49, pp. 33-44 (1989). 11. W.P.Blair and D.D.Sworder," Feedback control of a class of linear discrete-time systems with jump parameters and quadratic cost criteria," Int. J. Contr., 21, pp. 833-841 (1975). 12. A.S.Willsky," A survey of design methods for failure detection in dynamic systems," Automatica, 12, pp.601-611 (1976). 13. R.L.Moose, H.F.Van Landingham, and D.H.McCabe," Modelling and estimation for tracking maneuvering targets,"IEEE Trans.
Aerosp.
Electron. Syst., AES-15, pp. 448-456 (1979). 14. D.D.Sworder and D.S.Chou," A survey of design methods for random parameter systems," Proc. of 24th IEEE CDC, Ft.Lauderdale, FL, pp.894-899 (1985). 15. E.Yaz and X.Niu," Stability robustness of linear discrete-time systems in the presence of uncertainty," Int. J. Contr., 50, pp. 173-182 (1989). 16. E.Yaz," Control of randomly varying systems with prescribed degree of stability," IEEE Trans. Aurora. Contr., 33, pp. 407-410 (1988). 17. Y.Ji, H.J.Chizek, X.Feng, and K.A.Loparo," Stability and control of discrete-time jump linear systems," Control - Theory and Advanced Tech., 7, pp. 247-270 (1991). 18. T.Morozan," Stabilization of some stochastic discrete-time control systems," Stochastic Anal. Appl., 1, pp. 89-116 (1983).
118
EDWIN ENGIN YAZ
19. T.Morozan," Optimal stationary control for dynamic systems with Markov perturbations," IBID, pp. 299-325 (1983). 20. E.Yaz, 'v A generalization of the uncertainty threshold principle," IEEE Trans. Autom. Contr., 35, pp. 942-944 (1990). 21. E.Yaz," Robustness of discrete-time systems for unstructured stochastic perturbations," IEEE Trans.
Autom.
Contr., 36, pp.
867-869
(1991). 22. E.Yaz," Linear state estimators for nonlinear stochastic systems with noisy non-linear observations," Int.
J. Contr., 48, pp.
2465-2475
(1988). 23. E.Yaz and X.Niu," New robustness bounds for discrete systems with random perturbations," IEEE Trans. Autom. Contr., 38, pp.18661870 (1993). 24. K.J.Astrom, Introduction to Stochastic Control Theory, Academic Press: New York (1970). 25. R.Bellman, Introduction to Matrix Analysis, McGraw-Hill: New York, 2nd Ed. (1970). 26. H.Purstenberg and H.Kesten," Products of random matrices," Ann. Math. Statis., 31, pp. 457-469 (1960). 27. D.H.Shi and F.Kozin," On almost sure convergence of adaptive algorithms," IEEE Trans. Autom. Contr., 31, pp. 471-474 (1986). 28. F.Ma and T.K.Caughey," On the stability of linear and nonlinear stochastic transformations," Int. J. Contr., 34, pp. 501-511 (1981). 29. F.Ma and T.K.Caughey," Mean stability of stochastic difference systems," Int. J. Nonlinear Mechanics, 17, pp. 69-84 (1982). 30. C.S.Kubrusly," On discrete stochastic bilinear systems stability," J.
STABILITYOF RANDOMLYPERTURBEDSYSTEMS
119
Math. Analy. Applic.s, 113, pp. 36-58 (1986). 31. T.Morozan," On the stability of stochastic discrete systems," in Control Theory and Topics in Functional Analysis, Vienna: IAEA, 3, pp.
224-254 (1976) 32. X.Yangand G.Miminis,"Stabi]ityof discrete deterministic and stochastic nonlinear systems," J. Math. Analy. Applic.s, 168, pp. 225-237 (1992). 33. l~.R.Bitmead and B.D.O.Anderson," Lyapunov techniques for the exponential stability of linear difference equation with random coefficients," IEEE Trans. Autom. Contr., 25, pp. 782-787 (1980). 34. M.K.P.Mishra and A.K.Mahalanabis,"On the stability of discrete nonlinear feedback systems with state-dependent noise," Int. J. Systems Sci., 6, pp. 479-490 (1975). 35. M.K.Mishraand A.K.Mahalanabis," On the stability of nonlinear feedback systems with control-dependent noise," IBID, pp. 945-949 (1975). 36. P.V.Pakshin," Stability of discrete systems with random structure under steadily acting disturbances," Automation Rem. Contr., 44, pp. 747-756 (1983). 37. E.Yaz and N.Yildizbayrak," Robustness of feedback - stabilized systems in the presence of nonlinear and random perturbations," Int. J. Contr., 41, pp. 345-353 (1985). 38. E.Yaz," Certainty equivalent control of stochastic systems: stabilizing property," IEEE Trans. Aurora. Contr., 31, pp. 178-180 (1986). 39. E.Yaz," Further results on the stabilizing property of certainty equivalent controllers," IBID, pp. 586-587 (1986). 40. E.Yaz," Sliding horizon optima] and certainty equivalent controllers
120
EDWINENGINYAZ for stabilizing stochastic-parameter systems," Optimal Contr.: Applic. Methods, 8, pp. 327-337 (1987).
41. E.Yaz,"Feedback controllers for stochastic-parameter systems: relations among various stabilizability conditions,"IBID, 9, pp. 325-332
(1988). 42. E.Yaz," Optimal stochastic control for performance - and stabilityrobustness," IEEE Trans. Autom. Contr., 38, pp. 757-760 (1993). 43. K.Furuta and S.B.Kim," Pole assignment in a specified disk," IEEE Trans. Autom. Contr., 32, pp. 423-427 (1987). 44. B.D.O. Anderson and J.B.Moore, Optimal Control, Prentice Hall: Englewood Cliffs, N.J. (1990). 45. M.Mariton, Jump Linear Systems in Automatic Control, Marcel Dekker: New York, 1990. 46. T.Morozan,"Dynarnic programming for some discrete-time control systems with random perturbations," Rev. Roum. Math. Pures et Appl., 25, pp.1065-1083 (1980). 47. E.Yaz," Moving horizon control of systems with independent and jump stochastic parameters," Preprints of IFAC World Congress,3, pp. 5358, Tallinn, Estonia (1990).
Observer Design for D i s c r e t e - T i m e Stochastic Parameter Systems Edwin Engin Yaz
Electrical Engineering Department University of Arkansas Fayetteville,AR 72701
I. I N T R O D U C T I O N In this work,we consider linear observer design to reconstruct the state vector of discrete-time systems with both white(time-wise uncorrelated)and jump-Markov type parameters based on noisy observations. Such stochastic parameter systems have been used in many applications.Systems with white parameters have e.g. been used in satellite attitude control[l], chemical reactor control[2], macroeconomics[3], population dynamics[4], time-sharing and roundoff error modelling in computer control[5], and recently in robustification of controllers[6]-[8]. Other uses include modelling intermittentmeasurement problem where the signal content of the measurement may be absent at times such as in radar tracking [9], [10], and in random amplitude modulation problems[l l] On the other hand, systems with jump-Markov parameters have been used in e.g. modelling large changes in operational conditions of economic models[12] and power systems[13], target tracking involving sudden ~vasive maneuvering[14], steam temperature control of solar receivers[15], failure in manufacturing systems[16], and modelling discrete event systems[17]. CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
121
122
EDWIN ENGINYAZ Other works on state estimation of discrete-time systems with white
parameters include [18]-[28] where [18]-[20] approach the problem from the minimum variance optimality viewpoint. However, [21]-[24] are on suboptimal observer design with guaranteed stochastic convergence. Robustness results are presented in [25]- [27] and [28] contains a fixed order observer design procedure for this class of systems.The use of jump-Markov parameter models in estimation dates back to [29].Later work in this area includes [30]-[36] and, currently, the research activity seems to intensify due probably to the potential application possibilities realized by researchers [37] [391. It is assumed throughout this work that the exact knowledge of the current operation mode of the system or the state of the Markov chain is available to avoid exponential increase in memory requirements of the observer [32]. So, based on this information and the noisy output measurements, full and reduced order observers which reconstruct either all or a subset of the internal states are designed. This work starts out with a discussion of the notation and the typical mathematical structure used in this framework.It continues with a characterization of all full order linear unbiased observers and specification of the optimal one in the minimal variance sense among these.To be able to establish the existence of the steady state and show the convergence of the resulting observer, stochastic system theoretic concepts like mean square (m.s.) stability and m.s. detectability are discussed.
Then the estimation counterparts of the convergence
theorems proposed previously for control Riccati equations in this context are presented.
It is then assumed that the steady state is reached and
several biased and unbiased reduced order observer designs are characterized.Robustness results for uncertain covariances follow.Finally, we discuss what can be done for stochastic models that include correlated a n d / o r semi-
STOCHASTICPARAMETERSYSTEMS
123
Markov parameters. II. N O T A T I O N
AND MATHEMATICAL
PRELIMINARIES
x E R ~ will denote an element of the real Euclidean space of dimension n with 2-norm II x II= (xTx) 1/2 and A e R nxm will be an element of the set of n x m dimensional matrices with real elements. A -1, A t, vec(A), p(A), A | B, and AU2 will respectively represent the inverse of A, the MoorePenrose pseudo inverse of A, vectorized A (A operated on by the stacking operator), the spectral radius, the Kronecker product of A and B matrices, and the square root of the A matrix [40]. E{.} and E { . / . } "will denote expectation and conditional expectation operators, respectively. Extensive use will be made in this work of monotonic transformations on finite-dimensional Banach spaces.
All of the following results
involving such finite dimensional operators can be found e.g.
in [41] -
[42]. X E Mn L will denote an element in the Banach space of all symmetric matrix sequences X = { X 1 , X ~ , . . . , X L } , Xz E R n•
1 < l << L,
with the norm II X ]]= maxl
A E Mnm L will represent a matrix
sequence {A1,A2, ...,An} where Az E R n•
Mn L+ will be the set of all
elements X+ = {X1, X2,..., XL} of Mn n with Xt >_ 0 (nonnegative definite), 1 < l < L, which form a partial ordering for the elements of Mn L as X+ 1 >_ X+ 2 meaning (X+ 1 - X + 2) E Mn L+ 9 Accordingly, for X+ C Mn L+, X+ > 0 means for all 1 < l _< L that Xt > 0. In the case where X E Mn L+, the spectral radius p ( X ) =]] X II. Let s space of all linear transformations r [[ ~4 II. ~4 E s (~4~(.) = A(r
Mnrn L} denote the Banach
9Mnm L ---* Mnm L with supremum norm
Mn L} is called stable if for some k > 1, II r
]]< 1
or equivalently the spectral radius of .A, p(jt) < 1.
For such linear transformations,-we have p(A) = limk_~ [I Ak ]]l/k -
124
EDWIN ENGIN YAZ
A transformation (not necessarily linear) A " Mn L ~ Mn L is called monotonic if for X 1 , X 2 C M~ L, X 1 >_ X 2 implies A ( X 1) >__jr(X2). For a monotonic .4 "
M n L+ ---* M n L+ , r k, k _
1 is
monotonic and ~4k(X+) E
M n L+ 9
If .4" Mn n --~ M~ L is both linear and monotonic, then i)]l j r ( I ) II=ll j t II where I = {In, In,...,In} E Mn L+ where In is the n-dimensional identity matrix and ii)~4 k, k >_ 1 is linear, monotonic and ,4k(X+) E Mn L+ . Jt is stable if and only if limk_.oo Ak(X+) = 0 = {0n, On, ..., On} E M~ L where On is n x n matrix of zeros for all X+ > 0. III. FULL ORDER
OBSERVATION
Consider the discrete time system with j u m p Markov and white random parameters ql
xk+, = A(rk)xk + ~
Bi(rk)xka[. + F(r~)vk
(1)
+
(2)
i=1
and the measurement equation q2
=
+ j=l
where xk E R n" is the state vector to be estimated based on the noise corrupted measurements y~ C Rny and the knowledge of the state (called operation mode here) of the Markov chain {rk " k >_ 0} on which the parameter matrices A , B i , F , C , D o., and G, 1 < i < ql, 1 < j < q2 are all dependent.
We assume that there are only a finite "L" number of
operation modes with probabilities 7rl,k -- prob{rk = l}, limiting values {Trl = limk-.oo prob{rk = l} > 0, 1 < l < L} (which are assumed to exist), and transition probabilities {rZm, 1 < l, m < L}. hi., [3~ are scalar and vk, wk are vector random sequences with white, second-order, stationary i ~, elements with zero means and respective (co)variances ha,
Zv, and L'w.
STOCHASTIC PARAMETER SYSTEMS
125
We assume for simplicity of presentation that rk, c~[., fl~, vk, and wk for all k >__ 0 and x0 are all mutually independent, and
l
GtSwGt T
> 0 for all
l < i < q l , l<j<_q2.
Let us first assume that at any time instant k > 0, both the measurement Yk and the operation mode rk = l is available and parametrize all finite time linear full-order unbiased estimators for system (1) and measurement equations (2). The general linear estimator
(3)
~k+~ = Sl,k~k + Kl, kYk
gives rise to the estimation error ek = xk - xk with dynamics qt
q2
ek+l = (Al--Kl,kCl)xk--,.ql,kYckA-E B[xkoz~TFlvk--Kl, k ( E l~xk~ TGlwk) i=1
j=l
(4) where At = A ( ~ = t) is used to simplify the notation. Also, from now on, we will use Al,k+l for A(rk+l ---l). Taking the expected value of both sides and imposing the condition that the estimator be unbiased, we obtain
Xk+l =
Alxk "4"Kl,k(Yk -- Clxk)
(5)
where the initial state estimate is to be set equal to the expected value of x0. This yields the estimation error dynamics qL
ek+l =
q2
(Al--Kl,k Cl)ek+ E Ulixkolki+Flvk--Kl,k(E DtJxk/3j +atwk) i=1
(6)
j=l
with E{ek} = 0 for k >__O. Let us now look at the estimation error covari-
126
EDWIN ENGIN YAZ
ance evolution. The formula P a l = E{eaearlrk = i} gives L
7rml[(Am- Kin,aCre)Pare(Am- K~aCm) T
Pa+ll = E
nv=l qt
+ 7rm'aFm~vFmT
+ E BmiXkmBmiT~ i---1 q2
(7a)
+ K,~a(E DjXamDmYra~'+ 7rm,kGmZ~oGm~)K,,~a y=l
for 1 < l < L where X a t = E { x a x a TIrk = l} are given by L
qt
L
rr~ iX am,-, 7r~AmXamAmT+E E 7rml[Dm IJmiTO'ai--Trm, -t- aFm~vFml~
Xk+ll= E m=l
i=1 m=l
(Tb)
Rearranging equation (7), we complete the squares to obtain L
~ 7ro~AmpkmA~v
Pk+l z -
m=l qt --
L
E E
7rrnl[BmiXkmBmiTorc~i-}-7rm'kFm~vFml)
i=1 m=l L
= --
E 7r~Km'kCmpkmAmT
m=l L L -- E 7rrrdAmpkmCmVKm'kT+ E 7rmIKm'k[CmpkmcmT m=l q2
(8)
m=l
+ E DrnJXamDrnJraflJ+ 7rm'aGmrwGmT]Km'ar j=l L
= ~
~.~(nm ~ - n~~
X~)(nm ~ - n.,~ ~ ~
m=l
-Km,
a~
Xa)Km,a~
where
Km,k~ = AmPk~CmTS-I(P~, Xa)
(9)
STOCHASTIC PARAMETER SYSTEMS
127
q2 S(Pk,Xk) = CmPkmCm r + ~
DjXkmDjra/
+ 7rm,kGmE~G~ r (10)
j=l
for all 1 _< 1 <__L. Rewriting (8) with substitution from (9), yields L
gk+ll-- E 7rmtAmPt~mAmr m=l qt
-- E
L
E
7rrnl[BmiXkmBmir(:raint- 7rm,kFm~vFm~
i=1 m = l
(11)
L
+ E 7r~AmpkmCmr~-l(pk'xk)CmPkmAmr m=l L = E
~.~(K.~k - K.~k~
- K.~k~ T
m=l Looking at the right hand side of (11), since the left hand side must also have the same properties, we must have L
Pk+l t = ~
u.~[AmpkmAm T
nv=l
- AmPkmCmr~-l(pk,xk)CmpkmAm I)
L
+E m--1
(12)
q~
Bm'X Bm'%o'+
+
m-=l
:= ~,k(P0t, Lk)] where Lm, k E R '~*• ~, 1 < l _< L. Theorem
1. Equation (12) with arbitrary Lm,~r C R n*•
characterizes
all possible conditional estimation error covariances Pk z = E{e~r162
= l}
that are achievable by a linear full-order unbiased estimator of the form (5) for a system model (1) and measurement model (2). Let us now consider equations (11) and (12) together and rewrite them as
Ln~kLn~k T = (gm, k - gn~k~
k - gm,~r176 T
(13)
128
E D W I N ENGIN YAZ
By the linear algebra result in [43], there exist orthogonal U~k such that L,~k U,~k = (K,,~k - K.~k ~ ~1/2 (Pk, Xk)
(14)
K,..k = L,n, kU.,,kS-1/2(pk,xk) +Km, k~
(15)
which yields
which parametrizes all filter gains that assign a given covariance set p l. T h e o r e m 2. The set of all gains which can be used in estimator (5) for model (1)-(2), to result in a particular estimation error covariance set Pk l that satisfies (12) is given by (15) where K.~k ~ ~.~k, and L.~k are specified respectively in (9), (10), and (12).
U.~k is an orthogonal but otherwise
arbitrary sequence. One can see that if one lets K.~k -- K.~k ~ in (1), then Lm, k ---- 0 for all 1 _ m _< L, k >_ 0, resulting in the minimal estimation error covariance given by (12). C o r o l l a r y 1. Equation (12) with L.~k -- 0 for all 1 <_ m _<_L, k >_ 0 gives the generalized Riccati equation for minimal estimation error covariance Pk* (in the sense that it satisfies (Pk - Pk*) E Mn L+ for all Pk E Mn L+ solving (12) for some L.~k) that results from using the operation mode dependent optimal filter gain (9) in the linear full-order unbiased estimator (5) for the system and measurement (1)-(2). IV. M E A N S Q U A R E
STABILITY
To be able to discuss steady state filtering, we need to consider under what conditions steady state can be reached. Let us start with the evolution of the second moment of the state in (7b) and replace 7r,~k with its limiting value 7rm for large k: L
Xk+l' = E m=l
qt
7r'~[AmX~'~A'~T + E i=1
Bm'XkmBm'T ao~'+ " 7rmFmW~vFmT] (7b)
STOCHASTICPARAMETERSYSTEMS
129
To see the convergence of this recursion to a steady state where Xtr
I=
Xk I = X z, 1 <_ t <_L resulting in L number of generalized algebraic Lyapunov equations, we use some well-known properties of Kronecker product and vector operation, and rewrite (7b) as
Xk+l I / yea
.
Xk+l L
( 7rllA1 o
7rlLA1
9..
7rLLA L
Xk L
(16)
Em=l 7rml + vec
L~ 1 7rmL~rmFmZvFm r kl ) 9
:= ~ vec
+f
Xk L where At = At | At + ~iq~=l Bl i | B~ia,~ i, for 1 < l < L. This difference equation has a steady state solution if and only if p(g') < 1. Since all Xk I are nonnegative definite, so will the limiting values
X t, 1 <_ I <_ L be. To show the uniqueness of {X 1, ...,X L} we can assume another solution { y1 ..., yL}, and take the difference
vec
g' vec X L_
yL
= ~ vec X L_
yL
X L_
yL
If p(O) < 1, we find X t = IA. Summing up, we have obtained the following result: T h e o r e m 3. Thele exists a unique nonnegative definite steady state solution {X 1, ...,X L} to (7b)if and only if p(g') < 1. Note that the condition p(O) < 1 is both necessary and sufficient for m.s. boundedness of the system state in equation (1) for any given
130
EDWIN ENGIN YAZ
r v > O.On the other hand,if L'v - O, the same condition is necessary and sufficient for the m.s. a s y m p t o t i c stability of system (1) [24].
V. M E A N S Q U A R E D E T E C T A B I L I T Y Given the system and the measurement equation xk+i = Alxk
(17)
~. = Gxk We say t h a t (At, CI) is m.s. detectable if and only if there exists a KI, l = 1, ...,L such t h a t the full order unbiased estimator (Luenberger observer)
xk+l = AlYc,k 4- Kl(Y~ -
Clxk)
(is)
gives rise to an error ek = x ~ - ~k evolution
ek+l = (At - K t G ) e k
(19)
which is m.s. asymptotically stable. We now derive a necessary and sufficient condition for assessing the m.s. detectability of system (17). Consider the transformation ~K "
MnL+ ---, MnL+ L
GK~(X) = Z
7r,,~(Am - g m c m ) x ' ~ ( A m - KmCm)T
(20)
m=l
with X = { X 1 , X 2 , . . . , X L } .
Note t h a t GK k is linear and monotonic for
k >__ 1 where {~Kk (X) := {]K [gK k- I ( X ) ] 9 W e have E{ekckrlr~ = l} = GK, k(r0)
(21)
with Z'0 = {Z~0,1, ..., G0,L} where r0,t = E{cocoTIro = l}. Obviously, (At, Ct) is mean square detectable if and only if there exist KI, l = 1, ..., L such t h a t GK is a stable transformation or
Pmin := minKp(~K) < 1
(22)
SIOCHASTIC
PARAMETER SYSTEMS
131
Note that P min always exists since p is a continuous convex function of K . L e m m a 1.
P~,r~- ~Lm Ii A0~(I)il 1/~
(23)
where Ao" Mn L+ ~ Mn L+ is L
.Ao,t(X) = E
A'~XmTr"a[I - C'~T( CmXmC'~T) t C"~Xm]A'~T
(24)
rn--1
Proof. Since L
aKz(X) = Ao,l(X) + E
7r,,~(K~- Ko, m)CmX~C~T(K,
,n - Ko,~)T
(25)
m=l
with
Ko, m = Ko,m(X) := A,~xmc~T(C,~X'~Cmr) ~
(26)
and ~K is monotonic, we have
Ao~(I) <_aKk(I)
(27)
This yields lim
k-~
Ii Aok(x) II~/~ <- -
lim
k-~
II ~z<~(I)II v~= p(~K)
(28)
The last equality follows from section II. Now, defining the linear transformation jtz k 9Mn L+ ~ Mn L+ as
A~(x) = ~Ko(~o~-,(~))(~
(29)
it follows from the definition of p min that
(30) for k >_ 1. Continuing in an inductive manner, we obtain
p(Ak) >_ p,,,~, k
(31)
132
E D W I N E N G I N YAZ
It can easily be shown that
II ,A0k (I) II---II .Az k (1) II
(32)
II ,mzk(1) I1=11 ,Az k II
(33)
but also from section II,
therefore, from the properties of the spectral radius and (15), it follows that
II Az k II~ p(Az k) ~ p,~, k
(36)
or substituting from (32) and (33) lim
k---*oo
II Ao~(I)II ~/~ >--Pmin
(35)
Finally, combining (28) and (35), we obtain (23).
T h e o r e m 4. (At, CI) is m.s. detectable if and only if lim II ~ k ( I )
k--,oo
IIx/k < 1
(36)
or equivalently lira II .Aok(I) II = 0 C Mn L+
(37)
k-*oo
Proof. (36) simply follows from lemma 1, and the fact that Pmin < 1 is necessary and sufficient for m.s. detectability as discussed before. Since,
(3s)
lim I1,40k(I) [[ - lim pmin k
k-~oo
k--,oo
conditions (36) and (37) are equivalent. It is interesting to specialize the above result to constant parameter systems where L = 1 or there is only one operation mode. In that case,
.Ao(X) - A X A T - A x c T ( c x c T ) t C X A
T
(39)
STOCHASTIC PARAMETER SYSTEMS
133
So, our result specializes to the one in [44] for the stabilizability of constant parameter systems when (A, B) pair is replaced with (A T, cT).
VI. S T E A D Y
STATE CASE
In this section, we present the convergence properties of the generalized Riccati equation (12) with Lt,k = 0 based on the system theoretic properties developed in the previous sections. L e m m a 2. Let p(~P) < 1, then (Al, G) is m.s. detectable. Proof.
If p(~) < 1, then by the results in section IV., there is a set of
second moments X = {X 1, ..., X i } > 0 such that for any FIZ'vFLT > 0, we have L
qt
L
Xl = E 7rmlAmXmAmT"3r-E Z 7rrnl[J~miXmJ~miTo'aint-7rmFm~vFm~ m=l
i=1 m = l
L
>-- E
7rml[AmXmAm T 4- 7rmFm~'vFmI]
m=l So, gg is a stable transformation with K - 0, meaning that (At, 6}) is m.s. detectable. T h e o r e m 5. Let p(~) < 1 and FtS,~Fz T> O, 1 <_ 1 <_L. Then limk-~oo 4~ (0, O) -- P E Mn L+ where P =
o)
and P is the unique solution in Mn L§ Moreover,
zk+l = ( A t - Kl~ is a m.s. asymptotically stable system.
134
EDWIN ENGIN YAZ
Proof. When p(g') < 1, based on theorem 3, we obtain limk-~ooXk t = X t, SO qL
L
qt
li'll E E 7/'m/SmiXk mSmiTorai k.---,oo i=1 nv=l q2 L
lim E
k----*~
E
.__
L
i m iT " E E 7frnISm X g m O'c~' i=1 m=l ,/2 L
(40)
7rtmDmJXkmDmJ :raJ = E E 7r'~DmJXmDmJ raJ
3"=1 m=l
j = l m=l
p(~') < 1 also guarantees the m.s. detectability of (At, Q) by lemma 2. Combining these with Fz~',,FtT
>
0 and GtS~Gt T > 0 and using the dual
results proposed for control Riccati equations for systems with jump Markov parameters [441-[46], the result follows. The following result shows that one does not need the m.s. stability of the original system if there is no white multiplicative noise in the model. C o r o l l a r y 2. Consider the model
Xk+I
--
A(rk)xk + F(rk)vk
(41a)
together with the assumptions concerning the model (1), (2). Then the results of the above theorem follow when the condition p(gr) < 1 is replaced with the m.s. detectability of (At, Ct). Proof. The proof directly follows that of Theorem 4. However, in this case, we do not need the existing dual Riccati convergence theorems [44]-[46]. VII. R E D U C E D
ORDER OBSERVATION
A. P r e l i m i n a r y R e s u l t s First, some auxiliary results which will be used throughout this section are introduced:
STOCHASTIC PARAMETER SYSTEMS Lemma
3. G i v e n
r
135
M n L+ ~ M n L, 1-' E I:t nxm, K E MmpL,
N " Mn L§ --~ Mp L, M " Mp L§ ~ Mp L§ M > 0 where in general, r K, N, and M are functions of X = {X1,X2, ...,XL} ~_ Mn L§ which satisfy L
7rmz(I~KmNm r + NmKmTFT + FKmMmKmrFT)
r = ~
(42)
m=l
for 1 <_ l <_ L. Then, the following are equivalent: i)The solution sequence X C Mr, L+ to equation set (42) is assignable by a g a i n s e q u e n c e K = {K1, ..., K L } E Mmp L.
ii)There exists s E Mnp L such that for 1 <_ t _< L L
r + E
L
7rmlNmMm-tNmr= E
m=l
7r~s163
(43)
m--1
and (I
-
rrt)r
-
rr*) = o
(44)
iii)There exists X E Mn L+ satisfying for all 1 __ l < L, L
r
E
7rmlNm Mm- l Nm T
m=l L
= E
7r~[(I - F F t ) N m + FFtZml]Mm-I[(I - F F t ) N m + FFtZml] T
nv=l
(45)
for some arbitrary Z 1 = {Zi 1, ...,ZL 1} with suitable dimensions. The proof of the above result is given in Appendix A. L e m m a 4. Suppose that an assignable solution sequence X E Mn L§ to (42) exists, then that X can be assigned by any one of the following gain sequences K = {K1, ..., KL } such that Kt = Ft[L, Ut ~-1 _ NzMt-1] + (I - FtF)Zt 2
(46)
136
EDWIN ENGIN YAZ
where /~l is given in (43) as a square root of the left hand side, 2~ is a nonsingular square root of ML > O, Zt 2 is arbitrary, ~ is given by Uz = Uz2blockdiag[I, ~]Ul ar
(47)
and (1 -
rvt)& =
(I - r F t ) N t T - r
u,
&vl 2.
= /~lrLUL3r
(48)
(49)
with the dimension of the identity in (47) being equal to the rank of L'z and u
L~ is arbitrary orthogonal. The proof of this lemma is given in appendix B. L e m m a 5. Let Ct in Lemma 3 be such that r - Xz - CL'(X) where eL' is linear in X E Mn L+, then the minimal solution set of (42) in the sense that tr[XL*] < tr[Xt], 1 < l < L over all X E Mn L+ satisfying (42) is given by L
{X* E Mn L+ " eL + E
rrmlNmMm-lNmT
m=l L
= E
r r ~ ( I - F F I ) N m M m - I N , ~ T ( I - FFt), 1 <_ l <_ L}
n'~-I
(50)
Moreover, this set of solutions can be assigned by the non-unique K* =
{ KI*, ..., KL* } where KL* = -FINIML -1 + (I - FtF)ZL 3, 1 <_ I <_L
(51)
with arbitrary Zt 3. The proof of this lemma is given in appendix C. Let us now apply the above results to the reduced order observation problem. First of all, the reader must be reminded that up to this point we
STOCHASTIC PARAMETER SYSTEMS
137
have assumed the availability of the noisy output measurements ytr yk-1, ... and the current operation mode rtr = i in estimating (actually predicting) the value of the state vector xtr
Second, since we are interested in
constant gain (one for each observed operation mode ) observers for easy implementability, it is assumed that the system possesses the necessary characteristics for the existence of steady state. B. L i n e a r b i a s e d e s t i m a t i o n o f a r b i t r a r y o r d e r Consider system (1) with measurement equation (2). In this section, we assume that one is interested in estimating a particular linear combination
zk = Hxk,
nz <_n~
H E R "~•
(52)
of the state variables. In doing that, the following linear observer will be used: ~,k+l =
Kl~,k + K12~
(53)
where Kzl and Kl 2 are gain sequences that depend on the current operation mode rk = I. Let us define
v =
(0) -I.~
Nz=
HA~
' g t = [g~ *, g~ 2]
(54)
-I
(55)
0
0
where
pt=limk_,ooE{(
zk xk - ~k ) [~k r , (zk - s k ) ~ l r ~ = z}
(56)
( H - I ) p, ( H T C,T) Ma =
Cz
0
-I
0
~ (0D,J 0)p,(0 D/~) ~ o o o
+~
j=l
+7rl
0
X'w
(57)
138
EDWIN ENGIN YAZ
for s o m e O < e < <
1.
L (Am Cz= P l - E Tr,,l[ HAm
O)pm(Amr 0 0
AmrH r ) 0
nv=l "It-Eqt( BmiHBm i o0)pm(t~mirOBmiTHT) O~ ( Fm~vFmT FmrvFmTHT )
(ss)
i=1
+ 7rm HFmSvFm r HFmrvFmrHr
]
Note that 0 < c < < 1 is introduced to have ML > 0, 1 _ l < L. We can now state the following results: T h e o r e m 6. When the observer (53) is used to reconstruct zk in (52) in the model (1) and (2) with the associated assumptions, then the steady state solution of the conditional covariance pl defined in (56) is given by equation (42) with the matrices F, KI, Nz, ~, MI, and r given by (54)-(58). A given conditional covariance P = {P1, ..., PL} E Mn L§ is assignable by a gain sequence K if and only if either (43) with Z2 E M L(~*+~)(n..+~) and (44) or (45) holds.If a P C Mr,L§ is assignable, then it can be assigned by the nonunique gain sequences given by (46)-(49). The minimal P E M,~L* in the sense defined in Lemma 5 is characterized in (50) which if assignable, can be assigned by the gain sequence in (51). The above results are based on finding the expression of the steady state conditional covariance pt in (56) which is done by first augmenting zk - zk dynamics to the state equation, then finding the difference equation for Pk t and finally considering the steady state case. Then the results simply follow from Lemmas 3-5. We must remark here that if one substitutes F in (54) and el in (58)
STOCHASTIC PARAMETER SYSTEMS
139
into one of the assignability conditions (44), then one obtains L
X l
qt
Z 7r~[AmXmA"r + Z BmiXmBmira~ m--1
i--1
(59)
+ mFm Fm r] which is the steady state form of (Tb). This means that for the existence of the steady state for the conditional covariances
PI~, it
is necessary for
the steady state Xk I to exist (or the system state must be m.s. bounded). This result trivially follows from the definition of
pt
in (56).
C. L i n e a r b i a s e d e s t i m a t i o n w i t h a g e n e r a l o b s e r v e r Consider system (1) with a more special measurement equation q2
Yk --
Cxk
+ E lflxkflj + Gwk
(60)
j=l
where the coeffients are not dependent on the Markov chain. This simplification is affected to estimate the same linear combination zk in (52) with a more general observer ik+l = Kli2k + KI 2Yk+i + Kz3Yk
(61)
Although it looks like, in computing 2k+1, we need to process the output measurement yk+l instantaneously, later on, it will be shown that an equivalent form to (61) which does not require Yk+l can be used in computations. This problem can again be reformulated to suit the required form for the application of Lemmas 3-5 in the following way. Let us define F in the same way as in (54) and pZ as in (56) and Kt - [Kl 1, Kl2, Kt3]
(62)
--
m
+
~
+
~
~
I
II
~
~i.i, o
~
~bd
~
~
~
~
Cb
--
""
II
STOCHASTIC PARAMETER SYSTEMS
141
T h e o r e m 7. When the general observer (61) is used to reconstruct zk in (52) for the system model (1) from the measurements described by (60) with the associated assumptions, then the steady state solution of the conditional covariance pt in (56) is given by equation (42) with the matrices F, IQ, Nt, MI, el defined as in (54), (62)-(65). A given conditional covariance set P E Mn L+ is assignable by a gain sequence K if and only if either (43)
with s e ML(~.+~)(~+2~) and (44) (or equivalently (59)) or (45) holds. If a P E Mn L+ is assignable, then it can be assigned by the nonunique gain
sequences given by (46)-(49). The P E M,, L§ with minimal trace satisfies (50) and if it is assignable, can be assigned by the observer gain (51). The proof of the above result can be done by first finding the equation (56) that pt satisfies and then applying Lemmas 3-5. We now show that it is not necessary to process the current measurement instantaneously in (61). Defining ~)k ~" "~k -- Kl,k-12 Yk
(66)
we find that its evolution is given by ~)k+l - -
K l l ~ 94 - ( K l 3 ~t_ K l l K l , k_12) yk
(67)
So, equation (67), that does not necessitate yk+l, can be used in computations and zk can be easily found by (66). D. Linear u n b i a s e d e s t i m a t i o n w i t h a g e n e r a l o b s e r v e r
Consider system (1) with the measurement equation (60). We again use the general observer given in (61) to reconstruct zk in (52) for nz >_ nz - n~ which places a lower bound on the observer order. Assume also without loss of generality that C and H are full row-rank. Defining the
142
E D W I N E N G I N YAZ
estimation error ek as eL -- zk - zk, its dynamics are given by qt (~k+l =
Kllek + (H[AI
+~
" i] -- IQ1H Bl*ak
i=1 q2
qL
q2
- K?[c + ~ ~~+~Jl[-A, + ~ B , % ~1- K,3[C + Z ~ J l ) ~ j =1
j--I
i=1
q2
+ (H - Kl2[C + E
DJ[3~+li])Flvk - K'3 Gwk - Kl 2 GWk+l
3=1 (68) To have an unbiased observer, we must have ~ - H E { x o } and
HAl-
K t l H - Kt 2 C A t - KIaC = 0, for 1 _< l _< L
(69)
Rewriting (69) as
HAt - Kt2CAz = [K~:3 Kl 1]
(c)H
and using the full row-rank property of the rightmost matrix, tile existence of the solutions for Kt 1 and IQ a is always guaranteed and they are given as: K l I -~-
(H - K,2C)A(CrC + H T H ) - I H T + Zl 4
(70) Kt 3 = (H - Kt2C)A(CTC + H T H ) - I c T + Zt '5
[Zl5, Zl4l -- Z(I- ( C)H (cTc+ HTH)-I[CT HT])
(71)
with Z arbitrary. This implies that, now, we need to find only the necessary Kz 2 for the observer design. Substituting tQ 1 and tQ3 from (70)-(71) into (68), we
STOCHASTIC PARAMETER SYSTEMS
143
obtain
ek+ 1 : [(H --
KI2C)A(CTC+ HTH)-IH T4.
Zl4lek-[" ( H E
ql
B I , o~k i
i--1
q2
ql
j=l
i=1
q2
qL
+ ~ ~ DJBliaJ~i/3~+lJ] j:1 i=1
(72) q2
- {[H -
Kt2C]A(CTC + HTH)-Ic T + Z{5} E i;PfltJ)xk j=l q~
+ (H - gt~[C + }[2 ~ k § j=l
- [ ( H - KI2C)A(CTC + g r g ) -~ + ZlS]Crawk - Kl~aW~+~ As can be seen from the above equation, even when we impose t h e unbiasedness requirement, it is not possible to decouple the state xt~ dynanfics from the estimation error ek dynamics which we also observed in full order estimator design before. Let us now define the necessary variables. F and pt are the same as in (54) and (56) respectively a n d / 1 2 / =
Kt 2 is
given by
(61), and
o
Nz : (Ago HA{HH T + ZI4
qt ( + ~.:
~(
+~
Bl i HBI i
•
o)~ ( ~ o) ~ ' 0 o
o
+ m HFI
H'H.AtTC T
0
(HA,'~c~+z,5)~
(~
j=1
)~(o)
"
o)~'(~~~'o )~
-(HA{NC r + z l s ) G
0
rw
0
0
0 ~Z w
TcT
o) 0
-GC'NAtTC T GT
(73)
O~
9
d~
~I~
--,1
~bd
~~
,.. ~
bd
-I-
"~-
-,,1
-t-
C~
~
-I-
bd
-I-
-l-
c~
-I-
-III
N
z
Z
Z
STOCHASTIC PARAMETER SYSTEMS
145
can be proved: T h e o r e m 8. If the general observer (61) is used for unbiased estim.ation of zk in (52) with the dimensional constraint nz >__n x - n~ for the system model (1) based on noisy measurements (60), where C and H are full-row
rank, then the steady state solution of the conditional covariance pt in (56) is given by equation (42) with the matrices F in (54), Kt = KI 2 in (62) and NI, /14l, el in (73)-(75). A given conditional covariance set P C A([nL+ is assignable by a gain sequence K if and only if either equations (43) with E, C iL(rL,+n.)n~, and (44) (or (59)) or equation (45) holds. If a P C ll,'lnL§
is assignable, then it can be assigned by the nonunique gain sequences tft 2 given by (46)-(49) and Kt 1 and Kt 3 in (70)-(71). The P E Mn L§ with minimal trace satisfies (50) and if it is assignable, it can be assigned by the observer gain Kt 2 given by (51) and Kt 1 and K1:3 given by (70)-(71). The proof of the above result can be done by first imposing the unbiasedness condition (69) on the estimation error evolution, next finding the expression for the conditional covariance pt, and then applying Lemmas 3-5. As commented in the previous subsection, the need to process the current measurement instantaneously can be circumverted by introducing the auxiliary variable ~. given by (66). VIII. R O B U S T E S T I M A T I O N In this case, we assume that the unknown (co)variances belong to nonempty compact sets a,~ i E 0,~ i, az j E Ozj, L'~ E O., and r ~ E O~ and minimize the performance index j l _. E{li eN II2
Iro--l}
(76)
for some fixed positive integer N over the worst possible values of these (co)variances where ek is given by equation (6) for the linear unbiased full
146
EDWIN ENGIN YAZ
order estimator in section III. So, the mathematical problem we are tackling is to optimize JmM l = rain max K
jt, 1 <_1 <_L
(77)
O
subject to (6) where K is the set of operation mode dependent estimator
...0,~ q' x O~ 1 x ...O/3q2 x O~, x O~. If a saddle point
gains and O := Oa I x
exists, then the solution to (77) is the same as the sol~tion to
JMm t = max rain JN l 0
(78)
K
The necessary and sufficient condition for the equivalance of (77) and
(78) jl(Kt*,a~', oa 3 , Z'v, S~) _<jl(Kt*, (a~i) *, (crJ)*, ~v', rw*) (79)
<_ Jt(Kt, (a~i) *, (aJ)*, ~v*, Sw') The characterization of a saddle point is given below" T h e o r e m 9. (Kz*, (a~z) *, (aZJ)*, ~v*, rw*) is a saddle point for (79) if and only if for every 1 _< l < L, the following hold L
E
L -,
,
7rm/Trm,ktr[Fn,A F;. -,r
--,
--,T
Q,r l] > E 7r..Trr~.t,r[F~AF;~ Q,~+I/]
m=l
(8o)
m=l
L
ql
rn:l
i=1
Bmnk
Bm)V(o')
L
7r"~tr[E
m=l L
m:l
(81)
qi
>- E
E
l
[~iRkm([3i)Va~
i=1
q2
7r.atr[ E
([Ym)*Rkm([Nm)'r(aJ)*Qk+ll]
j=l L
>- E
(82)
q2
7r"atr[E
m-----1 . . . .
(f)~
32-1 :
Qk+l'l
STOCHASTIC PARAMETER SYSTEMS
147
where we define -~l = blockdiag[Az, A l - KICt] A = blockdia,g[S,,, S~]
-" ~i* --
(t~l i O) Bl i
(0
0
El--( FFlI O -
' D~l :
g l Gl
)
O)
- g i Ol j
' Rkl = E {
-
(83)
0
( )xtr" ek
( xk
r
e~ r ) I~k = l}
and these quantities together satisfying the forward running covarialme equation L
Rk+l I = ~
ql --
rrml[.4 a r k
m
JBml~km(Bi)Taa i "i
-T
Am+ Z
m=l q2
i=1
(84)
+ ~ ~mR~m(Z>;~)~o~; + ~ f ~ A f ~ j=l
and the backward running co-state evolution equation L
Ql l = Z
7rml[ATm Qk+IA,,-, l - +
qt
~
n~=-I q2
-i (Bin)
i=1
T'4k+1 r, l~,,~a~ v-,i i (85)
j=l
with QNZ= blockdiag[0, I]. /~t* and (/~t)* are given by (83) with IQ = IQ* and Qk I is given by (85) with Kt
Kz*, aa i = (as/) *, and aZJ = (aZJ) *.
Proof. The proof first establishes that (80) - (85) are necessary and sufficient for the existence of the saddle point for the Hamiltonian and then shows that the Hamiltonian and the performance index jl have the same saddle points. The Hamiltonian for this problem is "/~t = tr[Rk+l', Qk+l']
148
EDWINENGINYAZ
Necessity part of the proof follows fl'om the left hand side inequality in the saddle point condition for the Hamiltonian
"~l(gz*, aa', oJ , P-,v,~w) <_ 7~t(K/*, (aai) *, (aj)*, ~v*, rw*) 9
(86) _< "/~Z(K,, (a~i) *, ( a j ) * , r~*, r~*) The sufficiency part follows by assuming that there is a point ((aoi)*,(aeJ)*,r.~*,r~ *) for If, = IQ* the optimal gain, and combining (80) - (82) to give the left hand side inequality in (86). The right hand side inequality ill (86) follows fl'om the optimality of Kz*. The proof is completed by showing that the performance index and the Hamiltonian have the same saddle points as in [26] and
[47].
If the set of (co)variances has maximum points like a s1 _< (o~),, ..., ~
_<
rw*, then these elements will satisfy the saddle point conditions. One can apply the same principles used in this section to design robust observers of reduced order. The reader needs to be reminded at this point that in addition to the minimax approach to robustness taken here, it is possible to design robust observers by other means e. g. by the use of an exponentially data-weighting observer as in [27] and [48].
DISCUSSION We have considered the observer design problem for a general class of discrete-time stochastic parameter systems. Both full and reduced, biased and unbiased linear observers have been parametrized for this class of models. The minimum variance optimal ones have been found in the class of linear observers. Robustification of full order observers has been achieved by minimax techniques. The reader needs to be aware of the strengths and shortcomings of the type of observers advocated here. First of all,all observers proposed in this
STOCHASTICPARAMETERSYSTEMS
149
work have linear structure.This implies that structural simplicity (which directly translates into implementation ease) is preferred over further performance enhancement potentially obtainable by the use of nonlinear observers. Second, it is assumed that the exact knowledge of current state of the Markov chain is awfilable, which may not be true in some applications. The reader is referred to the available works in this m'ea [14] , [29]- [39] which assume the availability of varying levels of information about the Markov chain. Some continuous-time observer design results can be found in [50]. We have assumed for the simplicity of presentation in this work that the additive noises in the system and measurement equations are white, mutually independent and also independent of the multiplicative white noises. For a discussion of how these requirements can be relaxed, see reference [23] which considers linear minimum variance filtering with mutually correlated additive and multiplica.tive white noises and applies the results to measurement differencing ,which is a scheme to design filters having the same order as the system when the measurement contains colored noise effects (whose dynamics increase the dimension of the necessary state estimator). In cases where the stochastic parameters do not behave like white noise or a Markov chain, more complex models can be used like colored multiplicative noise [51] or more elaborate jump processes as in e.g. [14] and [37]. Throughout this work, it has been assumed that the past and present data are available and one step prediction of the state vector is needed. It is also possible to tackle the problems of smoothing where a past value of the state is estimated or higher step prediction using similar techniques to
[52].
150
E D W I N E N G I N YAZ
APPENDIX
A
Proof of Lemma 3: Let us first show that statements i) and ii) are equivalent.
(J ive n
equation (42), one can complete the squares to obtain L
dpl 4- E
rCrdNmMm- 1Nm T
m=l
(A1)
L
= E 7rn~(FKm 4- NmMm-1)Mm(Fl(m 4- NmMm -1) T rrv=l Since each term on the righthand side is nonnegative definite and of ra.nk not exceeding "p", we have the same properties for the terms on the lefthand side of (A1). This gives (43). So, for any 1 _< 1 _< L, considering ~l~l T - - (I'ts 4- Nll~l-1)l~/[l(I'gl 4- N l ~ 1 - 1 ) T
(,42)
there must exist an orthogonal matrix sequence U~ [43] such that ~cI gl -- ( F Kl 4- Nl Ml -1) Ml 1/2
(A3)
F K I --" ~C,lal n~1-1/2 -- Nl l~1-1
(A4)
or
One can solve these matrix equations for Kt if and only if [49] we have = 0
(AS)
(I - FF~)s Ut = (I - FFt)NI Mr- T/2
(A6)
(1 - FU)(Lt
Mz -1/2 - N z M , - ' )
for all 1 < l < L. Rewriting (A5) as
multiplying both sides in (A6) by their transposes, using UIUtr =
I, and
substituting from (43), we find (44). So, assignability of X in (42) by some
STOCHASTIC PARAMETER SYSTEMS
151
Kz is equivalent to the existence of an s e M~p L such that (43) and (44) hold. Let us now show that ii) and iii) are equivalent. Substituting qhl fl'om equation (43)into (44) and rearranging, we obtain ( I -- F F * ) N I M I - I N I T ( I
- 171 "*) -" ( I -- l ~ F * ) ~ l ~ l T ( ]
I"I"*)
(At)
Using the same matrix decomposition result we applied above, this yields (I - InI"()Nl~/[l - T -- ( I -
(AS)
Inln'f)~i gl
for some orthogonal sequence ~ or (I -
rr*)c,
= (i - rr*)N,M,-r~
-'
(A9)
This equation has always a solution El [44] and the nonunique solutions are given by
s = (I - F F t ) t ( I - F F * ) N z M ; - r ~ -1 + [I-
(I - vv*)*(I
-
where Zt are arbitrary. It is easy to see that ( I matrix and ( I - FFt) ~ = I -
(A10)
vv*)]2,
F F t) is an idempotent
FF~, therefore,
c, = ( 1 - r r * ) N , M , - r O , -1 + rr*
,
(All)
or for some arbitrary m a t r i x Zl 1, we have Z:l = [ ( I - / - ~ r t ) N / q - FF~Zll]MI -T
(A12)
Substituting Z;t in (A12) into equation (43) gives equation (45). Therefore, statements ii) and iii) in Lemma 3 are equivalent.
152
E D W I N E N G I N YAZ
APPENDIX
B
Proof of Lemma 4: We start the proof with equation (A4) which, if solvable, has solutions given by (46). To find the necessary orthogonal Ut in (46), we look at equation (A6). Considering the singular value decompositions of matrices (48) and (49), Ut can be solved to give (47). The details of this procedure are provided in [43]. APPENDIX C Proof of Lemma 5: Rearrange, take the trace of both sides in equation (45) and apply trace properties L
t r [ X t ] - -tr[r
+ E
7rmzNmMm-lNmT]
m=l L
+ E
7rmt{tr[(I- F F t ) N m M m - l i m r ( I - FFt)]
m=l
+ tr[FFt(I - FFt)NmM~-Iz~ it] 4- tr[ZmlMm-lNmT(I- FFt)FFt]}
(el)
L
-- -tr[r
+ E
~"YmMm-lym~
m--1 L
+ E
7~,,,tr[(I- FFt)NmMm-INmT(I- FFt)]
m=l
+ tr[FF t Zm1Mm-1 zmlrFF*] The cancellation of terms follows from the definition of pseudo-inverses. tr[Xt] in (C1) can be minimized by assuming Zm 1 such that FFtZm 1 = O, which gives equation (50). Substituting f-..t*Ul = ( I - FF?)NIMI -T into
STOCHASTIC PARAMETER SYSTEMS
153
equation (46), we obtain Kl* -- F t [ ( / -
I~I-'t)Nl i~/[l- Ti~/[1-1/2 -- Nl~f1-11 + (I - Ct F)Zz 2
(C2) = - F t N ~ M t -1 + ( I -
FtF)Zt 2
or equation (51) with arbitrary matrix denoted by Zt 3. ACKNOWLEDGEMENT This research was partially supported by the National Science Foundation under grant no. ECS-9322798.
REFERENCES 1. P.J.McLane,"Optimal stochastic control of linear systems with state and control- dependent disturbances, "IEEE Trans. Automat. Contr., 16, pp. 793- 798 (1971). 2. N.J.Rao, D.Ramakrishna, and D.J.Borwanker, "Nonlinear stochastic simulation of stirred tank reactors, "Chem. Engin. Sci., 29, pp. 1193 - 1204 (1974). 3. M.Aoki, Optimal Control and System Theory in Dynamic Economic Analysis. New York: Elsevier (1976). 4. R.R.Mohler and W.J.Kolodziej, " An overview of stochastic bilinear control processes, "IEEE Trans. Syst. Man Cyber., 10, pp . 913 - 918 (1980). 5. A.J.M.Van Wingerden and W.L.DeKoning, "The influence of finite wordlength on digital optimal control, "IEEE Trans. Autom. Contr., 29, pp. 385- 391 (1984). 6. J.L.Willems and J.C.Willems, "Robust stabilization of uncertain systems, "SIAM J. Optim. Contr., 21, pp.352- 374 (1983).
154
EDWINENGINYAZ
7. D.S.Bernstein and D.C.Hyland, "The optimal projection / maximum entropy approach to designing low- order, robust controllers for flexible structures, "Proc. 24th Conf. Decision Contr., Ft. Lauderdale, FL, pp. 745- 7,52 (1985). 8. E.Yaz, "Feedback controllers for stochastic- parameter systems: relationships among variolls stabilizability conditions, "Optim. Cont.r. Applic. Methods, 9, pp. 325- 332 (1988). 9. N.E.Nahi,"Optimal recursive estimation with uncertain observation," Trans. Informat. Theory, 15, pp. 457- 462 (1969). 10. W.NaNacara and E.Yaz,"Linear and nonlinear estimation with uncertain observations, "Proc. Amer. Contr. Conf., Baltimore, MD, pp. 1429- 1433 (1994). 11. W.L.DeKoning, "Opt.imal estimation of linear discrete-time syst.ems with stochastic parameters, "Automatica, 20, pp. 113 - 115 (1984). 12. W.P.Blair and D.D.Sworder, "Feedback control of a class of linear discrete- time systems with jump parameters and quadratic cost criteria, " Int. J. Contr., 21, pp. 833- 841 (1975). 13. R.Malhame and C.Y.Chong, "Electric load model synthesis by diffusion approxinmtion in a higher order hybrid state stochastic system, "IEEE Trans. Autom. Contr., 30, pp. 854- 860 (1985). 14. R.L.Moose, H.F.VanLandingham, and D.H.McCabe, "Modelling and estimation for tracking maneuvering targets, "IEEE Trans. Aerosp. Electron. Syst., 15, pp. 448- 456 (1979). 15. D.D.Sworder and D.S.Chou, "A survey of design methods for ra.ndom parameter systems, "Proc. 24th Conf. Decision Contr., pp. 894- 899 (1985).
STOCHASTIC PARAMETER SYSTEMS
155
16. R.Akella and P.R.Kumar, "Optimal control of production rate in a. failure prone manllfacturing system, "IEEE Trails. Autom. Cont.r., 31, pp. 116- 126 (1986). 17. H.J.Chizek and Y.ai, "Applying jump systems control theory to discrete e v e n t - driven hybrid systems, "Proc.
Amer.
Contr.
Conf.,
Boston, MA, pp. 1_569- 1573 (1991). 18. P.K.Rajasekaran, N.Satyanarayana, and M.D.Srinath, " O p t i r n l l r n lil iear estimation of stochastic signals in tile presence of multiplicative noise, "IEEE Trans. Aerosp. Electron. Syst. 7, pp. 462- 468 (1971). 19. M.T.Hadidi and C.S.Schwartz, "Linear recursive state estimators under uncertain observations, "IEEE Trans. Autom. Contr., 24, pp. 944 -
948 (1979).
20. a.K.Tugnait, "Stability of optimum linear estimators of stochastic signals in white multiplicative noise, "IEEE Trans. Autom. Contr. 26, pp. 7 5 7 - 761 (1981). 21. E.Yaz, "Observer design for stochastic parameter systems, "Int. J. Contr., 46, pp. 1213- 1217 (1987). 22. E.Yaz, "Implications of a result on observer design for stochastic parameter systems, "Int. J. Contr. ,47, pp. 1355- 1360 (1988). 23. E.Yaz, "Optimal state estimation with correlated multiplicative and additive noise and its application to measurement differencing, "Proc. Amer. Contr. Conf., pp. 317- 318 (1990). 24. E.Yaz, "On the almost sure and mean square exponential convergence of some stochastic observers, IEEE Trans. Autom. Contr., 35, pp. 935- 936 (1990). 25. E.Yaz, "Robustness of stochastic parameter control and estimation
156
EDWINENGINYAZ schemes, "IEEE Trans. Autom. Contr., 35, pp. 637- 640 (1990).
26. E.Yaz, "Mirfimax state estimation for jump-parameter discrete time systems with multiplicative noise of uncertain covariance, "Proc.Amer. Contr. Conf., pp. 1574- 1578 (1991). 27. E.Yaz, "Estimation and control of stochastic bilinear systems with
prescribed degree of stability, "Int. J. Systen~s Sci., 22, pp. 835- 8::13 (1991). 28. E.Yaz, "Full and reduced order observer design for discrete stochastic bilinear systems, "IEEE Trans. Autom. Contr., 37, pp. 503- 505 (1992). 29. G.A.Ackerson and K.S.Fu, "On state estimation in switching environments, "IEEE Trans. Autom. Contr., 15, pp. 10- 17 (1970). 30. H.Akashi and H.Kumamoto, "Random sampling approach to state estimation in switching environments, "Automatica, 13, pp. 4 2 9 - 434 (1977). 31. C.G.Chang and M.Athans, "State estimation for discrete systems with switching parameters, "IEEE Trans. Aerosp. Electron. Syst., 14, pp. 418- 424 (1978). 32. J.K.Tugnait, "Comments on "Sate estimation for discrete systems with switching parameters, " "IEEE Trans. Aerosp. Electron. Syst., 15, p.464 (1979).
33. J.K.Tugnait, "Detection and estimation for abruptly changing systems, "Automatica, 18, pp. 607- 615 (1982). 34. J.K.Tugnait, "Adaptive estimation and identification for discrete systems with jump Markov parameters, "IEEE Trans. Autom. Contr., 27, pp. 1054- 1064 (1982).
STOCHASTICPARAMETERSYSTEMS
157
35. H.A.P.Blom and Y.Bar-Shalom, "The interacting multiple model algorithm for systems with Markovian switching coefficients, "IEEE Trans. Autom. Contr., 33, pp. 780- 783 (1988). 36, Y.Bar-Shalom, C.Y.Chang, and H.A.P.Blom, "Tracking a maneuvering target using input estimation versus the interacting multiple model algorithm, "IEEE Trans. Aerosp. Electron. Syst., vol, pp. 296- 300 (1989). 37. L.Campo, P.Mookerjee, and Y.Bar-Shalom, "State estimation for systems with sojourn- time- dependent Markov model switching, "IEEE Trans. Autom. Contr., 36, pp. 238- 243 (1991). 38. W.D.Blair and D.Kazakos, "Second order interacting multiple model algorithm for systems with Markovian switching coemcients, " Proc. Amer. Contr. Conf., San Francisco,CA, pp. 4 8 4 - 487 (1993). 39. W.D.Blair and D.Kazakos, "Estimation and detection for systems with second order Markovian switching coefficients, "Proc. Amer. Contr. Conf., Baltimore, MD, pp. 1427- 1428 (1994). 40. R.A.Horn and C.R.Johnson, Topics in Matrix Analysis, Cambridge Univ. Press:Cambridge (1991). 41. T.Kato, Perturbation Theory of Linear Operators, 2nd Ed., SpringerVerlag: Berlin (1980). 42. V.Hutson and J.S.Pym, Applications of Functional Analysis and Operator Theory, Academic Press" London (1980). 43. R.E.Skelton and T.Iwasaki, "Lyapunov and covariance controllers," Int. J. Contr., 57, pp. 519- 536 (1993). 44. W.L.DeKoning, "Infinite horizon optimal control of linear discretetime systems with stochastic parameters, "Automatica, 18, pp. 443 -
158
EDWINENGINYAZ 453 (1982).
45. T.Morozan, ))Stabilization of some stochastic discrete - time control systems, "Stochastic Analy. Applic., 1, pp. 89- 116 (1983). 46. P.T.Nhu, ))On a system of matrix Riccati equations with applications to optimal stabilization under stochastic disturbances, "Revue R olin1. Math. Pures et Appl., 26, pp. 991 - 1004 (1981). 47. Y.A.Phillis, "Optimal estimation and control of discrete mld(,iplicat,ive systems with unknown second-order statistics, "JOTA, 64, pp. 153168 (1990). 48. E.Yaz and R.E.Skelton, "Parametrization of all linear compensators for discrete- time stochastic parameter systems, "Automatica, 30, pp. 945- 955 (1994). 49. A.Ben-Israel and T.N.E.Greville, Generalized Inverses:Theory and Applications,Wiley:New York (1974). 50. M.Mariton, "Jump Linear Systems in Automatic Control, "Marcel Dekker:New York (1990). 51. B.S.Chow and W.P.Birkemeier, "An error analysis for the Kalman filter applied to data corrupted by multiplicative noise, "Proc. Allerton Conf., Monticello, IL, pp. 453- 462 (1987). 52. B.D.O.Anderson and J.B.Moore, Optimal Filtering, Prentice Hall: Englewood Cliffs, N.J. (1979).
The Recursive Estimation of Time Delay in Sampled-Data Control Systems
Gianni Ferretti Claudio Maffezzoni Dipartimento di Elettronica e Informazione Politecnico di Milano Piazza Leonardo da Vinci 32, 20133 Milano, Italy
Riccardo Scattolini Dipartimento di Informatica e Sistemistica Universit~ degli Studi di Pavia Via Abbiategrasso 209, 27100 Pavia, Italy
I. I N T R O D U C T I O N Many industrial plants have inherent time varying delays, the on-line estimation of which is of major importance for a number of reasons. Among them, it is possible here to recall that time delay variations can represent a symptom of process behaviour degradation, as is the case of grinding processes. Hence, monitoring the delay value can easily lead to the development of efficient diagnostic procedures [ 1]. Furthermore, since delay systems are difficult to control with classical feedback regulators, the knowledge of the delay can be exploited to CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
159
160
GIANNI FERRETTI ET AL.
enhance closed-loop performance by resorting to time delay compensation schemes, such as the Smith predictor [21. However, it is recognized that these predictive techniques, if applied with an incorrect delay estimate, lead to poor control performance, see e.g. [31, [41. Finally, it has to be recalled that many rules for the automatic tuning of simple and popular PI or PID regulators are based on a simplified first order or second order process model plus time delay, such as the Ziegler-Nichols rules or related ones, see e.g. [51, [61. All the above reasons motivate the research of simple and reliable identification algorithms for recursive time delay estimation. However, when the time delay appears as a parameter to be estimated, the model to be identified becomes nonlinear in the parameters even when the rest of the system is a linear one. This makes the identification problem particularly complex and justifies the huge literature nowadays available in the field. The aim of this paper is to review the main approaches to the recursive delay estimation problem, to discuss their applicability assumptions and to compare the related performance in a number of simulation experiments. More specifically, five approaches are dealt with, namely extended B-polynomial methods, correlation methods, nonlinear least squares methods, rational approximations and variable regressor methods. For anyone of them, some specific algorithms proposed in the
literature are briefly presented in order to provide the reader with useful implementation considerations and, sometimes, tricks which often make the algorithms successful in practice. The presentation is organized as follows. Section 2 is devoted to the formulation of the problem of the delay estimation of single input, single output systems originally described in continuous time. Then, Sections 3-7 present the considered techniques, while in Section 8 many simulation results are discussed and compared. Finally, Section 9 closes the paper with some concluding remarks.
II. M O D E L F O R M U L A T I O N Consider the sampled data system of Fig. 1, where
RECURSIVE ESTIMATION OF TIME DELAY
161
H(s) = H ( s ) e -r = D(s) N(s) e-z*" Hr(S) is a proper rational transfer function and h is the sampling period, and split the continuous time delay z into an integer and a fractional part: 1: = d' h + e h, with 0 < e < 1. If (F,g,h) is a realization of Hr(S), the pulse transfer function between
u(k) and y(k) is given by (z-] is the backward shift operator): G(z -1) = h (2:I - (1))-I (TO + z-I "YI)z- , dt
h--eh
eh
. = eFh, ~[0= : eFx dx g, Tl = eF(h"eh) : eFX dx g. 0
0
The sampled data system can be also described by a linear difference equation of order n = deg{D(s)}"
A(a -I) y(k) = B(z -]) u(k - d), A(z -1)
=
1 + a i z, 1 + ... + a n
(1.1)
z "n
(1.2)
B(z-1) = bo + bl z-1 + ... + b n z -n
u(k)
ZOH /I
(1.3)
i y(k)
,.~1
9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fig. 1 Sampled data system being d = d' + 1. Note that the fractional delay e h gives rise to a
unitary increment
of the discrete-time delay, to an additional zero in the numerator of the pulse transfer function and to an additional parameter (bo) in the B-polynomial (1.3). Without loss of generality, from now on it will be assumed b o r 0.
162
GIANNI FERRETTI ET AL.
III. E X T E N D E D B - P O L Y N O M I A L M E T H O D S A. BEBA" BASIC EXTENDED B-POLYNOMIAL ALGORITHM
The key idea is the following. Assume a lower and an upper bound for the integer delay: d i n < d < dax; then model (1) can be rewritten as: A ( z -1) y ( k ) = B*(z -1) u(k - d i n ) ,
B*(z-1) = b o* + b *I Z - 1
+ ... + b n* g - n + ... + b n* + dmax- amin z-'(n + areax
(2.1) amin)(2.2)
so introducing ( d a x - d i n ) extra parameters. Therefore, since: bi = 0
0
bi = bi-(d-dmin)
d- din < i < d- din + n
b, =o
a_am, +.
am _am,
the delay d can be estimated as d = i + d m, being i the index of the first nonzero
A
parameter of the l~lynolmal B* (z- 1).
f bA,i = O
i
A.
b7~0 which can be in turn estimated through a standard RLS algorithm. This is a quite simple approach, but it suffers from various drawbacks, first of all the fact that the first ( d - d i n ) estimated parameters are never exactly zero or even definitely "small" in comparison to the succeeding parameters of the
A
polynomial B*(z-l), so that it can be difficult to determine the correct 7 in real cases. Particularly in presence of noise it could be very difficult to discriminate between zero and nonzero terms in the numerator model. Moreover, it must be pointed out that the computational burden of the RLS algorithm increases with the square of the number of the estimated parameters.
RECURSIVE ESTIMATION OF TIME DELAY
163
B. RLSVT: RECURSIVE LEAST SQUARES METHOD FOR VARYING TIME DELAYS (KURZ AND GOEDECKE, 1981, [7])
As already mentioned, the main pitfallof the extended B-polynomial method,
in its basic formulation, is the fact that the discrete-time delay is actually identified on the basis of just one term of the impulse response of the estimated extended
model. To overcome this pitfall,Kurz and Goedecke [7] proposed a more robust
way to estimate the delay d, based on a comparison, extended to N terms, between
the impulse response of the estimated extended model and the impulse responses of several original models (1), each one characterized by a different value of delay d. A
Precisely, in their algorithm the best estimate d of the actual time delay is given by A
d = d mlll . + Ad, being Ad the value of Ad that minimizes the error function: N
N
i=0
i=O
where ~*(i) and ~,Ad(i) are respectively the estimated weighting functions, defined as follows: A
oO
B2(r') = Z X(z-l)
A(z
=s /=0
~=o
r,,
z-', (~ad(i) =
(3.1)
O, i < At/)
(3.2)
and N is chosen such that N > t9~/h, being t95 a time interval equal to the 95% of the settling time of the process response. The search range of Ad is limited to the interval 0 < Ad < Admax, where Admax is defined as: A,
A,
b Admax = max{ bi, i = O, ..., n + d a x - d m}.
164
G I A N N I F E R R E T T I E T AL.
An essential saving of computational effort is obtained by recursively ^
computing the errors ~gz~/(i) without explicit calculation of ~*(i) and gz~d0)- The ^ parameters b i of the process model (3.2) are finally computed equating the values ^
^
.
g *(i) with the values g-~0), for i = A-d, .... n + 5d: i
b~-
5
-i+A--~, with~0 1.
j--0 C. A D T E :
ADAPTIVE DEAD-TIME ESTIMATOR (DE KEYSER, 1986, [81)
A much simpler but quite empirical algorithm, requiring the estimation of just one additional parameters of the B-polynomial, has been proposed by De Keyser [8]. At each sampling instant the following model structure is estimated ^
with a given d" A
A
A
^ b o + b I z -1 + ... + bn+ 1Z - ( n + l ) ^ G ,(z- l ) = ,, 9 z-d .
A(z-I)
A
then, the estimate d is increased or deereasexl according to the following rule: f
A
A bn+ I =~ Ad = d A- 1
A
A
b,+ 11
> 0~ 16o]
=
A
(4)
A
d=a+ 1 A
Essentially, at each sampling instant it is tested whether the structure G'(z-1) can be best fit by the candidate structure G_(z' l ) or G+(zl):
G_(z-1) =
b o + b I Z'-I + ... + b n z -n
A(z._I)
z-d,
b I z - l + b 2 z - 2 + ... + bn+ 1 z"(n+l)
~+(z-l) =
A(z._l )
z-a,
indicating that the dead-time should be respectively decreased or increased. The criterion (4) can be understood by considering the hypotetical experiment depicted
RECURSIVEESTIMATIONOFTIMEDELAY
165
2
in Fig. 2, where u(k) = WN(0,~,2). Defining with 0"_ 2, ~+ the variances of the two error sequences e_(k) and e+(k):
2 1 E { [ U ( k - A(z_I) d-n-1)7 ty_2 = E[e2(k)] = bn+
}
(5.1)
2 E[e2+(k)]=b 2E{ ( k - d) 7 } ; a+= o I .A(z_l)
(5.2)
2
2
2
if 0"2_>> Or+ the candidate structure G ( z -l) is selected, otherwise if cr_ << 0"+ the 2
2
2
2
structure ~+(Z-1) is selected. From (5) it follows cy-+/cy_= b~/bn+1 as well as rule (4)
(De Keyser suggests o~= 5).
u(kl l -ll
e_Ckl
Fig. 2 ADTE: Hypothetical experiment D. OTHER ALC~RITHMS Several other recursive algorithms have been proposed based on an extension of the B-polynomial, which however do not result in an explicit estimate of the delay, but in an approximation of the whole transfer function of the process, suitable to adaptive controller design. Keviczl~ and B~y~sz [9] consider a second order lag plus a time delay as the continuous process transfer function and a discrete-time model obtained through the first-order Euler-approximation:
166
GIANNI FERRETTI ET AL.
(6)
(1 + a 1 z-1 + a 2 z-2) y(k) = b o u(k- d'~,
where d " = d' + 2. An estimate of b 0 and d" is then determined by imposing a model matching at zero frequency between models (6) and (2), thus imposing: A
A
n+
~
AS
b 0 = B*(1) = 2.a bi'
with n + = n +dma x - dram
i=0
d (boe-J~d")
dro
A
_ d B*(e-J ~ o~-o dm
A
A,
d ''=int
to=o
i b~ + 0 .
.
kbo i= 1
The estimated model (6) is then used in an adaptive PID-based pole cancellation control strategy. Vogel and Edgar [ 10] adopt the extended B-polynomial method just to avoid an esplicit on-line estimate of the discrete delay in an adaptive pole/zero placement control strategy: they simply retain the whole estimated B-polynomial as the numerator of the desired closed loop transfer function. A similar approach is adopted by Chandra Prasad et al. [11], applied to the case of a continuous process described by an integrator plus a time delay. Clough and Park [ 121 apply the extended B-polynomial method to determine the optimal prediction step (which may not coincide with the true delay d) in a standard minimum-variance self-tuning controller, while Chien et al. [13] replace completely the d-step ahead predictor with a Smith predictor, based on the extended B-polynomial. Time delay estimation and model extension have been also considered in the realm of Kalman filtering methods, first by Biswas and Singh [14] and then by Abrishamkar and Bekey [ 15].
RECURSIVE ESTIMATION OF TIME DELAY
167
IV. C O R R E L A T I O N M E T H O D S
A. DECA'DELAY ESTIMATION BASED ON CORRELATIONANALYSIS (ZHENG AND FENG, 1990, [ 16]) Stochastic discrete systems of the form: A(z -'1) y(k) = B(z "1) u(k - d) + v(k),
(7)
are considered in the approach proposed by Zheng and Feng [16], where v(k) and
u(k) are assumed to be mutually uncorrelated, stationary, ergodic, random sequences with zero means. Multiplying both members of ( 7 ) b y u ( k -
T) and taking mathematical
expectation leads to: /'t
ryu(I) = - E
ai ryu(T- i) + E
i=1
b~ru.(T- i - d) + rw(73
i=0
where %u(T), ryu(T), rvu(7) are respectively the auto-correlation function of the input u(k) and the cross-correlation functions between the input u(k) and the output
y(k) and between the input u(k) and the disturbance v(k). Letting u(k) be a white 2
random signal with variance cru and recalling that rvu(T) = 0 by hypothesis, one obtains"
f %(r) = o
T
%(a) = b o au
Therefore, the least time argument at which the estimated cross-correlation function ~yu(T) is definitely different from a null value can be assumed as an estimate of the discrete delay d. In turn, the function ~vu(T) can be recursively estimated as follows:
168
GIANNI FERRETTI ET AL. ^
A
^
y(k + 1) u(k + 1 - T) - r vu(T,k)
ryu(T,k + 1) = ryu(T,k ) +
k 4- 1
"
A straightforward estension of the algorithm can be considered if u(k) is a
more general stationary random sequence, modelled by an invertible I ARMA process:
C(z -1) u(k) = D(z-I)r(k),
(8)
2 where r(k) is a zero-mean white sequence with variance (Yr. In fact, combining eqs.
(7) and (8) yield: A(z -1) y(k) = z -d B(z "1) r(k) + v(k), 2(Z "1) - A ( z -1) C(/-l), B(z-1) = B(z"I) D(/-1),
~(k) = C(z -~) v(k). Therefore, once the polynomials C(2~-1) and D(~ -1) and the white sequence r(k) have been estimated from the sequence u(k) [17], the above estimation criterion can
be still applied to the cross-correlation function ~yr(T) instead of ~yu(T). Moreover, in the simpler case where the input sequence u(k) can be modelled as a MA process with index of auto-correlation p" u(k) = F(z"l) r(k),
=fo +fl <-I + ... +f, : ,
lall the rootsof D(z-1) are assumedoutsidethe closedunitdisc
RECURSIVEESTIMATIONOF TIMEDELAY
169
the estimation of both the input model (namely of the parameters f~) and the white sequence r(k) can be avoided, since it can be shown that: %(7)=0
for
T
%(d-p) bofo4 ~, =
A
A
A
Therefore, the delay d can be estimated as d - T + p, being again 7' the least time A
argument at which the estimated cross-correlation function r (7) is different from zero. In particular, Zheng and Feng propose the following criterion to determine A
the delay estimate dk+l at instant k + 1"
I I A% ( r , k + l ) A
I = < e k+~
A
; T < dk+ ~ - p
A
A
[ ryu(dk+1 -p,k+l) [ >> max{ [ ~yu(T,k+l) ] ; m < dk+ ~ - p } where {ek} is a non-negative constant sequence which monotonically decreases to zero 2. Note that a basic assumption of the approach is the postulate of uncorrelation between input and disturbance, which does not obviously hold in closed loop operations. An extension of the approach to this case is also proposed by Zheng and Feng, based on the computation of the generalized cross-correlation function, defined as: n
p~(r) = %(r) - ~ a, % ( r - O, i=I
where the parameters a i are estimated using the Yule-Walker equations. The extension is however based on the hypothesis of a finite and known crosscorrelation order between input and disturbance and of an input white sequence, which is a much less reasonable hypothesis in closed loop operations.
2The authors suggest to relate the rate of convergence of { ek} to the Signal-to-Noise ration value
170
GIANNI FERRETTI ET AL.
B. FMVRE: FIXED MODEL VARIABLE REGRESSION ESTIMATOR (DUMONT, ELNAGGAR AND ELSHAFEI, 1991, [18 - 20]) The key idea of the method [18-20] is to apply the correlation analysis to a simpler fixed first-order auxiliary model, relying on the fact that many industrial processes are actually over-damped and well described by a single time lag plus delay. The estimate of the integer delay is then found by minimizing the performance index: J~dm)
(9)
= E[y(k) - Y m ( k , d ) l 2,
where Ym(k, dm)= am y ( k -
1) + b m u ( k - 1 - din)
Expanding (9) then yields [181: 2
*
J ~ ~ - ( d m ) = (1 + a 2) ryr(0 ) + b m ruu(O) - 2 a m ryy(1) - 2 b m JFMVRE(dm)
J
a) = %(1 + a m) -
r(a)
so that minimizing (9) is equivalent to maximize J m 4 v / ~ d ) ; moreover, choosing a m = 1, defined by Elnaggar as the best choice [201, one obtains"
JFMI~(dm) = rAyu(1 + d m)
(10)
where A = 1 - z-1 is the backward difference operator. According to (10) the delay A
A
A
can be estimated as d = T - 1, being T the time argument that maximizes the crosscorrelation function rAyu(T) between the input variable and the output increments.
RECURSIVE ESTIMATION OF TIME DELAY
171
C. OTI-ER ALGORITHMS The correlation analysis approach is quite popular in the signal processing community, with reference to the particular case of the estimation of the time delay between two spatially separated sensors in the presence of uncorrelated noise. No dynamics other than the time delay is considered, and the delay estimate is simply assumed as the time argument at which the cross-correlation function between the signals measured at both sides of the delay achieves a maximum; several filtering techniques are analysed in [21-22] in order to facilitate the estimation. V. N O N L I N E A R L E A S T S Q U A R E S M E T H O D S A. NTDI: NONLINEAR TIME DELAY/DENTIFICATION
The parameters of the pulse transfer function (1) and the discrete delay d can be simultaneously estimated by minimizing the classical (equation error) Least Squares loss function: 1
1
JNTDI(O) = ~ E{e(k,O) 2 } = ~ E{ [y(k) - ~k,d) r
Ol2}
(11)
where O is the vector of the unknown parameters (inclusive of the discrete delay d) and ~k,d) is the observation vector: O=[a
1 ... a n b 0 ... b
d] T
q~k,d) = [ - y ( k - 1) ... - y ( k - n) u ( k - d) ... u ( k - d - n) 0 ]1".
One major problem arising with this approach is due to the fact that the loss function (11) is nonlinear with respect to the discrete delay d, calling for the adoption of nonlinear estimation algorithms, such as Newton's method [23]. Adopting the following classical approximation of the Hessian of the loss function
JNTDI(O):
172
GIANNI FERRETTI ET AL.
02sNro~ . O) = oo oo T Oe(k,O) ~(k,O) = - ~ o=o
[ -y(k-
E[~(k,O) ~(k,O)T],
1) ... - y ( k -
n) u ( k - at) ... u ( k - d - n) ...
n
"'" - Z
bj u ( k - d - i) l T
J=0
and exploiting the matrix inversion lemma, the identification algorithm can be given the following re~ursive form:
e(k) = y(k)
-
A
P(k) = P(k - l) -
1 + ~(k, O(k A
A
1)~(k,O(k- 1))~(k,O(k- 1)) T P ( k A A
P(k-
A
(12.1)
tp(k)T O(k - 1)
1))T P(k
1) (12.2)
- 1) ~(k, O(k - 1))
A
O(k) = O(k- 1) + P(k) ~(k,O(k- 1)) e(k).
(12.3)
It must be pointed out that the discrete delay d is considered as a real parameter in (12.3), as well as in the computation of the error sensitivity functions ~(k,O), while A
A
the best integer approximation dI of the estimate d(k - 1), given by (12.3), should A
be used when detennimng the k-step observation vector [241 (es." u ( k - dl))" A
A
dI = int[d(k- 1) + 0.51. The estimation algorithm described has been applied by B~hay~sz and Keviczky [24] with a slightly different model representation, namely decomposing the system into Elementary SubSystems (ESS) [25]. Another major problem related to any time delay estimation algorithm based on the minimization of a Ixast Squares loss function must be also emphasized,
RECURSIVE ESTIMATION OF TIME DELAY
! 73
namely the fact that the loss function is in general multiextremal with respect to the time delay [26-27], so involving the risk that the recursive algorithm gets stuck in a local minimum. This fact may clearly occur independently from the particular recursive algorithm used, unless a suitable data filtering technique is adopted. B. EMVRE: ESTIMATED MODEL VARIABLE REGRESSION ESTIMATOR (ELNAGGAIL DUMONT AND ELSHAFEI, 1989, [28]) Apart from being nonlinear, the performance criterion (11) is actually a discrete function of d so that, defining with O' the vector of the parameters of the pulse transfer function: O ' = [ a I ... a n b 0 ... bn]T,
the solution of the minimization problem should be more properly defined as the one that satisfies the following equations: /tJNTD/(Or, d)
00'
=0
(13.1)
mintJNTDi(O',d)l V d e [ d~., d
(13.2)
I
Consider now that, for a fixed d, the solution of r
(13.1) can be found
through any recursive Least Squares technique. This fact can be suitably exploited in devising an efficient two-steps algorithm [28], requiring a minimum additional computation and data storage with respect to standard recursivr algorithms. In the first step a recursivr algorithm is used to update the parameters 0', with d fixed to the value estimated in the last sample period; in a second step the estimate of the delay is updated by solving eq. (13.2) through a search routine, being the loss function recursively computed for different values of d, assuming the A last estimate 0'(k) is correct: A
A
J(k,d) = ~,J(k-
A
1,d) + [y(k) - ~ k , d ) T O'(k)]2
(14.1)
174
GIANNI FERRETTI ET AL.
A
A
(14.2)
d(k) = arg min[d(k,d)] V d ~ [ d m, dmax I
where ~ is a forgetting factor. A two-steps algorithm is also proposed by Pupeikis [27], where however the delay, instead of being determined by (14), is recursively estimated through a gradient-like algorithm, similar to the one described in the previous section. Moreover, a complex adaptive filtering technique is adopted to transform the multiextremal criterion into a unimodal function of the time delay. C. OTHER ALC~RITHMS The identification of the parameters of the continuous-time system is directly dealt with by Zhao et al. [29], considering the following model in the continuoustime domain: ?1
tl
cP-iy(O= ~ o~
dny(O + dt n
Or
dtn-i
i=1
d~-~u(t-x) dtn-i
(15)
9
i=1
Performing a multiple integration on both sides of (15) and approximating the continuous-time integration as a function of the sampled measurements through the linear integral filter defined in [29,30], yields: tl
9n~k)
tl
+E ~ i=l
where
9n-Y(k)=
E [J~9...,u(k- "t'*) i=l
nl .
j=o.
i=O i
I is the length factor of the linear integral filter, the coefficients pj are defined [30] according to the adopted formula of numerical integration (trapezoidal, Simpson, etc.) and x* = x/h = d' + e. The estimation of the time delay z = Z* h and the
RECURSIVE ESTIMATION OF TIME DELAY
175
parameters ~ , /3j is then performed by minimizing, via Newton's algorithm, the following LS performance criterion:
JNTDI~O) = 1 E[e(k,d)2] = I E{ [~twv(k)- ql(k)T 0]2} '
o = [
... %
...
i r,
Some bias-eliminating techniques are also proposed by Zhao et al. in [29], in order to deal with noisy measurements. Finally, a rather complex algorithm, based on a Bayesian approach and on the estimation of a set of different models, each one related to a different value of the delay, has been proposed by Juricic [311. VI. M E T H O D S B A S E D O N R A T I O N A L A P P R O X I M A T I O N S A. BASIC CONCEPTS The fact that systems described only by rational transfer functions are much easier to handle than time delay systems, from the point of view of both identification and control system design' has motivated a number of algorithms based on rational approximations of the delay transfer function. Three different forms of rational approximations have been considered in the literature: all-pass, all-poles and all-zero transfer functions. In particular, Salgado et al. [32] showed with a detailed analysis the superiority of the all-pass or Pad6 approximation:
e-rS =
I~ L - s ~)L L +s '
(16)
with respect to all-poles approximations, in obtaining a good fit of the delay frequency behavior. A perfect matching in magnitude can in fact be obtained through (16) over the whole frequency range, while the accuracy in matching ~ e
176
GIANNI FERRETTI ET AL.
delay phase response may be arbitrarily increased by increasing the order L of approximation, at the expense however of an increased model order. Just to avoid model order augmentation, Roy et al. [33] proposed the adoption of an aU-zero approximation, giving also some criteria to choose the order of the approximation, so as to limit the gain and phase error over a frequency band of interest. On the other hand, by adaptively modelling through (16) only a small uncertain fraction of the overall delay affecting the system (as shown in the algorithms explained below), the phase error can be maintained limited over a wide frequency range, even with low-order approximations. An other major problem affects however the approaches based on rational approximations, namely the fact that the overall identification model, obtained by cascading two separately rational transfer functions (i.e. the delay approximation and the rational part of the system), is again rational but anyway nonlinear in the parameters. This fact, when an explicit estimation of the delay is required, imposes the adoption of nonlinear estimation algorithms [34] or, at least, the solution of a nonlinear problem, once parameter convergence has been reached adopting simpler identification algorithms [35]. Clearly, this problem does not hold when only a good fit of the overall model to the process frequency response is required over the bandwidth relevant to control, as in the case of adaptive control strategies. In fact, in the case of adaptive pole-placement control [36], rational approximations in the form (16) have been proven to give good results with standard linear parameter estimation algorithm, even with L = I or 2. B. DPDIA" DISCRETE-TIMEPADI~-BASED DELAY/DENTI~CATION
ALGORITHM (BJORKLUND, NIHTILA, AND SODERSTROM, 1991, [34]) In this algorithm the Pad6 approximation (16) is separately discretized from the rational part of the system. Among the various alternatives to transform (16) into a discrete-time model (ZOH-sampling, Elder's method, etc.) the bilinear transformation is here preferred, since it maintains a unit gain for all frequencies. Therefore, replacing the argument s in (16) with 2 (z - 1)/h (z+l) the following discrete delay approximation is obtained:
RECURSIVE ESTIMATION OF TIME DELAY
N(z-l,p)
(17.1)
kl + p z - l J
Lh-z P="Lh+z
177
9
(17.2)
The overall model (18), defined below, is then considered by BjOrklund et al. [34]"
M(z- 1,p) B(z- 1) u(k - ~ + C(z- 1) v(k),
A (z-I) y(k) = N(z.q,p)
(18)
where d is an a-priori known part of the total delay, v(k) is assumed as zero mean, white noise and the parameter p accounts for the uncertain part of the delay to be estimated. As expected, ff an estimation ofp separate from the other parameters is required, model (18) results rational but nonlinear in the parameters. Anyway, resorting to nonlinear algorithms (in [34] a re.cursive algorithm that minimizes the error prediction model [23] is adopted), an estimate ~ of the delay can be obtained A
A
from an estimate of the parameter p as z = d h + L h (1 - ~)/(1 + ~). As already mentioned, the Pad6 approximation (16), for large values of x, is accurate in phase only for low frequencies, unless a high order L of approximation is adopted; on the other hand, when z = h L an exact description of a delay corresponding to L samples is obtained from (17), i.e. M(z-1)/N(z -1) = z -L. Exploiting this property and allowing the a-priori estimate d to be time-varying, it is possible, with a slight modification of the algorithm, to reduce the model error. Defining z = h d + zp, on the basis of the previous considerations it is desirable to have xp = h L, and this can be achieved through the following steps, to be performed ^ateach sample: 1. Choose d = d(k), being d(k) defined by the following minimization problem: A
d(k) = a r g m i n [h L - "~(k- 1) + m h] 2
2. Determine an initial condition ~~
,
'ff m > 0
from ~(k) = ~ k - 1):
178
GIANNI FERRETTI ET AL.
~0
^0 ^ ^ (k) = ~ p ( k - 1) + d ( k - 11 h - d(k) h
,~~
^0
^0
= [L h - "rp(k)l/[L h + "rp(k)l
3. Update the parameter vector according to the chosen algorithm. A great emphasis has been also given in [341 to the problem of multiple minima in the loss function, in particular, the following remedia have been proposed to face the problem: a) low-pass filtering of the data before the identification; b) the use of high order approximations (L > 1); c) the use of a model structure where the noise model does not depend on the delay approximation. C. CPDIA" CONTINUOUS-TIME PADI~-BASED DELAY/DENTIFICATION
ALGORITHM (AGARWAL AND CANUDAS, 1987, [351) A linear, continuous-time system in a form equivalent to (15): I'1
Y(O +
Z
m
~
, a"y(O Z dt i
i-1
' a~u(t-~)
Oi
"~
, n >m
(19)
.
i=0
is considered by Agarwal and Canudas [35], which recall an early work of Robinson and Soudack [37]. They consider the total time delay x as splitted into an a-priori known part u and an unknown part zp, the latter being approximated in the frequency domain by a rational transfer function of the form (16) (with L = 1), so that the Laplace-transform of both members of (19) can be written as:
1+
~. i=1
Y(s) +
=
fi s i U(s) e -'~s .=
P
+ ~u
,
(20)
1 + 0 . 5 ~p
where *F~ and ~u collect all the terms due to the initial conditions of the i-th derivatives ofy(O and u(O (see [35]).
RECURSIVE ESTIMATION OF TIME DELAY
179
Eq. (20) can not be however exploited for parameter estimation after simply reverting to the time domain through inverse Laplace transformation, since the time derivatives of input and output variables can not be measured in general. A multifilter technique is therefore introduced, applying a dynamic stable operator F(s) 3 to both members of (20) which yields: r/
ro(~) + (u, + 0.55) r,(~) + y ' (~; + 0.55
~;_,) L(~) + o.55 u. r +,(~) + 2 % =
i=2 m
= flo Uo(s) e-'~s + Z
([~ - 0.5~p fl'i-,) U.(s> e - ~ s - 0.5~p ~'m Um+l(S) e-~S + Z~(s),
i=1
being
Y.(s) = s i Y(s) F(s) , Z~
Z~ F(s) (1 + 0.5 "rp s) , i = 1.... ,n
Ui(s) = ss U(s) F(s) , Z ~ ( s ) = X u0F ( s ) ( l + O . 5 ~ p s )
, i = 0 .... ,m
, ;
reverting now to the time domain by inverse Laplace-transforming one obtains n+l
m+l (21)
J=l
i=0
where )!
!
cf
~ = ~ + 0.5 ~ ov
!
!
!
v!
!
, a~ = ~ + 0.5 ~ a~_~ , ~,+~ = 0.5 ~ ~ . to
!
!
/~o =/~o , ~, = / ~ , - 0 . 5 ~/~,_~
)!
!
, /~m+] = - 0 - 5 ~ p / ~
3The authors propose the following expression for the filter F(s):
cf F(s) = (s + c)f with f'd_ n + 1
(22.1)
;
(22.2)
180
GIANNI FERRETTI ET AL.
finally, a discrete-time model suitable for recursive parameter estimation is obtained by sampling the filtered signals: n+ 1
m+ 1
Z yo(k)
= -
y
tt
(k) ,
i=1
(23)
.
i=0
Note that the terms ~-l{~y(S)} and ~-l{~u(s)} in (21) vanish as the time increases, provided that F(s) is stable, so that they can be properly neglected in (23); moreover, for the sake of the recursive identification, the a-priori known part of the delay u should be specified as an integer multiple of the sampling period. Two estimation algorithms have been proposed, both based on the minimization of the same least squares criterion:
JCPDIA(O)=N
Yo (k) + k=l
ai Yi(k) i=l
fli u i ( k - u
.
(24)
i=O
but characterized by different definitions of the unknown parameters vector O. In a first algorithm the vector 0 is defined as 0 = [ ~' /~i~]T so rendering the criterion (24) quadratic with respect to 0 and allowing for the adoption of a standard RLS algorithm; once vector 0 has converged to its true value, the original parameters ~, and Zp can be obtained through a nonlinear transformation, defined by (22). In particular, tlus step can be performed by searching the common root of the following l~lyno~als in z 9 P
n+l
i=1
y',, m+l
fli (0.5 y ? m + l - i = 0
i=0
(25.2)
RECURSIVE ESTIMATION OF TIME DELAY
181
and then back substituting in (22) to obtain parameters ~. and fl't (note that in transient conditions the solution of (25) may not even exist or produce wrong values of zp, ~ and ~, even starting from a reasonable estimate of ~: and fli~). In a second algorithm, the identification of the parameters O = [ ~
]3'i ~ ]r is directly
dealt with, therefore, since parameters ct~ and/3~ in (24) are nonlinear functions of O, recursive nonlinear least squares estimation algorithms must be adopted [38]. In order to improve the tracking capabilities of both algorithms, the authors then propose a stepping mechanism (similar to the previous one of BjOrldund et al.) on the a-priori known part of the delay. First, the convergence of the parameters is detected at instant k , according to some user-specified criterion 4, then the values of u and zp are updated as follows: u
+ 1) - u
z p ( k + 1) =
A
+ int[zp(k)/h + 0.51 h ,
) - int[
)/h + 0.51 h
This mechanism, in that it improves accuracy by reducing the part of the delay subject to an approximation, allows convergence to unbiased estimates. The authors give also in [35] some hints on the choice of the cut-off frequency of the filter F(s) and of the input signal u(t), present some procedures in order to detect and recover the case of multiple solutions, possibly occurring in the case of non-minimum phase systems, and show the robustness of the algorithms with respect to high frequency measurement or process noise.
4The authors suggest to test the absolute relative change over two succesive instants for each of the estimated parameters, i.e. convergence is inferred at instant k c if
A
<e Vi
182
GIANNI FERRETTI ET AL.
D. OTHER ALGORITHMS The adoption of a first-order Pad6 approximation for the delay and of a filtering technique to emulate system input and output derivatives has been also proposed by Gabay and Merhav [39]. Their algorithm is particularly suited for closed-loop identification but involves a considerable computational effort, requiting a search procedure among several estimated models in order to identify the correct delay. Gawthrop and Nihtil~i [40] proposed the adoption of an all-poles approximation for the identification of a single time delay (no other dynamics is considered). The polynomial identification method of Nihtil~i is then applied to the resulting approximate model, which results a polynomial in the parameter ~:. The identification of a single delay, based on a first-order Pad6 approximation, is also considered by Bojr and Eitr
[41], which also put into
evidence the band requirements on the excitation signals, namely the fact that for the identification (based on the said approximation) to be successful the input and output signal bandwidth must be limited to about 2/x rad/s. A Luenberger observer and a Kalman filter are then proposed as identification algorithms.
VII. V A R I A B L E R E G R E S S O R M E T H O D TIDFA: TIME D E L A Y ESTIMATION A L G O R I T H M (FERRETH,
MAFFEZZONI A N D SCATTOLINI, 1991, [42])
The key idea of the method consists in noting that, for a given sampling time, fractional delays or anticipations in the continuous time model, give rise to real negative zeros in the discrete time sampled system. Thus, by inspection of the phase contribution of these discrete time zeros, it is possible to infer the value of the continuous time fractional delay. The main steps in the derivation of TIDEA are the following. First, provided that Hr(s ) does not have any zero on the imaginary axis, there exists a polynomial
RECURSIVE ESTIMATION OF TIME DELAY
183
Dr(S) of suitable order n such that 1/Dr(S) approximates Hr(S) with any desired accuracy over the imaginary axis interval [-jcob,/%]. Then, replace H(s) with H(s)=e -'rS/Dr(S). Discretization of fI(s) generates the so-called sampling zeros, which, for h--90, tend to predefinite points in the complex plane [42]. As shown in [37], these zeros can be given a low frequency approximation with a suitable number of zeros at the origin. When also the delay term e-rs is considered in the discretization of H(s), an approximate discrete time model of the system is ^ (Z-Z')(Z-Z") z'(d+l)= b~ H(z) =
A(z)
z -(d+l)
A(z)
where Ar(Z) is the denominator....fpolynomial of order n, d is the maximum integer
less than or equal to ~ - ~ - ) and z', z" are the two zeros, to be estimated on line, which are related to the fractional delay value as follows: 1
1-z' + ~
1
1-~ A
Then, the estimation of the parameters of the approximate model H(z) directly leads to the value of the fractional delay ~. When the delay uncertainty is larger than the adopted sampling period, in principle one could enlarge the sampling period itself to cover the delay uncertainty. This could be very impractical if the time delay uncertainty is large, especially when the estimated model is used for control purposes. Alternatively, since widening the sampling period with a ZOH is equivalent to data low-pass filtering, one can also maintain the adopted small sampling period and process the signals used by the estimator with an appropriate low-pass filter L(z). In view of the above considerations, the algorithm TIDEA consists of the following steps.
184
GIANNI FERRETTI ET AL.
TIDEA A
A
Step..Olfflfialize the parameter estimate (including d) of H(z), written in equation A
A
A
A
error form, such that the estimate e of e is equal to zero; that is b 0--b2=0, b l;e 0.
Step 1 At any sampling period, prefilter with L(z) the system input and output data, then update the estimates of the unknown parameters bo, b s, b 2 and the ones of A r by means of any recursive estimation algorithm. A
A
A
A
A
A
Step 2 Compute z' , z" and e=(b o-b 2)/(b o~ l+b2) A
Step 3 If --e< ~'2, where ~ is "something less than one", typically g'--0.8, go to step 5, otherwise go to step 4. A
A
A
A
A
A
Step 4 Repeat the following steps until ]et<~: if e<-~ then set d-,d-1 and e-,e+l; if A
A
A
A
A
A
e>~ then set d ~ d + 1 and e~e-1. Furthermore, if e<-~ then give the estimates of b o, b l, b 2 the following new values A n
A
A
A n
^
^
A n
A
bo=bl+3bo ' bl=b2--3bo ' b2=bo A
Conversely, ff ~.>~, then set A n
A
A n
^
^
A n
A
A
b o=~ 2 ' b l=b o"3~ 2 ' b 2=b l+3b A
Step 5Update the estimate z of the time delay as follows: ^ ,=
~_^n-1 h (e + +-T )
Go to step 1. Note that the rationale behind Step 4 is as follows: ff the estimated fractional delay or anticipation -eh tends to one sampling period (1~1>o.8), the delay estimate is increased or decreased of one step and the numerator coefficients are modified in order to guarantee that the first three terms of the series expansion around z - 1 of the new and old transfer functions coincide. For the algorithm convergence, the role of the low-pass filter L(z) is crucial (as discussed in [42]).
RECURSIVE ESTIMATION OF TIME DELAY
185
VIII. S I M U L A T I O N E X P E R I M E N T S
The experimental set-up In order to test the performance of the methods described in the previous Sections, some specific rccursive algorithms for delay estimation were implemented and many simulation results wcrc collected. In the following, the results relevant to two systems, hereafter referred as S1 (system 1) and $2 (system 2), will bc presented and discussed. The transfer functions G(s) of S1 and $2 are: 1 S 1" G(s) = 'l+80s e ~
1-20s S2: G(s) = (i +s)(l + 100s) c3~ Systems S1 and $2 arc representative of the dynamics of typical processes with time delay. In particular, the structure of S 1 is that considered in the ZieglerNichols tuning rules for PID regulators, while $2 is characterized by a real positive zero which produces an undershoot in the output step response. Note that in both cases, the magnitude of the delay is comparable with that of the time constants. Furthermore, in $2 the negative phase contribution of the delay and of the zero at low frequency are roughly equivalent; this indeed makes complex the estimation of the true delay in $2. In all the algorithmic implementations the correct model structure was used and, unless otherwise specified, the Rccursive Least Squares algorithm was employed with constant forgetting factor 2, [23]. A significant effort was also made to select the value of L , as well as the value of all the parameters in the various algorithms, in order to guarantee the "best" performance in the simulation runs considered. Save for the simulations performed to test the correlation methods, the reference signal was selected as a square wave of magnitude 1 and frequency f, again the value o f f was chosen so as to get good performance. All the initial values of the estimated parameters wcrc set to 0, while the initial delay value d~n and the minimum dram and maximum dmax delay values, if required, wcrc chosen from time
186
GIANNI FERRETTI ET AL.
to time. The initial covariance matrix of the estimates was set to the identity matrix of appropriate size. The integration time was set to Is for S 1 and to 0. ls for $2, while the sampling times h~ and h2 for identification were varied as specified in the following. Note that a variation of h~ and h~ can also be viewed as a change in data filtering. For any algorithm and for any system, the three experiments described in the following were carried out.
Experiment 1 Purely deterministic, as previously described.
Experiment 2 In order to analyse the sensitivity of the estimation algorithms to the presence of noise corrupting the data, a white noise signal with zero mean value and standard deviation p.=0.01 was added to the system output. Note that, while the true model has an output error form [23], the RLS algorithm was used with reference to an equation error model. This was done to test also the robustness of the methods with respect to a plant/model mismatch.
Experiment 3 Same as in experiment 2 with ~t=0.05.
EXTENDED B-POLYNOMIAL METHODS A. BEBA: BASIC EXTENDED B-POLYNOMIAL ALGORITHM S1
Experiment 1 The free design parameters were chosen as follows: h1=8s, L=0.98, dmi,=l, f~.00125Hz. Eight B-polynomial coefficients were estimated, considering non zero those with magnitude greater than or equal to the threshold value 11--0.0001.
RECURSIVE ESTIMATION OF TIME DELAY
187
The estimated value of d is reported in Fig.3, which shows that the correct delay value d=-5 is achieved (the true delay is d*hl=5*8s).
S 1 - estimated delay with BEBA
30
_
20
10 _l' 0
0
I000
2000
'
3000
4000
time (s) Fig. 3 System $1 - estimated delay in experiment I Experiment 2 In the noisy case, the possibility of determining the correct delay estimate is strictly related to the threshold r I considered in judging whether an estimated bi coefficient is zero or non zero. In order to illustrate this, Fig 4 reports the transients of the estimated parameter b 4 in experiments 1 and 2. Recalling that the estimated value of b 4 should be zero and comparing the estimates to the adopted threshold value + rl, these results clearly illustrate that in the noisy case the proper delay value is not identified by BEBA. Fig 4 also puts into evidence the difficulty in tuning the parameter r I for a correct delay estimation. Experiment 3 The same conclusions as in experiment 2 apply with much greater evidence.
188
GIANNI FERRETTI ET AL.
5
x l O .3 S1 - e s t i m a t e d b 4 c o e f f i c i e n t
] 0
[
l
J
I____
,
-5 I000
2000
3000
4000
time (s) Fig. 4 System S] - estimated b4 coefficient in experiment 1: continuous line, estimated b4 coefficient in experiment 2: dashed line.
$ 2 - e s t i m a t e d d e l a y w i t h BEBA
50
_
0
,
|
200
,
400
time
-
600
.
_
_
800
I000
(s)
Fig. 5 System $2 - estimated delay in experiment 1
$2 Experiment 1
The free design parameters were chosen as follows: h2=ls. X=0.98. d,,..=l, f--O.00125Hz. Fourty B-polynomial coefficients were estimated, considering non zero those with magnitude greater or equal the threshold value TI-------O.0001.
RECURSIVE ESTIMATION OF TIME DELAY
189
The estimated value of d is reported in Fig. 5, which shows that the correct delay value d=30 is achieved (the true delay is d*h2=30s ).
Experiment 2 and 3 The same considerations reported for system S1 apply here; that is, in the noisy case, the proper estimation of the delay value is quite impossible. B. ADTE: ADAPTIVE DEAD-TIME ESTIMATOR
S1 Experiment 1
The free design parameters were chosen as follows: hl=ls, )~=0.9, drain=l, f=0.00125Hz. Moreover, the procedure suggested in [81 to update the delay value was activated every 8 identification steps. As a result, the true delay value d=40 was identified after about 6300s, as shown in Fig. 6.
:51 - e s t i m a t e d
40
delaywith A D T E
J 20
[__g-
O 0
2000
4000
time
6000
8000
1OO00
(s]
Fig. 6 System S1 - estimated delay in experiment I Experiments 2 and 3
In this case, all the simulation runs were unsuccessful, even though many different settings of the free design parameters were considered.
190
GIANNI FERRETTI ET AL.
$2 All the experiments were unsuccessful, even in the deterministic case.
CORRELATION METHODS A: DECA" DELAY ESTIMATION BASED ON CORRELATION ANALYSIS According to the algorithm formulation of Section IV, in all the experiments the input was chosen as a white noise sequence with zero mean and unitary standard deviation. In this case too, as in the BEBA method, the delay estimate can be obtained as the least time argument at which the cross-correlation function rye(T) is different from zero. Then, instead of fixing a threshold, it is preferred in the following to show the cross-correlation function at the end of the simulation run.
SI 9
0.015
-
crosstmrrelat,_'on .ryu.
0.01 0.005
, O
,
9
,,
,,.
0
20
40
60
80
1
I00
d Fig. 7 System S1 - crosscorrelation in experiment 1 S1
Experiment I The free design parameters were chosen as follows: hl=ls, drain=l, dm~x=100, while the true delay value to be estimated is again d=40. The estimated cross-correlation
RECURSIVE ESTIMATION OF TIME DELAY
191
is reported in Fig. 7, which clearly shows the discontinuity of the function r~ corresponding to the true delay value. Experiments 2 and 3
The crosscorrelation functions ryu computed in these experiments are reported in Figs. 8 and 9. It is apparent that the method is largely insensitive to the presence and amplitude of the noise.
S1 - erosscorrelation
0 . 0 2 ~-
.rN..
0.01 0
-O.O 1
,,
O
|
20
.....
i
,
,
40
i
60
.
i
,
80
100
d Fig. 8 System S1 - crosscorrelation in experiment 2
$2 Experiment 3
Only the results of the hardest experiment on $2 are now reported. The free design parameters were chosen as follows: h2=O.ls, dm~,=l, dm~--600, while the true delay value to be estimated is d=300. It is apparent from Fig. 10 that the estimated crosscorrelation has a spike corresponding to d=-300. From these results it is possible to conclude that the method based on correlation analysis may lead to the correct estimation of the delay and is quite completely insensitive to the noise acting on the output. However, the system excitation must be a white noise.
192
GIANNI FERRETTI ET AL.
.SI - r
0.02
.at.ion .ryu.
0.01 0 -0.01
0
20
40
60
80
100
d Fig. 9 System $1 - crosscorrelation in experiment 3
O. 0 1
$ 2 - ~. o ss<x>rrd _~•
.ryu
0
-0.01
-0.02 0
200
400 d
Fig. 10 System $2 - crosscorrelation in experiment 3
600
RECURSIVE ESTIMATION OF TIME DELAY
193
NONLINEAR LEAST SQ UARES METHODS
A. NTDI: NONLINEAR 7IME DELAY/DENTIFICATION
S1
Experiment 1 The free design parameters were chosen as follows: hl=8S, L=0.9, drain=20, f--O.O 1Hz. The delay estimate, reported in Fig. 11, shows that the true delay value d=-40 is achieved in about 5800s. The evolution of the estimate is quite slow, but essentially monotone.
S1 - e s t i m a t e d d e l a v w i t h NTDI
40
~lid
9 ,,,
'
'1
9
-
9
...........
30
!" 0
-
0
, i
2000
i
4000
6000
8000
time (s) Fig. 11 System S1 - estimated delay in experiment 1
Experiment 2 With the same setting of the free design parameters as in experiment1, convergence towards the true delay value is still essentially achieved, see Fig. 12, although the presence of noise makes the estimated value fluctuate around the true one.
Experiment 3 Unsuccessful, we were unable to obtain convergence of the algorithm to any estimated delay value.
194
GIANNI FERRETTI ET AL.
S l - e s t i m a t e d d e l a y with.NTDI
50 40 30 20 0
0.5
I
t i m e (s)
1.5
2
xlO 4
Fig. 12 System S1 - estimated delay in experiment 2
$2 In this case, experiments 1, 2 and 3 were unsuccessful, even though many simulation runs were performed with different tunings of the free design parameters.
B. EA4I/7~: ESTIMATED MODEL VARIABLE REGRESSION ESTIMATOR
S1
Experiment 1 The free design parameters were chosen as follows: hl=lS, ~=0.94, drain=l, dmax=50, f--O.O1Hz. The delay estimate is reported in Fig. 13, which shows the extremely quick convergence to the true delay value d=-40.
Experiments 2 and 3 These experiments were unsuccessful. This may lead to conclude that the surprisingly good performance in the deterministic case must be paid by a very small robustness with respect to the presence of noise.
RECURSIVEESTIMATIONOF TIME DELAY
50
195
$1 - e s t i m a t e d d e l ~ w i t h EMVRE
o__J0
200
400
600
time
800
I000
(s)
Fig. 13 System $1 - estimated delay in experiment 1
$2
Experiment 1 The free design parameters were chosen as follows: h2=O.ls, ~=0.94, dmin=lO0, dmax=600, f=0.025Hz. As depicted in Fig. 14, the true delay value d=300 was estimated very quickly.
$2 - es)imated d d a y w!th I}MVI~.
400 300 200
100
0
50
I O0 time
150
(s)
Fig. 14 System $2 - estimated delay in experiment 1
200
196
GIANNI FERRETTI ET AL.
Experiments 2 and 3 In this case too we were unable to obtain convergence of the delay estimate, thus confirming the results of Experiments 2 and 3 associated to S 1.
METHODS BASED ON RATIONAL APPROXIMA TIONS
A. DPDIA" DISCRETE-TIME PADI~-BASED DELAY /DENTIFICATION ALGORITHM
SI
Experiment 1 The free design parameters were chosen as follows: h l=lS, L=0.98, din=20, f~.00125Hz. The delay estimate is reported in Fig. 15; it is apparent that the true value d=40 is identified after about 10000s.
50
-
SI
-estimated
delay wi.thDpDIA r
it a
,
0
l
i
,
=
O.S
i
1.5
1 time
{s)
xlO 4
Fig. 15 System $1 - estimated delay inexperiment I
Experiments 2 and 3 These experiments were unsuccessful.
RECURSIVE ESTIMATION OF TIME DELAY
197
$2 The free design parameters were chosen as follows: h2=O.ls, L=0.96, din=150, f~.0016Hz. The delay estimate d=-329 was obtained after 4000s. Note that in this case the true delay value d=-300 tends to be overestimated, due to the nonminimumphase characteristics of the system $2, see Fig. 16.
$2- estimat~
400
delaywith DPDIA
200
0 0
I000
2000
3000
4000
time (s) Fig. 16 System $2 - estimated delay in experiment 1
Experiments 2 and 3 These experiments were unsuccessful.
VARIABLE REGRESSOR METHOD
TIDEA: TIME DELAY ESTIMATION ALGORITHM
S1 In this case, the initialization of the discrete time model was chosen as the discretization of the continuous time system described by
198
GIANNI FERRETTI ET AL.
1 G(s) = l + 3 0 0 s Furthermore, it was set h~=2s, the RLS method was implemented by keeping constant to l O0 the trace of the covariance matrix and the filter L(z) appearing in TIDF~ was chosen as the discretized version of a continuous time third order Butterworth filter with cutoff frequency O.078rad/s.
Experiment 1 The results of experiment 1 are reported in Fig. 17, which shows that the continuous time delay estimate after 3000s is about 39.5s and that the correct estimate is practically achieved after about 800s.
S l - e s t i m a t e d d d . a y w i t h TIDEA
50
J
,
0
j
t
1000
2000
time
3000
(s)
Fig. 17 System S1 - estimated delay in experiment I
Experiments 2, 3 Both experiments were successful; so, only the most severe experiment 3 is reported in Fig. 18. The estimate of the continuous time delay at the end of the experiment is about 38.4s. This demonstrates that the robustness of the method with respect to additional noise is quite good.
RECURSIVE ESTIMATION OF TIME DELAY
$1 - e s t i m ~
50
,
0
5OO
199
d e l a y w i t h TIDEA
,
i
1000
,
i
1500
2000
t i m e (s) Fig. 18 System S1 - estimated delay in experiment 3
50
$ 2 - e s t i m a t e d del
w i t h TIDEA
0 0
2000
4000
6000
t i m e (s) Fig. 19 System $2 - estimated delay in experiment 1
$2 The initialization of the discrete time model was chosen as the discretization of the following discrete time system:
200
GIANNI FERRETTI ET AL.
1 G(s) = (1 +s)(1 +3 00s) The estimation sampling time h2, the RLS algorithm and the filter L(z) were kept the same as for system S 1.
S ~ m a g r f l t u d e of zthe true and estimated model I
0 -50
10-4
l o l I I I I | I I I I I I I *
I
l l I l I I I , I I I I I I I J
I I I .... l l I I I I I
I I I I I I I *
I l I l I i I
l l I l I i I
e e I e i i I
l II l lO l ll i Ii i II i ii III
I I I I I I I I
I I I I I I I I
I I I I I I I II
I II I II III I II I II I II III jj
I
e l l l I I I
9 w i illw
w
I I I I I I I
I
10-3
I
I
I I I IIII I
I
I I I I I
I I I II I I I III I I I I II I I I III I I I III . - - . - - ~ I I I III I I I III I I I III I I I III I I I III I I I III I I I III i i i i ii
.....
I I I I I i I I I I I I I l
I I I I I I I i
I
I III
,,,,
I
I
,
,, ......
I
I I I I I
I
I I I I" I
I I I I I I I I I -- "II --i-_ I I I I I I I I I I I I I I I I I I .. I i
I
I
I II
I I I I I I" I
I I I I I I" I
I II I II I II I II I II l'eT I II
I I I I I i
l'Vlt'l,,4. I I II I I II I I II I I II i i i i
10-2
10 -1
|,|,l
frequ~
~2 -phase of the true and estimated model
-200
[
[~ : I ; ; ; ; Qi
I i i i i i i
I i i i i i i
I I i i i i i
I i i i i i i
I i i i i i i
I i i I i i i
III i ii i ii i ii iii i ii i ii
"
~
I i i i I i i
I i i I I i i
I i i i i i i
I I
i I
I I
I I
I I
i iii I I II
I I
I I
I I
|
|
"Irl-ll.~_lll i i i i i i Yl~Mlll,l._ i i i i i i iii - ' ~ i i i i i i iii 1 ~ i i i i i iii i-"%ii4~ i i i i i ii i ~ i i i i i ii i i -~i
[ [ : [[
J i i i I
i i i i i i i
I I
I I
-400 10-4
I I I
I I I
I I I
I I I
I III I I II I I II
10-3
I I I
I I I
I I I
I I I
I I I
|
I III I III
I I
I I
I I
I I
I
I III I I II I III
I I I I
I I I I
I I I I
I I I I
I I I I
. . I. .I .I .I I. I .II . . . I. . I. .I .I I. I. III. . . .
I I I I
[
10-2
ii i i ii ii ii i i
ii l llJ
I I I I
~,'I I~I~ II.11 II| I I
10-1
frequency Fig. 20 System $2 - Bode diagrams of the true (continuous line) and estimated (dashed line) model
Experiment l
The result of system 1 is reported in Fig. 19, where the continuous time delay estimate at the end of the experiment is about 43s. In order to evaluate the performance of TIDEA, it is worth recalling that this method is based on an allpoles approximation of the rational part of the true sampled transfer function. Since
RECURSIVE ESTIMATION OF TIME DELAY
201
the model used here is simply of second order, the presence of a large nonminimum phase zero in $2 is indirectly accounted for by overestimating the time delay. The comparison between the frequency responses of the true and of the estimated models (Fig. 20) clearly illustrates the global accuracy of the estimate.
Experiment 3
Even in this case, experiment 2 is omitted because experiment 3 is still successfull and more significant. The delay estimate at the end of the experiment, shown in Fig. 21, is about 42.5s. Thus, the noise has not any practical effect on the estimate accuracy.
50
$2 - e s t i m a t e d d e l a ,,-
O 0
with TIDF~
w
2000
4000
6000
time (s) Fig. 21 System $2 - estimated delay in experiment 3
IX. COMPARISONS AND CONCLUDING REMARKS In the introduction of this survey, many reasons have been recalled which motivate the development of efficient and reliable recursive methods for the estimation of time delay in discrete-time systems. Then, the main approaches proposed in the literature have been presented together with the analysis of some specific algorithms. Finally, some simulation results have been reported to illustrate the performance of these techniques. It now comes natural to wonder whether some of the algorithms here considered can be of help in the fulfillment of the initial goals.
202
GIANNI FERRETTI ET AL.
To this regard, it is ditiicult to draw definite conclusions from simulation results. However, we believe that some guidelines can be identified for the proper use of the methods here examined. In particular, it seems apparent that correlation methods guarantee excellent performance even in the noisy case provided that the input signal is correctly chosen and the system operates in open loop. Conversely, in closed loop operations this approach can only be used provided that some very fight assumptions are made. Then, correlation techniques can be considered in a preliminary analysis or, e.g., for diagnosis purposes. As a further conclusion, we can say that the methods based on the overparametrization of the B-polynomial in general fail to be successfull for the proper delay detection, since they require to discriminate the zero/non zero pattern of the estimated coefficients, which is a hard task whenever some noise acts on the system. However, if the model has to be estimated for control synthesis purposes and the proper delay estimation is not strictly required, what really matters is that the frequency response of the estimated system matches the true one, at least in the frequency band of interest. In these cases, even the extended B-polynomial method, see [101-[131, as well as other techniques, can be of help. Nevertheless, if a fixed structure PI or PID regulator has to be synthesized from the estimated model, an overparametrized B-polynomial is not the best starting point, while a first or second order model plus a time delay is to be preferred. From this point of view, other algorithms, e.g. TIDEA [6], [421, should be considered. Most of the methods here analysed have shown to be extremely sensitive to the presence of noise; as such, some of them, e.g. EMVRE, although very fast and of high performance in the noise-free case, must be used with great care in practical situations. In general, robustness and performance are conflicting properties, as quite usual in identification (compare the results of NTDI and EMVRE). In order to achieve robusmess and reliability with respect to the presence of noise, we believe that a significant filtering action must be employed in all practical circumstances, as illustrated by the results reported for TIDEA. This has recently been demonstrated for any estimation technique based on Least Squares minimization in [431, where it is shown that only filtering can eliminate local minima in the performance index to be minimized. Finally, it has to be recalled that the aim of this survey was only to present and discuss recursive estimation techniques, while in the literature other off-line methods have been proposed for the delay estimation, see [44]-[50]. However, most of these approaches can still be plugged into one of the basic categories presented here.
RECURSIVE ESTIMATION OF TIME DELAY
203
X. R E F E R E N C E S
G. Ferretti and C. Maffezzoni, "Monitoring and Diagnosys of a Pulverization Process", 1FAC/1MACS Symposium SAFEPROCESS'91, Baden-Baden, Germany, pp. 199-204 (1991). O. J. M. Smith, "Closer Control of Loops with Dead Time", Chem. Eng. Progress, 53, pp. 217-219 (1957).
.
3.
K. Yamanaka and E. Shimemura, "Effects of Mismatched Smith Controller on Stability in Systems with Time-delay", Automatica, 23, pp.787-791 (1987).
4.
C. Santacesaria and R. Scattolini, "Easy Tuning of Smith Predictor in Presence of Delay Uncertainty", Automatica, 29, pp.1595-1597 (1993).
.
K. J. Astrom and B. Wittenmark, Adaptive Control, Addison Wesley, Reading, MA (1989).
6.
A. Leva, C. Maffezzoni and R. Scattolini, "Self-Tuning PI-PID Regulators for Stable Systems with Varying Delay, Automatica, 30, pp.1171-1183 (1994).
7.
H. Kurz and W. Goedecke, "Digital Parameter-Adaptive Control of Processes with Unknown Dead Time",Automatica, 17, pp. 245-252 (1981).
8.
R.M.C. De Keyser, "Adaptive Dead-Time Estimation", 2nd 1FAC Workshop on Adaptive Systems in Control and Signal Processing, pp. 209-213 (1986).
.
L. Keviczlqr and C. Bfiny~isz, "An Adaptive PID Regulator Based on Time Delay Estimation" Proc. 31st Control and Decision Conf., pp. 3243-3248 (1992).
10. E. F. Vogel and T. F. Edgar, "Application of an Adaptive Pole-Zero Placement Controller to Chemical Processes with Variable Dead Time", Proc. Amer. Control Conf., pp. 786-791 (1982). 11. C. Chandra Prasad, V. Hahn, H. Unbehauen and U. Keuchel, "Adaptive Control of a Variable Dead Time Process with an Integrator", 1FAC Workshop on Adaptive Control of Chemical Processes, pp. 67-71. (1985). 12. D. E. Clough and S. J. Park, "A Novel Dead-Time Adaptive Controller", IFAC Workshop on Adaptive Control of Chemical Processes, pp. 19-24 (1985).
204
GIANNI FERRETTI ET AL.
13. I.L. Chien, D. A. Mellichamp and D. E. Seborg, "A Self-Tuning Controller for Systems with Unknown or Varying Time Delays", Proc. American Control Conf., pp. 905-912 (1984). 14. K. K. Biswas and G. Singe "Identification of Stochastic Time Delay Systems", IEEE Trans. Aut. Control, AC-23, pp. 504-505 (1978). 15. F. Abrishamkar and G. A. Bekey, "Estimation of Time Varying Delays in Linear Stochastic Systems", Proc. 7th 1FAC Symp. on Identification and System Parameter Estimation, pp. 1793-1798 (1985). 16. W.X. Zheng and C. B. Feng, "Identification of Stochastic Time Lag Systems in the Presence of Colored Noise",Automatica, 26, pp. 769-779 (1990). 17. G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco. 18. G. A. Dumont, A. Elnaggar and A. Elshafei, "Adaptive Predictive Control of Systems with Time-Varying Time Delay", 1st European Control Conference, pp. 1081-1086 (1991). 19. A. Elnaggar, G. A. Dumont and A. L. Elshafei, "Adaptive Control with Direct Delay Estimation", Proc. 12th IFAC World Congress, 9, pp. 19-23 (1993). 20. A. I. Elnaggar, Variable Regression Estimation of Unknown System Delay, Ph.D. Thesis, Dept. of Electrical Engineering, University of British Columbia, Vancouver, B.C. Canada (1990). 21. C. H. Knapp and G. C. Carter, "The Generalized Correlation Method for Estimation of Time Delay", 1EEE Transactions on Acoustic, Speech and Signal Processing, ASSP-24, pp. 320-327 (1976). 22. G. C. Carter, "Coherence and Time Delay Estimation", Proc. 1EEE, 75, pp. 236-255 (1987). 23. L. Ljung and T. S&terstrfm, Theory and Practice of Recursive Identification, The MIT Press, Cambridge, Mass. (1983). 24. C. B~iny~sz and L. Keviczl~, "A New Recursive Time Delay Estimation Method for ARMAX Models", 8th 1FAC/1FORS Syrup. on Identification and System Parameter Estimation, 3, pp. 1452-1457 (1988). 25. J. Bokor and L. Keviczlqr "Recursive Structure, Parameter and Delay Time Estimation using ESS Representations", Proc. 7th IFAC Syrup. on Identification and System Parameter Estimation, pp. 867-872 (1985).
RECURSIVE ESTIMATION OF TIME DELAY
205
V. Kaminskas, "Parameter Estimation in Systems with Time Delay and Closed Loop Systems", Proc. 5th IFAC Symp. on Identijication and System Parameter Estimation, pp. 669-677 (1979). R. Pupeikis, "Recursive Estimation of the Parameters of Linear Systems with Time Delay", Proc. 7th IFAC Symp. on Ident~jicationand System Parameter Estimation, pp. 787-792 (1985). A. Elnaggar, G. A. Dumont and A. L. Elshafei, "Recursive Estimation for Systems of Unknown Delay", Proc. 28th Control and Decision ConJ, pp. 1809-1810 (1989).
Z. Y. Zhao, S. Sagara and K. Kumarnaru, "On-Line Identification of time Delay and System Parameters of Continuous Systems Based on Discrete-Time Measurements", Proc. 9th IFAC/IFORS Symp. on Identijication and System Parameter Estimation, pp. 1145-1150 (199 1). S. Sagara and Z. Y. Zhao, "Recursive identification of transfer function matrix in continuous systems via linear integral filter", International Journal of Control, 50, pp. 457-477 (1989). Dj. Juricic, "Recursive Estimation of Systems with Time Varying Parameters and Delays", Proc. 10th IFAC World Congress, Vol. 10, pp. 265-270 (1987). M. E. Salgado, C. E. de Souza and G. C. Goodwin, "Issues in Time Delay Modelling", Proc. 8th IFAC/IFORS Symp. on Identification and System Parameter Estimation, pp. 786-791 (1988). S. Roy, 0.P. Malik and G. S. Hope, "Adaptive Control of Plants using AllZero Model for Dead Time Identification", IEE Proceedings-D, 138, pp. 445452 (1991). M. Bjarklund, M. Nihtilii and T. Siiderstrom, "An Algorithm for Identification of Linear Systems with Varying Time-Delay", 9th IFAC/IFORS Symp. on Identijication and System Parameter Estimation, pp.1254-1259 (199 1). M. Agarval and C. Canudas, "On-Line Estimation of time Deley and Continuous-Time Process Parameters", Int. J. Control, 46, pp. 295-31 1 (1987). C. E. de Souza, G. C. Goodwin, D. Q. Mayne and M. Palaniswami, "An Adaptive Control Algorithm for Linear Systems having Unknown Time Delay", Automatica, 24, pp. 327-341 (1988). W. R. Robinson and A. C. Soudack, "A Method for the Identification of Time Delays in Linear Systems", IEEE Transactions on Automatic Control, AC-24, 97-101 (1970).
206
GIANNI FERRETTI ET AL
G. C. Goodwin and K. S. Sin, Adaptive Filtering Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ (1984). E. Gabay and S. J. Merhav, "Identification of Linear Systems with TimeDelay Operating in a Closed Loop in the Presence of Noise", IEEE Transactions on Automatic Control, AC-21,711-716 (1976).
P. J. Gawthrop and M. T. Nihtilq "Identification of Time Delays Using a Polynomial Identification Method", System & Control Letters, 5, pp.267-271 (1985). E. Boje and E. Eitelberg, "On Identification of Time Delay via the Approximation e-Ts z (1-sT/2)/(l+sT/2)", 9th IFAC/IFORS Symp. on Identification and System Parameter Estimation, pp. 1501-1505 (199 1). G. Ferretti, C. Maffezzoni and R. Scattolini, "Recursive Estimation of Time Delay in Sampled Systems", Automatica, 27, pp. 653461 (1991). G. Ferretti, C. Maffezzoni and R. Scattolini, "On the Identifiability of Time Delay with Least Squares Methods", Internal Report Dipartimento di Elettronica e Infomazione, Politecnico di Milano (1994).
G. P. Rao and L. Sivakumar, "Identification of Time Lag Systems via Walsh Functions", IEEE Trans. Aut. Control, AC-24, pp. 806-808 (1979). E. V. Bohn, "Walsh Function Decoupled Parameter Estimation Equations for Dynarmc Continuous-Time Models with time-Delay", Proc. 7th IFAC Symp. on Identrfication and System Parameter Estimation, pp. 799-802 (1985). Y. P. Shih, C. Hwang and W. K. Chia, "Parameter Estimation of Delay Systems via Block Pulse Functions", ASME Jou. of Dyn. Sys. Meas. and Control, 102, pp. 159-162 (1980).
M. Marchand and K. H. Fu, "Frequency Domain Parameter Estimation of Aeronautical Systems with and without Time Delayn, Proc. 7th IFAC Symp. on Identification and System Parameter Estimation, pp. 669-674 (1985). A. E. Pearson and C. Y. Wuu, "Decoupled Delay Estimation in the Identification of Differential Delay Systems", Automatica, 20, pp. 761-772 (1984). W. Song and X. Yu, "Structure and Parameter Identification for a Kind of Multivariable Linear Systems with Unknown Time Delays", Proc. 7th IFAC Symp. on Identification and System Parameter Estimation, pp. 793-798 (1985).
P. Nagy and L. Ljung, "Estimating Time-Delays via statedpace Identification Methods", 9th IFAC/IFORS Symp. on Identification and System Parameter Estimation, pp. 1141-1144 (1991).
St ability Analysis of Digit a1 Kalman Filters Bor-Sen Chen* Sen-Chueh Peng t "Department of Electrical E ~ ~ g i n e e r i n g National Tsing Hua University Hsin Cllu, Taiwan, R.O.C.
t Department of
Electrical Engioeering
National Yun-Lin Polytecll~iicInstitute Huwei, Yunlin, Taiwan, R.O.C.
Abstract A new stability collclitio~~ is introduced for the I
BOR-SEN CHEN AND SEN-CHUEH PENG
208
I.
Introduction
Digital computers are wiclely applied t o control engineering. control, or ellgille co~ltrol For instance, flight control, ~~avigatioli in aircraft. In these systenls, filtering algorith~nscan be realized either wit11 special-purl>ose digital t~ardwareor in programs for
a general-purpose digital coml>uter. In these situations, state values ant1 coefficie~ltsare stored in registers with a finite ~lutrlber
of bits. Tlle values rnust be quantized, because the size of the available register for each value is finite. Extremely, the results ~ l i u s talso be quantized after every aclditioll or nlultiplicatioli (floating- or fixed-point arithlnetic.). These q~lantiz;~tions cal.lse rourlcloff errors. LVhru implemeatiug a filtering algorithm
011
a floating-point conil>uter., our slloillcl cousicles the ~>roblems t h a t arise in dealing with floati~ig-point coml,utation and finite wordlengtli.
A great deal of works [1],[2],[:3], [4],[5],[6], have bee11 clevotecl to the an:t.lysis of rouncloff error in digital signal processing. Tliese c1igita.l signal processing ideas serve as the ha~ ~ e s wit11 ro~~ncloff noise for optimal state sis of t e ~ h u i c ~ clealiug estiniatiou in this 11aper. Sonic rrsearc:hs have also been clone on finite-wortllent ar~alysisof cligital controllers. Those controllers riiaybe be in fixecl- and floating-poilit arithmetic [l],[S], [9],[I 11,[12]. For exanll>le, to1.1ntloffnoise a ~ i dscaling in the digic coml>ensators ital iml,lementation of linear q ~ ~ a t l r a t C:aussian are co~~siclerecl by Moroney et al. [9]. Rink ancl C l ~ o n g[ l l ] have deterlniliecl a11 upper borincl on the mean-squa.re error in state regulator systems. This influence of linear optinlal cliscretetime systenls cl1.1r to finite wortllength is introcl~lc:ec\ by Van
209
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
IVingercleri and De Iutecl gains. Williar~lsori[4] has presented a Illore specific form~.llationof a sirnilar problem in [7]. In particular, be llas taliell the state souncloff quantizatiori into account arid applierl the error spectrur~ishaping technique
[lo] in the cligital Kalman filter tlesign. However, less research has been ~>~.iblislietl on tlie s~.lt>jectof cligital Iiits, ol.ltputs, coefficients, ancl stat.es. In considering floating-point arithn~etic:,sllcl~dyna.nlic ra.nge consiclerations generally can be r~eglecteclbec:a~.ise of tlie large ra11ge of rel)reseutablr nl.lmbers, but cluantization is ititrocl~iceclfor both m~.iltiplicationand acldition. These quantizations leacl to a cleterioration
i11
the icleal
(i.e., irifiriite worrllength) performances of filteririg algorithms.
In extreme cases, tliese cluantizations car] even lead to an iclea.lly stable filtering algorithln 1,ecoming ~lnstable.This instability issilt. cleeply infl~lenc:esthe cIesig.11of I
BOR-SEN CHEN AND SEN-CHUEH PENG
210
111 this paper, the effect of compl.ltation ro~.indofferror, which s tlie Iosecl. kIeanwliile, tlie a c t ~ ~ estimation al error bound and t h e s t a t e signal b o ~ i n din this systeni are also clisc-~~ssed.
11.
Problem Formulation
A continuous-time system is given in the following state-space form:
wliere c o l [ u ( t ) e ( t ) ] is a zeso-mew white noise process with intensity
\lrheu the state a t the sannpling time t k is given, and tlie sampling periotl
It
= fk+, - tL.
t h e cliscrete nlodel of Eels. ( 1 ) and (2) cat1 be expressecl as
STABILITYANALYSIS OF DIGITAL KALMAN FILTERS
v ( k h ) and e ( k h ) forlii secll~enc-esof white noise pro(-esses with zero-nlean val~lesant! tlie covaria~~ces
Note that, in the case of a high sa~iiplingrate, tlie sanll~ling periocl h will approacl~zero such t11:~ttlie eigenval~.lesof A a.re near tlie unit circle ancl covariance ,S1 brconnes snnall. We shall see that, under tliese sit~latiolis,t l ~ eetfects of ro~i~ictoff errors become more ilnportant, l>artic.illarlyfor tlie cases of large S2
[4],[15]. We c-onsicler the oljtiln:~Istate-rst,iui;~tion~)rol.)le~ii for a c1isc:retetiliie system that is clesc-rilxcl by tlir state e ( . l ~ ~ i ~[16] ti~li
where [ ) ( A * ) arid E ( k ) are seqlleuc-es of wl~ite11oibe processes wit11 zero-meat1 valiles arlcl the c-ovarianc-es
Here E[.]cle~~otes the expectation of
[.I.
For siniplicity, the sam-
pling time is used as the tinie 1.11nit,h = 1.
BOR-SEN CHEN AND SEN-CHUEH PENG
212
S111)l)oseyyl(k)is tlic-' 011t1)llt~ i i e a s ~ l r e ~ of ~ i an e ~ lL-hit t analogto-digital cunverter (AT)(:) whose i11p11tis tlie S ~ L I I ~ ~ > Iolltl>~lt ~'CI
y ( k ) , ancl tlie AT)C is 11si11ga synlmetrical refereuc-e system. T h a t is,
y a ( k ) = y(k) + d(k)
(10)
where rl(k) is the AD(: cluautiz:ttiou error occurring a t the kt11 sample time and satisfying Irl(k)l
5 TLof full scale (FS). T h e
probability distributioli of d(k) is uniform over tlie clila~ltizatioli error and also a white noise process [:3]. Sometimes, the consicleratiou of AT)C pro1,lems is very troubleso~ne,since an ADU may reach saturation ariil clepenil highly on tlie liarilware specifications. However, it is necessary to convert analog signals to digital from using a n AD(> in most digital systeuis containing analog as well as digital colnponents. Tlleli tlie iliscrete state eil~~at,ions [Eils. 7 ari~l81 c:ali then he rewritten as
where
€(A*) = e(X)
+ rl(X.)
is a w l ~ i t erioise process with mean zero ancl covariance [.C2
+
(2-L . F,C' . 2)' . I h l / ~ ' ] , wI~ereIn, is all iclrntity niatris with approlxiate cliuleusioils. I11
tlie one--step-alie:t(l prerlic-tiou I>rol)lerii,let tlie process 1)e
described by Eqs. (7) and (8). The state estinlator has the form
aricl the perforn1;tnct.t is defined as
213
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
where t ( k ) = i ( k ) = . c ( k ) , called estilllation error. Tlieli tile I i a l ~ n a nfilter is to clioose l i ( k ) in E(l.(1:3) to ruininlize the perforlua~iceI , and in an infinite-prec-ision case, tlie solution is given as
To implemerit a digital I
+ bj
clenote the reslllt of floating-point
addition atit1 F L, [ah] denotr tlie res111tof floating-point multiplication. A state estil~iatorwith a digital Iialnlan filter wit11 floati~~g-poi11 t c-onip~ltittiouis given l,y
where .i'*(k
+ 1 ) is a s s ~ ~ l n eto( l I)r d (sigtiecl) floating-point
rep-
resentation tvi t l ~a IV-bit ~nantissapart. It is ctss~lnleclthat tlie val~lesof A ancl C: here are retrievrcl from memory when they are neeclecl, ancl the 1l;allnan gain I<
is precornlx~tetlby solving tlie algebraic Riccati equation (1 6) in foswarcl time and is stored it] the computer.
Thus these
values liave alreacly been ro~lntlecl to exact values with finite worclleugths. Now we can precisely state tlie main ~ > ~ . o b l econm sidered in the paper.
Problern. Co~isiclert l ~ state e estimator [Ecl.(lci)] in which the coefficielit ~natric-esA, C, alicl I< exist as finite words. Under
BOR-SEN CHEN AND SEN-CHUEH PENG
214
what conclitions will the state estimator that has been deteriorateel by tlie effec-ts of the finite wol.tllengths still be stable :)
111.
Stability Analysis of Kalman Filters
T h e ol,jective of this section will be to o l ~ t a i na sufficient conclitiol~of stability for the state estirnator
[Eel. (17)] in tlie non-
icleal sit~lation,wliere rolllicloff errors may occllr with every ele n l ~ n t a r yfloating-poi tit acl(1ition ancl
111111tiplic-ation. A11
uyller
1lo1111tlon t h e estimation error tlegracied for fi~~ite-worrllength eff~c-tsis tlerivetl. Hefore f~lrtlieranalysis, sollie ~natheniatic-s tools a~irlclefinitions neetletl for solvi~lg0111. ~)rol)lenlare intlwcluc-ecl.
Let tile norm of real stoc-liastic- vec-tor .r E h"', be clefi~~ecl l,y [I-!] 11.1.u
=
tleuotecl by IlxIJ,
Jq7q
(18)
Definition 2 2
T
T
T
T
IIA.cll = E [ x A AJ:] = t r ( E [ . ~A A.c]) = ~ ~ - ( E [ A ~ A . c s ~ ] )
(19) where t r clel~otesthe trace operator, ant1
where IlAll clel~otesthe il:cl~~cetlnorm clefiliecl as follows :
215
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
Le~llllla1
The rourtded JEoatirlg-point s u m of t w o k l - v e c t o r s cart be expressed b y j'l [ a 1i1ht.r.t. A = 2-",
+ b] = ( I + 4 H ) ( n + b)
(21)
I/V begilt tht. rnnrltissa lt.rigth, a n d
b~t~ilt.t:n - 1 n u d I , s o that u:her-t: each ri is distrib.c~tt.cl~r~ifor~rz,lrj
T h e rorlnded jlontlug-point y rotluc-f o j art :\.Ix il! ~ u n t r i zA ntld ill A pl)t. 11,di.z:A , is g i u ~ (1s . ~fo110~11!.s ~ : art il!-oec.tor. : c , al.qo sho~i~rr,
u)ht.i.e h , has ze1.o rrlenlt nrld the ~ ~ n r i a n c - eart. s approzir~antcly
E [ h : ] = [(i+ 1)/:3],
fir. i = 1 , 2 , . . . , A 1
E[hi,] = k 1 / 3 E[h;ltj] = i
-
1 (24)
(25) for j
>i
(26)
216
BOR-SEN CHEN AND SEN-CHUEH PENG
Using the represelitatiolis of Eqs. (21) alicl (23) in Le~llllla
1, Eq. (1 7) beco~lies
Upori substitution of Cx(k)
+ e ( k ) for y8(k), we liave
+ terliis of higher-orcler
A" A', A4
Co~nl,iningEcls. (1 I ) an11 (28), it follows tliat
where 0 is the zero matrix, ant1 I is the identity lnatrix with appropriate dimensiou. In Eq. (%9), because tliese t e r ~ n sof higher-orcler A', A', arlcl A4 are small, tliey are ig~iorecl.Let
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
217
The11 Eq. (29) car] be rewritten as
Next, we shall al~alyzethe esti~nationerror dile to finitewordlength irl~ple~lle~ltatioi~ of cligi tal I(a11nan filters. Although tile allalysis gives precisely t l ~ rerror clue to finite-worcllength effects, the calculation is slightly c~.in~l,ersome. Tllris, by ignoring higher quantities iu Eq. ( a s ) , the ac:tr.lal t.stin~at,ionerror of the digital I\;a.l~nanfilter clan be rxpressecl by
where
BOR-SEN CHEN AND SEN-CHUEH PENG
218
BB
= [ A ( I C H 3 C + K R 1 C : + R 2 1 i C ' )i A ( R 2 A + AHI - KH:,C' - Ii'RIC - I
+
c,'~ =
[ I i +LI(ICII:~ IiR1 + R21i)]
~ f l=
1
Becailsr B, C , BE, and Cfl are stochastic, 11 Bll, IICIl, 11 Bill, and IIC:nII call be evalilatetl as follows :
where
Referring to Appelldir A , the ~ i i l ~ l of r b E[HJHJ]. E [ H IliTltainetleasily.
Jm Jx--(Q,
ll~ll=
=
(I${)
219
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
Doirig a silliple c-alculatiuo, it is folrlicl that tlie val~lesof ant1 IIC:#ll are tllr s;itne as tlie val~lrsof
jlHll
ant1
11 ~ * j l
IICII. respec-
tively. Since D, A#, ancl
D# are cleterministic-, from
Etls. (2O), tlie
v a l ~ ~ of e s 11 Dl(, ] ( A # ( (aucl , ( ( D # j are ( ol,tainerl aucl listed below
It is easy to see that, i f digital filters in
irifilii t r
-
T h a t is,
I.
/
is a. st;ll,le transition matrix for
prec-ision, tlieli
> 0 aucl 0 5 I . < I . Silnply c-lioose
for sollie constant r n
wliele X,(A), for
A
= I , 2, . -
. , 71,
tIr~iotrstlie eigenvaliles of A.
is tllr absol~itevalue of the rigrlivalile of A (or the
pole of the digital filter) nearest tlir unit circle. A n estimate uf 171
can be nlade froln
I(A"I/rL -< ?71 for
all k . How to get
711
is
sonietinies very tlifficult. Fortunately, rn (-all be obtained wit11 the aitl of a rornl)uter. To tlerive tlie stability corirlitio~iant1 the a c t ~ l a lesti~liatioti error bounrl under the finite-wordlength effects, tlir BellnlanC:ronwall lenlma listed as follows :
iri
tliscrete form is ernl)loyrtl. The l e ~ n m ais
220
BOR-SEN CHEN AND SEN-CHUEH PENG
L e ~ l l ~ n2a[Is] Let ( f ' ( k ) ) r ,a i d ( h ( k ) ) r br mal-valued .seqcie~~c.cs on the set of the positive inttyer Z+. Lct
(~i(X:))r,
lJ,tder. these cot~ditiorrs,if
<
~ ( k )f (k)
+
k-1
k = 0,1,2,..-
h(i)t~(i),
(39)
j=i+ 1
is set equal t o 1 ,rr,hen i = X: - I
Renlark 1 If for. so11tf c.otl.star~f11, h ( r )
< h , V I , tht
IL
Eq. ( 4 0 ) brcor~te.5
Reinark 2 I f f o r sollze con.s.taiit f , f ( i )
5 f , V7,
then Eq. (40) becornes
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
22 1
LVe make the obsesvatio~~ that Eq. (30) has the pesturl~ation term sela.tecl to
[
x(k) .c*(k)
]
It is feasible for us to apply the Bell~nan-C;ro~lwalll e l n n ~ ato obtain the stability csitesiou. Basecl on L e m ~ n a2 a.nd the preceding definitions, we call relate a si~fficie~it co~iclitiotlof stability on the e s t i ~ ~ l a t o[Eq. r (:30)] to soilndoff esrors in the followir~g theorem.
Tl~eorenl1
C o n s i d e r the s t a t e - e s t i ~ r ~ a t os;y.f;terr~ r [Eq. (30)] .u~iththe induced (3211, Ile(k)ll = g1 aud IIu(k)ll = 92, aud suppose the t~-a,tt.sitio,rtn ~ a t , r iA: ~fulfills the requirerncut of Eg. (37). If the stability inequality I L ~ I - I ~ L[Eq. S
is snti,qficd, f h e r ~the dcter~oi.atcdstate estirrzator err*or:s i,s .sfill stable.
~ I L Ft o
i-ou~~dofl
['roof. See Apl>enclix B. Reillark 3 T h e ~ ' e l a t i o f ~ s l ~b.ti-pt ( l ~ t . ethe ~ ~ l o ( : a t i o ~of~ the pol? ncar.e.st t o the 11 nit circle and the co 1 ~ ~ 1 ) ~ u t a t iroo~~ ~ ~a l~ l deor fr lo m i s revealed. Frorn the stability inequality [Eq. (43)], it is S C E T L that the smaller 7. is, the stronger th,e stability will be.
222
BOR-SEN CHEN AND SEN-CHUEH PENG
Remark 4 Front Eqs. (32), (."?7), a n d ( 4 3 ) , the u ~ o r ~ d l e ~ I/\/ ~ g tcart h be dctermirred t o gi~ararrtet,the stabrlity o f t l l c deter*zor.ated system cinder the firinif c-wo~.dlengtl~t-ffec-t.
Remark 5 For a given wordZeilgth LV, the stability i~tcquality[Eq. (43)] c a n be ~ i s e da.s a criterion t o test the stability of the e s t i m a t o r s y s t e m deteriorated by the rou.rldofl noise.
Remark 6 F r o m Theoreirl 1, it i s assurued that
i.s eaalvated, and ~ ill.s'id€ t h e all of t h e P ~ I J E T L D ~ ~ UofE . ~the I h l r n a n filter ? I L ? L . ~ be disk ,ci~%tl~ 1-ndius 7-
<
~rlll
1 - ~ r ~Bll l l t o grlar-antee
tilt.
stability of the
if I t' h, e I
01.A - l<(k)C7 in Eq. ( 1 3 ) n . r ~riot all c ~ ~ i t h tllc i n disk e%grrl~!nlt~cs l)~'ol)o.s.edb;3 A tt,dfr.sor),[I?] is cr~tl)loZ/~C1 ~oitlr,radi.cls 7., a ~c~1t~1rt.c t o t r m t the 1)7'061eir3. ,)lc~pl)o.c.t.1 1 1 ~ ar.tific*ially rnciltiply-the covnr.inrcce.s .C1 n r ~ d,SZ i n the .sy.sterr~of Eqs. ( 7 ) arid ((5') u ~ i t h( l / ~ . ) ~ '
, = ,5',(1/1.)" n r d 5'; = 5 ' 2 ( l / ~ - ) 2 kA n d c r a d 7' < 1 ; i . ~ . 5'; s o n h a s shouln that t h e sy.l;tc.m of' Eq. ( l 3 ) , the c o m p u t a t i o n of Iati eq11,ation ( 1 6 ) call be changed to
STABILITYANALYSIS OF DIGITAL KALMAN FILTERS
223
T h e n the eatir7zntio7~ error ~ ( k of ) the filter iu Eq. (14) u~ill converge at least as fast as r k ,u~h,ertk incr'ra~e.';;i e . , all of the eigcnvnlues oJ ( A - I\I,(X:)Il) are irlside the disk ,coith radia.~I . .
After rlerivi~lgthe sufficient c o l ~ r l i t i ouncler ~ ~ whicli the deteriorated 1l;alnlan filter is stable, we call f i r ~ c lone fo1.111 of bound
by tlle estimate given in Theorem 2. Theorem 2 C o ~ i s i d e rthe statc-e.~ti7nator. systent of The~i'ernI . If the stnBility criterion of Eq. (43) is snti.?fit-d u ~ h e iX:~ -t
m,
then the
can be es~nl,untedas
t-.stitnc~fiotr, frrorIl:i."(X:+ 1)--.t:(X:+ 1 ) l ) is ho(iu(lm! and fhc nc%tr~nl by
Proof, See A~>penclis(:.
We clo not claim that the bo1111clgivrn here is the illtilllate tool for s o l v i ~ ~the g problem of the ~liilli~nal worcllrngth of imple~ n e n t i n gdigital I
224
BOR-SEN CHEN AND SEN-CHUEH PENG
a conservative design. This paper does pernlit an approximate
arialysis of the performance/cost tradeoff for wordlength design choic,es.
IV.
Numerical Example
To illustrate the stability criterion proposecl herein, we collsider an iliertial liavigatioli sys tell1 (INS) clue to wind-induced bending. Tlie state-space representation of the system is [19]
where pb(t), ,ub(t)alicl (lb(t) are tlir Ilorizontal clisplacenlent, velocity, and acceleration, respectively. Tlie wliite C:a~.rssiannoise processes v ( - ,.) is of appropriate strength to yielcl the desired root-liieati-squaw
(1.nis) val re of wind,
a,,,;,,,l, wi tli
correlation
time 1/X. If tlie bentling clyna.mics ~noclrlis of se-tconcl orcler with unclalnpecl natrlral frrq~rencyw,, alicl cl:r~ul,ing ratio11 ,
tlieli
2 the three paralneters a , [j, ant1 y are sprcifiecl by cr = -Xu,,,
,L3 = -wi - 2CwIL,and y = -2Cw,, - A. The cliscrete state-space represent;ttiou of the INS wit11 w, =
:3 racl/s, X = 0.6667 l/s,
C = 0.5, ancl
0.025 s can be espressetl by
the sa.~nplingtime h =
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
where v(k) alicl
e are wlii te noise processes with zero-meau
225
val-
ues and the covarialices
Hence,
From Eqs. ( 3 2 ) arlcl (:3:1), we have
Using the robust stability criterion of Eq. (43), we obtain
From the preceding analysis, we suggest requiring the wordle~lgth t o be greater tlian or eclllal to 9 bits; otherwise, it may lead to a n unstable response. S u l ~ s t i t u t i ~these l g values of
11 B#Il, (Ic:#I~,
IIC'II, 11 011, IIA#ll,
ancl IID#II, we can estilnate the upper bouncls of
226
BOR-SEN CHEN AND SEN-CHUEH PENG
0
2
4
6
8
10
12
14
16
18
20
Mantissa length (bits) - T -
Figure 1: Expectatiori of [x* x*] (solid curve) with the wortllerigth b o u ~ l d(dashed line).
ancl 1/2*(k+ 1) - x(k + I ) / / . The estimator systenl using the I i a l ~ n a filter ~ i was sini~~latecl 011 an IBM-AT coulpl.~tersystern with a very long worclleugth, and the ADC for the output measuremerlt was 10 bits. Tlie wind input was si~llulatedby use of a pseudo random Gaussian ~lunibergenerator. Si~llulationswere done for wordlengths fro111 1 t o 20 bits, and the results are sliowrl in Figs. 1 and 2. Notice in Fig. 1 that the estimator is riot well behaved and in Fig. 2 that the estimation error is very large when the wordlength is small.
227
STABILITYANALYSIS OF DIGITAL KALMAN FILTERS
10-~
2
4
6
8
10 12 14 Mantissa length (bits)
16
18
20
Figure 2: Expectatiorl of I(:> - x ) ~ ( $ - x)] (solid curve) with the wordlengtli bouncl (dashed line).
V.
Conclusions
A sufficierit coriclitior~has bee11 presented to ensure the stability of tlie state esti~natorwith a I
BOR-SEN CHEN AND SEN-CHUEH PENG
228
tion of Eq. (43) is satisfiecl,
i*(k) alicl .c(k) in Ecl. (30) will be
bo~iuelecl,and the estilnation error will be also bo~~nclecl. This point agrees with tlie resl~lt~ n ~ u t i o n r in c l Theorem 1. T h e res~iltsof the paper have I~eeual>pliecl to the stability analysis of finite-wol.cllengtll I
VI.
Appendix A : Error Representations of Floating-Point Computat ion
Tlie error reprrseutations of aclcli tiou ancl 1n11ltiplication with floating-point c-onil~~tation are giver1 here [ I 1],[1:3]. Wilkinson [13] psovicletl the ~nethoclof analyzing the errors of floating-point aritlinietic-a1 res~llts.The r o ~ ~ n t l efloating-l>oint d Slllll
of two llllllil>rrh. (1 alld I))
(.ill1
e ~ ~ > r ~ s sl>y ecl
wlierr fl(-) is the floating-point operator, ancl
1.
is a ran(lom
varial>leunifornlly tlistril~uteclIwtween -TWancl T WW , being tlie worcllength of the mantissa. If n aricl b are two M-vectors, then the floating-point s11111 is expressed by
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
229
where a; aricl b; ( i = 1,2,:3,. . , Ad) are ele~iie~its of 1W-vectors a a1ic1 b, respectively, I is an iclentity, ancl
i = 1,2, - . , Ad, are ~iiutuallyinclepenclent a~iclare clistributed between -2-W alicl 2 - W . T h e rounclecl floating-point procI1.1ct of two numbers, a and Here
I-;,
where F L ( . ) is the floatiug-point operator ancl 6 is also distributed between -TWancl 2-W. Siniilarly, the rour:clecl inner protl~lctof two ,bl-vectors is
Each 6;a~iclr; car1 be cousiclerecl to be ~nutuallyilldepericlerit arid ~lliiforrlilydistributeel between -2-2W
ancl 2-".
Let 1.1s clefirle
230
BOR-SEN CHEN AND SEN-CHUEH PENG
If the preceding ecluations are expancled clirectly in orcler to obtain exact h,, the exl>ressiou of h, will be rather conlplicatecl. Therefore, we ignore the s~nitllhigher-ortler terms in Ecls. (59) and (GO), ancl each
11,
can he exl)ressed al,proxirnately by
si + IT 7.k , 2
hi =
for i = 1 , 2 , - . . , h i l - 1
k=l
ILM
=
b+
n
M-1
k= 1
T~I~ each I I 12, has zero Ineau ant1 the variances are approximately E[hf] =
(1
+ 1)2-~'"/:<,
for i = l , ' , . . . , h I
-
1
E [ h i i ] = nl2-""/:3
e[h,j,] = i')-2Lv/:3,
for j
>i
T h e r o ~ ~ n t l e~ni~lti~,lic-ation d of an !\/-vector
.E
l)y an iL1 x ill
matrix A call be written by
wliere n ; j is an e l e m r ~ of ~ t A, ancl h's
i11 clifferent
rows are statis-
tically inclepenclent and ~.lncorreIate-ttl.Hence, we s i ~ b s t i t r ~each te bI-tuple (Al, h2, . . . , h i t I ) for each hf-tuple (Ilil, hz2,- . . , 1 2 ; ~ ) ancl o l ~ t a i uthe per~~littecl rel'resentation
which is c o ~ l v e ~ l i efor ~ i t rnanipr.llation in the clerivatio~iof oirr results.
STABILITYANALYSIS OF DIGITAL KALMAN FILTERS
VII.
23 1
Appendix B : Proof of Theorem
Cousides tlie combine-?tlstate Eq. (30)
l7
=
[i]
Solving tlie ~)rrceclingtlifferruc-eecl~~:~tiou, we o1,tain the solution
Taliing norms, we get
232
BOR-SEN CHEN AND SEN-CHUEH PENG
Using Eq. (37), it is
fo111id
that
Divicliiig both sides of Eq. (65) by
7."
we obtain
Applying tlie Re1narl;s of Lemma 2, we have
+
(
I
)
{'
-" 1-
+ '"l "l/'~"k +
1I I1 ,::J:
(7~~11~11/7'>1
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
Next, ~ ~ ~ ~ l l t i p leither y i ~ l gside of Eq. (67), it follows that
When k
t
m,
if
I-
+ ~ r b l l P j l (< 1, then
the state estimatol. will
be r o b ~ ~ s t lstable. y
VIII. As k
-t
oo,
Appendix C : Proof of Theorem 2 Eq. (68) becomes
233
BOR-SEN CHEN AND SEN-CHUEH PENG
234
Hence, the bound of
1 I (
ask-tm. Fro111 Eq. (;31), t h e actual estimation error is
Taking nornls, we I ~ a v t .
STABILITY ANALYSIS OF DIGITAL KALMAN FILTERS
235
TIie~iEqs. (69) to (71) together iniply that
References [I] G. Aillit a ~ i dIJ. Shamkeel,"Sunall rou~idoffnoise realization of fixetl-point cligital filters aucl controller," IEEE Trans. OIL Ac~ustics,Speech, and Signal Proces.si.r~g,Vol. ASSP36, pp. 880-891 ,(1988). [2] L. R. R.al>iner alicl 11. Golcl, Theory and applicatiorl o j digital signal pi-octcs.sir~g,Prentice-Hall, El~glttwoodCliffs, NJ. (1975). [:3] A. v. Oppeuheinl aucl R. FV. Scliafer, Digital Sigrlnl Praoc.f:ssing,Prentice-Hall, E~iglewooclCliffs, N.J , ( 1975). [4]
r). Willia.ulison a~iclS. Sriclliara.n, "An approat:li to coefficient worcllength retl1tc:tiotl in cligital filters," IEEE Tr.ans. Vol. CAS-32, pp. 893-90:3,(1!185). or1 Cir.cuit.5 nrtcl S;y.tit~vz..s,
[5] H. Homar, "(:om~~l.ltatio~ially efficient low rouncloff rioise secontl-orcler state spa(-e str~.lctures,"IEEE Trans. on Circuits aild ,S'ystems, Vol. (;AS-3, p1). 35-41,(1986). [6] D. V. Bliaskar Rao, "Analysis of coefficient quaritizatiorl el.rors in state-space digital filters," IEEE T r a ~ ~OIL s . Acoustics, Speech, a,kl ,5'ign;l ~ r o c c . s . s iVol. ~ ~ ~ASSP-34, , pp. 1311:39,(1956).
[7] A. B. Stripacl, "Prrfor~nanc-etlegraclation in cligitally implrnientecl Iialnia~ifilters," IEEE Trarls. on Aerospace arld Electror~ic.,Systents, Vol. AES-17, pp. 626-6334,(1981).
[8] P. Morouey, A . S. lVillsl;y, and P. I<. Houpt, "The digi-
tal i~nplenleutatio~i of colitrol com~>rnsators:the coetficie~it wortlle~lgthiss~le,"IEEE Trans. on Azlton~aticCorltrol, Vo1. AC-25, pp. 621-630,(1980).
236
BOR-SEN CHEN AND SEN-CHUEH PENG
[9] P. Moroney, A. S. Willsky, and P. I<. Houpt, "Roundoff noise and scali~igin tlie cligital ilnl)lemantatio~iof co~itroi colnl>ensators," IEEE Trans. orr Acoustics, Speech, and Signal F'r*oc.es.sirrg, Vol. ASS P-:3 1, pp. 1464-1477,(1983). [lo] PV. E. Higgilis ancl D. (2. hli~nson,Jr., "Noise recluction strategies for cligital filters: error spectruni shaping versus tlie optillla1 linear state-space formulation," IEEE Trarls. OIL Automatic Corrtrol, Vol. ASSP-30, pp. 963-973,(1982). [ l 11 r. E. rink and H. Y. Clioug, "Performance of state regualtor IEEE Trans. on syste~liswit11 floating-point co~n~iltatiou," Automatic Control, Vo1. AC-24, pp. 41 1-42 1,(1979). [12] A. .I. M. Vari Wingerclen and W. L. De Koni~ig, "The influeuce of finite wordlength on digital optillla1 co~itrol," IEEE Tmns. on Aautoln.atic Control, Vol. AC-29, pp. 085891,(1983). [13] J . H. Wilkinsou, Rounding errors in algebr*aic pr-ocesses, Prentice-Hall, Englewoocl Cliffs, NJ, (196:3). [14] C;. (:. C;ooclcvin ancl I<. s. Sin, Ado.l)tz~~e filtering predictiorr nrtd contr.ol, F'rentice-Hall, Englewootl Cliffs, N.1, (1954). [l5] C'. A. Desoer ancl RII. Viclyasagar, Feedback systerns: inputo i ~ f pp~'ol)crties, ~~t Acacleniic, New York, (1975). [16] I i . 3 . Astroln aticl 11. Wittenniarl<, Cornputcr. corrtroll~dsyatevz~: theory arld dc.r;igrr, F'rrutic-e-Hall, Englewoocl Cliffs, N.J, (1984). [17] B. D. 0. A~iclerson, "Exl)onential data weightirig in tlie I-30, pp. 930-9;39,( 1985). [19] P. S. Ma.ybeck, Stochastic rnodel.5, estimntiorr and co~strol, Vol. 1, Acaclemic, New York, (1979). [%O] C. T. KI.IO,R. S. C;Iien and Z. S. Kuo, "Stability aliadysis of digital I
D i s t r i b u t e d D i s c r e t e F i l t e r i n g for Stochastic S y s t e m s with Noisy and Fuzzy M e a s u r e m e n t s
Lang Hong Dept. of Electrical Engineering Wright State University Dayton, OH 45435
Abstract This chapter discusses distributed discrete filtering of noisy and fuzzy measurements, which employs both Kalman filtering and fuzzy arithmetic. Due to the property of fuzzy arithmetic, fuzziness of the parameters in a system under the extended operations will unlimitedly increase and finally reach an unacceptable range. A new compression technique is presented here to solve this problem. An algorithm for distributed discrete filtering using this compression technique is introduced and is applied to multisensor integration. 1. I n t r o d u c t i o n Since fuzzy set theory was introduced by L. A. Zadeh [8] in the 1960s, people began to appreciate how uncertainty originating from human thinking can affect scientific problems. During the last two decades, fuzzy logic has been successfully used in working with numerous practical applications. The most well known and popular work is the combination of fuzzy logic and expert systems. Fuzzy set theory working "numerically" in engineering applications hasn't received much attention until recently. Schnatter [5] pointed out that the existence of another kind of uncertainty, which can be described by fuzzy sets should, be taken into consideration during system processing. This uncertainty is due to the inaccuracy of the data measuring process. Thus, uncertainty in the stochastic sense doesn't dominate modeling of unknowns in a system anymore. In order to consider the system from a more realistic viewpoint, one has to face both stochastic uncertainty and fuzziness together. Many published papers have discussed the issue of inserting fuzzy information into a system, but few of them mentioned the difficulty of controlling the fuzziness expansion of the elements in the system when applying fuzzy arithmetic. This is a general and practical problem, since CONTROLAND DYNAMICSYSTEMS,VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
237
238
LANG HONG
due to the property of fuzzy arithmetic [4, 8], data fuzziness will unlimitedly increase and finally reach an unacceptable range after a series of extended operations. In this chapter, a new compression technique is presented to solve this problem and to keep the fuzziness within a reasonable range. An algorithm of distributed discrete filtering is introduced for stochastic systems with both noisy and fuzzy measurements. Computational efficiency and better fault tolerance are achieved using the distributed filtering scheme. The immediate application of the algorithm presented in this chapter is multisensor data integration where the data contain both stochastic and fuzzy uncertainties. This chapter is organized as follows. Section 2 briefly reviews some basic concepts of fuzzy numbers, fuzzy arithmetic, and Kalman filtering. Section 3 discusses the operations of discrete Kalman filtering on noisy and fuzzy data and the compression of data fuzziness. Distributed discrete filtering of noisy and fuzzy measurements is discussed in Section 4. In Section 5, an example is provided to demonstrate the effectiveness of the algorithm. Finally Section 6 concludes the chapter. 2. B a c k g r o u n d In this section, fuzzy numbers, fuzzy arithmetic, and Kalman filtering are briefly reviewed. Since these techniques have been well developed, this chapter will not include their proofs which can be easily found in [1, 3.4, 8].
2.1 Fuzzy Numbers and Fuzzy Arithmetic A fuzzy number is a fuzzy set A*, where (-)" denotes a fuzzy set, whose membership function UA: R ~ [0, 1] satisfies the following conditions 1. A" is normal
3 a
2. A* is convex
for every a-cut {C(A*)}~; a C [0, 1] is convex
such that UA(a)
=
1.
where ~ is equivalent to implies. For the convenience of performing fuzzy arithmetic, all the membership functions in this chapter are expressed in an L-R fuzzy set triplet form, i.e., X = (rn.,a,b), where X is a fuzzy number, rn is modal value (Ux(rn) = 1), and a and b are
the spreads. The spreads may be considered as the measure for the accuracy/fuzziness of the fuzzy number. Fig. l(a) illustrates the triangular membership function X" = (m, a, b) in one
DISTRIBUTED DISCRETE FILTERING
239
dimension (l-D) and Fig. l(b) shows fuzzy numbers A * = (rn, a l , a 2 ) and B * = (n, bl, b2) in two dimensions (2-D). The example in Section 5 is in the two-dimensional form.
U
,
1
U
...........
Y
x a
rn
b2
;
b
(a) X*= (m,a,b)in 1-0
(b) X* = (m,al,a2) and Y*= (n,bl,b2) in 2-D
Figure 1: Membership functions in 1-D and 2-D. All the extended operations (or fuzzy arithmetic) in this chapter are performed using the technique of c~-cut representation [1] and defined as follows. Let A* and t3" be two fuzzy numbers, and As and Bs their c~-cuts such that As = [al, a2]s and Bs = [bl, b2]s, for o~ E [0, 1]. The following properties hold for fuzzy number operations. 1. Addition:
A; +/3'~ = [al, a2]s + [bl, b2]s = [al + bl, a2 + b2]s; c~ C [0, 1]
(1)
2. Subtraction:
A2-t3;=[al,a2]s-[bl,b2]s=[al-b2,
a2-bl]s;
a E [0,1]
(2)
3. Multiplication:
A; 9 B2 : [a~, a2]~ 9 [bl, b2]~ = [~1, c2]~;
~ ~ [o, 1]
(3)
where cl : rain(a1 9 bl, al 9 b2, a2 9 bl, a2 9 b2)~ and c2 - m a x ( a 1 9 bl, al 9 b2, a2 9 bl, a2 9 b2)~
240
LANG HONG
4. Division:
A*~/B; : [al,a2]~/[bl, b2]~ = [al/b2, a2/bl]~;
a C [0,1] and 0 ~ [bl,b2]
(4)
5. Reciprocal"
A'21 = [al,a212' = [1/a2,1/al]o"
c~ C [0,1] and 0 ~ [al,a2].
(5)
The results of the above operations are still fuzzy numbers and the proof can be found in
[4]. 2.2 The Kalman Filter Since the algorithm of distributed discrete filtering described in this paper is built on the Kalman filtering technique, Kalman filtering is reviewed here. Consider a linear dynamic system described by X__k
--
zk =
~k_lX.X_k_l -~- W_W_k_l,
Hkz__k + ~ ,
W_k_1 ~ N ( 0 , q k - 1 )
v_k ~ N ( 0 , R k )
(6)
(7)
where x_k is the stochastic state vector of the system and z_k is the measurement vector (noisy data) at time k.
w k and v__k are the modeling uncertainty and the measurement
uncertainty, respectively, with known statistics. Ok is the state transition matrix and Hk is the observation matrix, both of which are known. For the above system model, the most important linear estimator was developed by R. E. Kalman [3]. This estimator (which is referred to as the Kalman filter) can be described in terms of either a discrete-time or continuous-time formulation. The major portion of the presentation of this chapter is concerned with the discrete-time model, since it seems to be particularly suitable for implementation on a digital computer. Furthermore, it accurately describes the common physical situation in which measurement data are obtained at discrete instants of time. The Kalman filter provides an estimate of the state of the system at the current time based on all measurements obtained up to and including the present time.
Table I lists
a s u m m a r y of the discrete-time Kalman filter equations and Fig. 2 illustrates the timing diagram of the discrete-time Kalman filter. The derivation of Kalman filter equations is in
[3, 7].
DISTRIBUTED DISCRETE FILTERING
241
Table I. A summary of discrete-time Kalman filter equations system model
x k --- (~k_lX___k_ 1 .qt_ t U k _ l ,
measurement model
_zk=Hkx_ k+v_k, v k ~ N ( 0 , R k )
initial conditions
E{z_o} = x__olo,E{(Xo
other assumption
E{wkv'j} = 0, for all j and k
state estimate propagation
~g___klk_1 = ~ k - l ~ k _ l l k _
tUk_ 1 ~ N ( O ,
~_olo)(Z_o X_olo)'} = Polo
-
-
1
error covariance propagation
Vk[k-x = ~k-lVk-l[k-1 ~'k-1 q- Qk-1
state estimate update
z--'klk -- Z---'klk-1+ Kk[zk -- Hk~klk-x]
error covariance update
Pklk = [ I - KkHk]Pklk-1
Kalman gain matrix
Kk : Pklk_lH'k[HkPklk-lH'k + ak] -1
Hk-1 Rk-1
Hk Rk ~k-1 ,Qk-1
Qk-1)
Ok, Qk
-Xk.1,k. --Xk.l,k.1
_XkJk.1
Xklk [~~
Pk-11k-2/ Pk-llk-1
Pklk-1
Pklk
k-1
Figure 2: A timing diagram of the discrete Kalman filter.
242
LANG HONG 3. K a l m a n F i l t e r i n g of N o i s y a n d F u z z y D a t a As mentioned by Schnatter, random variation is not the only uncertainty to be faced in
a dynamic system; another source of uncertainty is introduced through the impossibility of obtaining exact values of the measured data. Thus, the conventional way of treating the measurements as precise values should be challenged. Because of the rigorous demands for numerical precision and measurability of variables, it is unavoidable that a numerical system model will lose some information. Thus, there will be a discrepancy between a model and the reality. Fuzzy models do not have rigorous demands for the accuracy of the values of variables, where variables in a model may be determined imprecisely and approximately.
3.1 Fuzzy: Parameters Estimation and forecasting for precise data have been well developed. The Kalman filter, especially, has been discussed extensively in the literature, but applying the Kalman filter to fuzzy data has not received much attention. Schnatter has worked on this problem in [6], in which he showed an example of the application of the Kalman filter with fuzzy parameters. In that paper, he mentioned. Ut- (input) and }~)~ (observation) were fuzzy vectors (in the consideration of this chapter Uk = 0), so ~klk and ~klk-1 (estimates) would be fuzzy. But since the covariance Pklk did not depend on Uk and Yk, it remained a precise matrix although Uk and 1~- were both fuzzy. We totally agree with the fuzziness of z__*klk and ~klk-1, but feel that the covariance Pklk should be fuzzy too. In the dynamic equations of the system we notice that there is a parameter Ilk which specifies the uncertainty, of the measurement distribution. If the measurements are fuzzy, then Ilk should be fuzzy, too. Fig. 3 illustrates this idea, in which the circles around the measurements represent the fuzziness of the Ils and their diameters express how fuzzy the Rs are (the larger the diameters, the fuzzier the Ils). That is, the ranges of these circles which introduce the fuzziness into Rk and those circles can be seen as the uncertainty of
the uncertainty Rs. In our case, the diameters of the circles are fixed because we assume that we know the characteristics of the measuring tool well. Because of the fuzziness of measurements, the distribution of measurement error v k becomes v_k .-~ N(0, R~r
where
R ; = (Rk, Rkt,Rku). Since the measurement fuzziness is fixed, the fuzziness of R~ is also fixed and the fuzzy measurement distributions are shown in Fig. 4. Fig. 4 is plotted in
DISTRIB UTED DISCRETE FILTERING
243
one-dimensional sense, so the fuzziness is described by a ts.
measuredsignal
uncertaintydescribedby R k
true signal
(~)
uncertaintyof uncertaintyR k (fuzzinessof Ilk) measuredsignal / "~,-~.'I .[i / trui ~ a '
t
(b) Figure 3: (a) Regular measurements, and (b)' fuzziness of measurements.
244
LANG HONG
'"
0" kl",,',
Figure 4: Fuzziness of the measurement error covariance. Table 1 shows the dependence of covariance Pklk on Kalman gain Kk, and since Kk is a function of Rk, fuzziness comes to Pklk via the fuzziness of Kk. Thus, the fuzziness of covariance P klk is confirmed. For the similar reason discussed above, p a r a m e t e r Qk in the equations of the system is also fuzzy. Table 1 is fuzzified as shown below. Table 2. Discrete-time Kalman filter equations on fuzzy data. wk_ ~ ~, N(0,
q;-1)
system model
x__k = % _ l z , k_ 1 + wk_l,
measurement model
z ~ = H k x k + v k, v _ k ~ N ( 0 , R ; )
fuzzv initial conditions
E{~0} = ~ 0 , E{(~0 - EL0)(~0 - ~ 0 ) ' } = P;~0
other assumption
E{w__kv_'j}=
0, for all j and k
fuzzy state estimate propagation
r
-~- Q~--1
fuzzv error covariance propagation
P;Ik-, = Ok-, P ; - l l k - ,
fuzzy state estimate update
i;ik = i;ik-, + K ; M -
fuzzy error covariance update
P~-Ik = [ I - K;Hk]P~lk_ ~
fuzzy Kalman gain matrix
K ; = P;ik_,H'k[HkP;lk_,H'k + R ; ] - '
Hki;Ik-,]
Since the parameters are fuzzy, the Kalman filter must carry the fuzziness of each parameter during each step such as propagation, updating, etc. Schnatter showed the method
DISTRIBUTED DISCRETE FILTERING
245
of propagation of fuzziness; he proved that the a-cut representation C(A*) of the image of a fuzzy set under a mapping is simply the image of the a-cut representation of the fuzzy set, namely
C(f(a*))~ = f(C(a*)~); a C [0, 1].
(8)
This is very useful in the implementation of our example. We have already put fuzzy parameters on the Kalman filter. The next step is to compute and propagate those fuzzy data using the equations in Table 2. Here we face a major difficulty -
after a long-term operation of fuzzy arithmetic, the data fuzziness will keep increasing and
finally reach an unacceptable range.
Schnatter mentioned this phenomena in [6], but he
didn't mention how to deal with the expansion of the fuzziness of system parameters. In the following, we will discuss a new approach to keep the fuzziness in an acceptable range.
3.2 Compression of Fuzziness In working on the Kalman filter with noisy and fuzzy data, cumulative fuzziness always makes the forecasted results useless. In order to keep the fuzziness under control at each estimation step, we use the following approach to compute the fuzziness. For example, in Table 2 we have ~;ik -- ~;ik_ 1 q- K ; [ z ;
- HkX__~;lk_l].
(9)
In analyzing this equation to get the fuzziness of ~lk, there are three different sources of fuzziness which must be taken into account - the fuzzinesses of ~21k-1, K~, and z~. If we put all three fuzzinesses into the equation and perform the extended operations, the fuzziness of ~lk will be raised to a very high value, and the next propagation and prediction will push it further higher. As long as this process is repeated, the cumulative fuzziness will unlimitedly increase which makes the forecasted data nonsense. Thus, we don't put these three fuzziness sources into the equation together; we consider one fuzzy source at a time. In other words, we divide the original equation into three ,,.1
-i~:lk_ 1 -}- Kk[z_k - Hki;lk_l] ,,.2 z--klk -- iklk-1 + K;[z-k -- Hkiklk-1] ,,.3
Z__kl k
x_kjk =
iklk-~
+ K k [ ~ ; --
I-Ikiklk_i].
(10)
(11)
(112)
246
LANG HONG
Each equation carries one fuzziness from different fuzziness sources. The first equation has the fuzziness of ~lk-1, the second one has that of K ; , and the last one has the fuzziness of z_;. Since each ~;i~' i = 1,2, 3, is a fuzzy number (the results of the above extended operation are still fuzzy numbers), we now have three fuzzy numbers with the same modal values and different spreads (or fuzziness, confidence interval). We then take the intersection of these three fuzzy numbers, and get the intersection which is a new fuzzy number.
Proof: 9 Let A'. B ' . and ('" be fuzzy numbers .4" = ( m . a l . a 2 ) B* = (m, bl, b2) C* = (m, cl. c2)
9 L~. Lb and Lc are called the left sides of A'. B" and C" and R~, Rb and Rc are called the right sides of A'. B" and C ' . L~. Lb and Lc are continuously increasing functions and R~. Rb and R,: are continuously decreasing functions. 9 The intersection of .4". B" and C" is D ' . where D" = (m, dl, d2). The following holds a) UD(m)-- 1
~
normal
b) dl > m a x ( a l . b l . c l )
and d2 < ma:r(a2, b2. c2) -+ bounded
c) Ld - (L~ N Lb C/Lc) ---+ continuously increasing, where A is an intersection operator Rd = (Ha N Rb A Rc) --+ continuously decreasing
d) from b and c --+ D" is convex
9 D" is a fuzzy number. Fig. 5 illustrates the proof.
DISTRIBUTED DISCRETE FILTERING
Lot Lb
Ra
/' "
9 . '~
' i ; ",
cl bl
al (dl)
247
m
Rb Rc
,
c2 (d2)
b2 a2
Figure 5: An illustration of the intersection of A*,/3* and C*. The newly generated fuzzy number represents the most likely confidence interval of i~jk. Losing some information is unavoidable while taking the intersection of three fuzzy sets, but the advantage on the controllability of data fuzziness will cover this loss. The next example will show that this approach takes all the possible fuzziness sources into consideration, and still keeps the fuzziness in a reasonable range during long-term predication and propagation of the Kalman filter. 4. D i s t r i b u t e d D i s c r e t e F i l t e r i n g of N o i s y a n d F u z z y M e a s u r e m e n t s A discrete stochastic system with N sensors can be modeled by 3gk • z_.ik =
~k-lf-k-1
--~ Gk-lWh-1,
/~{Wk-1} -- 0, J~{Wk_lWkL1} -- Q ; - 1 ,
i i + v-k, i E{v~} = O, E { ~ (i~ i)T } I R k ,.i i - 1 , Hkz-~ -
---, x
where z_k E 7~~ is to be estimated at the kth moment. The system matrix ~
(13) (14) E T~~x~ is
determined by the relative dynamics between sensors and the sensed environment, and the modeling error is characterized by the error covariance matrix Q;. There are N measurement models in the sensor system, each of which corresponds to a sensor. The symbol z*~ C T~"~ .i describes the noisy and fuzzy measurements acquired by sensor i specified by R k- Note
248
LANG HONG
that there are two sources of possible measurement errors: one is stochastic and the other is fuzzy. The stochastic error is used to describe the degree of our knowledge of the measuring instruments, say a meter with +5% of accuracy. Fuzziness is employed to describe the error introduced during the instrument reading (or data recording) process. Even if we have a perfect meter, the reading from the meter may not be perfect. In Eq. 1:3. vector x k is represented in the central coordinate system at the kth moment, and ~xk i
~,
i = 1 ~ . . . , N in Eq.
moment.
14 are represented in the local coordinate systems at the kth
Each sensor is associated with a local coordinate system, and the integration
process in the central coordinate system is called the central process. The communication network from the central coordinate system to the ith local coordinate system (the forward communication network) is specified by function Wi xik -
~--i(s k, k) + dik, i = 1 ..... N
(15)
where d i denotes the uncertainty of the communication network whose mean and variance are assumed known
E{d' ~}
=
0, and
E { ~ (i~ ) i T } = U k ~
The communication network from the ith local c o o r d i n a t e s y s t e m to the central coordinate system (the backward communication network) is represented by function r i x~ k - ~ i ( x ~ , k )
+ a__ik, i -
1 .... , N
(16)
where -~--C x i k is the variable of x~ represented at the central coordinate system, and c( is the backward communication uncertainty with its statistics given by E{a_jr = 0,
and E{c~(a_~) r} = Vk i.
It is assumed that forward and backward communication network uncertainties are uncorrelated, i.e.,
E { 9_~(~_~) i i T } = o. Usually, the forward and backward communication networks satisfy the following constraint x k = ~ i ( ~ , i ( x k , k ) , k ).
However, in general, they could be any function.
(17)
DISTRIBUTED DISCRETE FILTERING
249
In this chapter, the communication networks are assumed to be linear transformations
(Fig. 6)
i
x k = Jikxk + Tik , i = 1,...,N,
(18)
where Jik is a rotation matrix and Tik is a translation vector, and both Jik and T__ik are assumed exactly known.
,hk,_Tlk ZT
Figure 6: A multicoordinated multisensor system. In centralized filtering, local sensors take only measurements and most of the work of filtering is left to the central processor, which causes a heavy calculational burden in the central unit. In order to achieve computational efficiency, distributed filtering is employed. In the distributed algorithm for dynamic sensor systems, each sensor has its own processor and the noisy and fuzzy measurements acquired by the sensor are locally processed. Only the statistics of local process results are sent to the central site for integration. A functional diagram of distributed filtering is shown in Fig. 7, where general backward communication links (Eq. (16)) are used. Since local processors can run in parallel, the distributed algorithm is more computationally efficient. In distributed filtering of noisy and fuzzy measurements, there are N local processors, each of which implements the fuzzy Kalman filtering equations ^.i in Table 2 and generates N local updated estimates z_ klk and P .iklk, i = 1, ..., N at the kth
moment. By mapping N local updated estimates as well as N local propagated estimates to the central integration site by the backward communication networks, Fig. 7, we have -
:~,iCklk' P*iCktk , and Z__ ^.iCklk-l' P*cklk-l' i = 1, ..., N ready for integration, here the subscript C
also denotes the quantities at the central site before integration.
250
LANG HONG
The integrated estimate ~*klk is derived as follows N i~lk = p~lk[(p;ik_l)-l_b;ik_ 1 + ~--~((p.i . c'/r i=1
9 -- (P .iGI~-, )-lx---~*Cklk-1 i ix-- Ckl/~ )]
(19)
which generates a m i n i m u m central error covariance
(eT~la-)
-1
= (e2l*-,)-
N ' -1 + Z[(P*~:klk ) - - ( e i=1
1
no,syand.11 no,syand"'l
fuzzy data z k
^"l Xkl k
^.1 ( Xkl k , k ) + _
.1
~Ckl
1 processor local i J
^"1
(2o)
I processor ,~ N I
1^'
Xklk
Xklk
^.i i _ ( Xkl k , k ) + ~k klk
G,k-,)-']"
noisy and .N fuzzy data z k
fuzzy data z k
I processor local 1 I
.i
^.N N Xkl k , k ) + ~_k
1
Cklk
[ estimate integration[
A*
integrated estimate Xklk
Figure 7: ,.\ scheme of distributed filtering of noisy and fuzzy measurements. The distributed filtering algorithm described here is optimal in the sense of minimizing local error covariances and the central error covariance. The algorithm is also dynamic, i.e.. the integrated results are colltinuouslv updated when new m e a s u r e m e n t s are available. The dvnainic capability of an algorithTn is important for real-time applications, because adding new measureme~llsdoes ilot reqtzit'e recomt)uting from scratch.
DISTRIBUTED DISCRETE FILTERING
251
5. A n E x a m p l e This example illustrates an application of the algorithm to multisensor integration. Consider one object moving on a two-dimensional surface in a near elliptical course with a constant speed, Fig. 8. Five sensors are located at five different places measuring the location of the object. Since each sensor provides only partial information about the object due to uncertainties in the sensor and fuzziness of measurement readings, we combine the measurements from these five sensors to get more accurate information about the object. The dynamics of the object is modeled by [ xk l Yk
=
I c~176 2 sin(1 ~
~k
I Xk-1 ~]k-1
- - 2l s i n ( l ~ )
cos( 1~) (I~k-- 1
.-~
~k--1
Wxk-1
,
__W_Wk_I
e~,
N(0, Q~-I)
(21)
Wyk_ 1 Wk--1
where z_k_1 and w__k_1 are 2 by 1 vectors, and Ok-1 is a known 2 by 2 non-fuzzy matrix. The measuring processes of the five observers are described by the following five measurement models, each of which is represented at their own local coordinate systems xk
=
xk
z yk
+
Yk
xk i ' v__~ ~ X(0, n*~),
i = 1,...,5
(22)
Vyk
i and v ki are 2 by 1 vectors (the fuzziness of the system parameters enters through where z'k, measurements z__. ik and Q~-I) and measurement matrices H i, i=1,...,5 are identity matrices. The communication networks among the sensors are given by z__.ik = J i z _ k +
T i'
(23)
i = 1, ..., 5
where the rotation matrices ji are
sin(0/)
cos(0/)
where 01 = 4 5 ~ 0 2 - - - 1 5 ~ 0,3=0 ~ 0 4 = 9 0 ~, and 0 5 = - 4 5 ~. The translation vectors
T i
are 10
T ~ = [010 ' --T2=
9
T 3
24] T4 [ 51 10
' --
-8
[6] ' --
-13
"
252
LANG HONG
The communication networks are assumed to be known exactly. Fig. 9 (a) presents the noisy measurements (due to stochastic uncertainties only) from each sensor, and Fig. 9 (b) shows the noisy and fuzzy measurements (the dotted line denotes the noisy measurements containing stochastic uncertainties and the squares represent the fuzziness). The a-cut of the fuzziness considered in this chapter is c~ = Pip-0; the other or-cut levels can be easily obtained using the c~-cut representation during the process. The fuzziness intervals for the example are given as follows.
Fuzzy elements
Fuzzy intervals
q;_,
0.1
R k . i = l
.....
0.2
5
Polo
0.3
The stochastic uncertainties for the example are specified by
Qk-1
=
[0.2 0
0 0.2
,
R/k
=
10 0
1
i=l
'
.... 5 a n d P o t o =
1 0] 0
1
9
Fig. 10 gives the result of distributed integration based on the noisy and fuzzy data. It's easy to see that the fuzziness of Fig. 10 is reduced compared to that of Fig. 9 (b). Fig. 11 depicts the optimally integrated estimate without considering the fuzziness of the data. It should be noticed that taking the measurement fuzziness into consideration doesn't mean that the estimated result of the Kalman filter should be improved. Actually, the estimation of the Kalman filter and the data fuzziness don't interfere with each other. The Kalman filter finds out the optimal estimation (in the sense of minimizing the trace of the estimate error covariance matrix), while the fuzziness represents the vagueness of the estimation. 6. C o n c l u s i o n s This chapter presents an algorithm for distributed discrete filtering of noisy and fuzzy measurements and applies the algorithm to multisensor integration. Both computational efficiency and the capability of dynamic sensor integration are achieved.
The approach
discussed here keeps data fuzziness away from the effect of long-term fuzzy arithmetic and compresses the fuzziness in a reasonable range.
DISTRIBUTED DISCRETE FILTERING 30
>,
.
.
.
253
True trajectory .
0
-10
-20
%
-~'0
-,'0
;
i;
2'o
~0
x
Figure 8: The true trajectory of the object.
40
.
Mutlsensor noisy measurements . . . .
.
20
10
>,
0
-10
-20
-30
-40 -40
-30
-20
-10
x
0
10
20
30
Figure 9 (a)" Noisy measurements of the object locations (containing only stochastic uncertainties).
254
LANG HONG Multisensor fuzzy and noisy measurements
4~I 3O
>"
0
-10
-20
-30
-40 -40
-30
-20
-10
0
10
20
30
Figure 9 (b)" Noisy and fuzzy measurements of the object locations (containing both stochastic uncertainties and fuzziness). Centralized integration of fuzzy and noisy data
>,
0
-10 -15 -20 -25
5
-10
-5
0 x
5
10
Figure 10: The result of integrating the estimates derived from the noisy and fuzzy measurements.
DISTRIBUTED DISCRETE FILTERING 25
.
255
Integrated estimate from noisy measurements only . . . .
0
-10 -15 -20
-25 •
Figure 11" The result of integrating the estimates derived from the noisy measurements only.
References [1] D. Dubois and H. Prade, Fuzzy Sets and Systems: Theory and Applications, Academic Press, New York, 1980. [2] L. Hong, "Centralized and Distributed Multisensor Integration with Uncertainties in Communication," IEEE Trans. Aerospace and Elec., Vol. 27, pp. 370-379, 1991. [3] R. E. Kalman and R. S. Bucy, "New Results in Linear Filtering and Prediction Theory,"
J. Basic Engin., Vol. 83D, pp. 95-107, 1961. [4] A. Kaufmann and M. M. Gupta, Introduction to Fuzzy Arithmetic, Van Nostrand Reinhold, New York, 1984. [5] S. Schnatter, "On Statistical Inference for Fuzzy Data With Applications to Descriptive Statistics," Fuzzy Sets and Systems, Vol. 50, pp. 143-156, 1992.
256
LANG HONG
[6] S. Schnatter, "Linear Dynamic System and Fuzzy Data," in Cybernetics and Systems (R. Trappl, Ed.), World Scientific, pp. 147-154, 1990. [7] H. W. Sorenson, Ed., Kalman Filtering: Theory and Application, IEEE Press, 1966. [8] L. A. Zadeh, "The Concept of Linguistic Variables and Its Application to Approximate Reasoning, Parts 1, 2, 3," Inform. Sci., Vol. 8, pp. 199-249, Vol. 8, pp. 301-357, and Vol. 9, pp. 43-80, 1975.
Algorithms for Singularly Perturbed Markov Control Problems: A Survey 1 Mohammed Abbad Departement de Mathematiques et Informatiques Faculte des Sciences de Rabat, Morocco
Jerzy A. Filar School of Mathematics University of South Australia
Introduction Finite state and action controlled Markov chains (CMC's, for short) are dynamic, stochastic, systems controlled by a controller, sometimes referred to as "decision-maker". These models have been extensively studied since the 1950's by applied probabilists, operations researchers, and engineers. Operations Researchers typically refer to these models as ::Markov decision processes". The now classical CMC models were initially studied by Howard [25] and Blackwell [7] and, following the latter, were often referred to as "Discrete Dynamic Programming". During the 1960's and 1970's the theory of classical CMC's evolved to the extent that there is now a complete existence theory, and a number of good algorithms for computing optimal policies, with respect to criteria such as maximization of limiting average expected output, or the discounted expected output (e.g, see Derman [14]). These models were applied in a variety of contexts, ranging from water-resource models, through communication networks, to inventory and maintenance models. Furthermore, in his book published in 1977 Kushner [31] proposed a computational method for solving controlled diffusions based on approximations by Markov decision processes. The effectiveness of this approach is now well accepted (e.g, see Kushner and Dupuis [32]). Over the past two decades there has been a steady stream of graduate-level texts and monographs by Ross [35], Federgruen [18], Denardo [13], Kallenberg [26], Heyman and Sobel [24], Tijms [40], Kumar and Varaiya [30], Hernandez-Lerma [23], White [41], Sniedovich [39] and Puterman [34] that indicate the continued research interest in these topics. However, in recent years a new generation of challenging problems in CMC's began to be addressed. One class of these problems focussed around the following question: In view of the fact that in most applications the data of the problem are known, at best, only approximately; how are optimal controls from the complete information models affected by perturbations (typically 1We are indebted to M. Haviv for a number of helpful discussionsand for his comments on an earlier draft of the manuscript. CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
257
258
MOHAMMED ABBADAND JERZYA. FILAR small) of the problem data?
From the practical point of view the above question is of vital importance; however, it leads to challenging mathematical problems arising from the following natural phenomenon: 9 If the perturbation of a Markov Chain alters the ergodic structure of that chain, then the stationary distribution of the perturbed process has a discontinuity at the zero value of the disturbance parameter. This phenomenon was illustrated by Schweitzer [37] with the following example:
P(e) - ( 1- e/2e/2 1- e/2e/2)
Let
be the perturbed Markov Chain whose stationary, distribution matrix is p . ( ~ ) _ ( 1 / 2 1 / 21/2 1/2) for all e E (0, 2]. Thus we have
~,0
1/21/2
-~ P ' ( 0 ) -
01
"
It will be seen in sequel that the above difficulty leads to a very natural, and challenging problem in the theory of singularly perturbed Markov Chains. As with all surveys, it is a difficult problem to determine which of the many interesting contributions should be explicitly cited and/or discussed. In order to keep the size of this survey manageable, and also to increase its potential appeal to control theory practitioners, we have tried to follow the guidelines below: 1. Only finite state/finite action CMC's were considered. 2. We have focussed on constructive methods and algorithmic procedures. 3. ~Ve emphasized the results which analysed, the more difficult, singular perturbations. While the decisions to include or exclude any given result were (of necessity?) made subjectively, we hope that the bibliography is comprehensive enough to enable the reader to at least begin tracing the contributions of most of the researchers active in this field.
SINGULARLYPERTURBED MARKOVCONTROLPROBLEMS
2
Singularly
Perturbed
Markov
259
Chains
In order to analyse the perturbed Markov control problems, we must first understand the uncontrolled case that is equivalent to the controller having only a single strategy at his/her disposal. This sub-topic is sometimes called the perturbation theory of Markov chains and is of interest in its own right. In what follows, we concentrate only on the results that have found extensions to the controlled case. We begin by considering a specially structured class of Markov chains that has received the most attention in the literature. A nearly decomposable Markov chain is defined by an N x N irreducible transition probability matrix P(e) of the form P(e) = P + eA, where
p_
/
P~ o ... o / o
P2 ...
o
0
0
P~
...
Pi is an ni x ni irreducible transition probability matrix, i = 1,2, ..., n. This class arises naturally in many applications of large scale finite state Markov chains. It is characterized by a decomposition of the states into groups, with strong interactions between states in the same group, and weak interactions between states in different groups. The strong-weak interaction structure was first introduced by Simon and Ando [38]. Courtois [9] developed the first analytical techniques for this class, and applied it to many problems in queueing networks and computer systems. The fundamental problem to be analyzed for this class of Markov chains is the computation of the stationary distribution. This problem suffers from high dimensionality and ill conditioning. Courtois gave an aggregation procedure for the computation of an o(c)-approximation of the stationaw distribution of P(e). His procedure is as follows: 1) Compute the stationary distribution p~ of Pi, i - 1, 2, ..., n. 2). Form an aggregated transition matrix
P(E) = s~ + ~VAW,
(1)
260
MOHAMMED ABBAD AND JERZY A. FILAR
where V is an n x N matrix and W is an N x n matrix, which are defined by:
V-
0 I~7 p[
... ...
0
... PT,
0
and 0 W=
In 2 ... 0
0
...
0 / 0
(2)
0/ 0
(3)
l n.
where In, is a column vector consisting of ni ones, i = 1, 2, ..., n. 3) Compute the stationary distribution ~" of P(e). Note that ~)" does not depend on e. 4) The approximate stationary distribution is given by: O'V.
(4)
The appeal of the nearly decomposable case stems from the fact that the originally uncoupled groups of states can still be treated in a largely independent manner, with the results connected only later on as is done in steps 2-4 of the above algorithm. This can be easily seen from the following simple example. E x a m p l e 2.1
P(e) = P + eA =
IZ I I II I~110 I 1
0
1
0
0
0
0
~
0
0
0
1 0
1
0
-i
+ e
I) 1 P T = (~
2 3 2 ~ ) and p ~ = ( -g -~) .
0
0 0
9
SINGULARLYPERTURBED MARKOV CONTROLPROBLEMS
261
2)
)
and
W =
I1~ 1 0 0
0 1 1
"
1 1 ) VAW
-
23 5
~2
9
5
3) ~) T h e a p p r o x i m a t e s t a t i o n a r y d i s t r i b u t i o n is:
i--i I---i
)"
Of course, it can be easily checked that Courtois's algorithm will fail in many cases where the nearly decomposable structure is no longer present. This points to the need to develop an anologous theory for the more general perturbed Markov chains. Based on the theory of Kato [27] for the perturbation of linear operators, Delebecque [10] derived a more general formula for the approximation of the stationary distribution matrix. Below, we present his approach. Let, d(e) = do + edl + e2A2 -k-...,
e e [0, eo] ,
be a family of perturbed generators of Markov chains 2. The unperturbed generator Ao is an mo x mo which contains my ergodic classes and possibly some transient states. It is assumed that A1, A2, ... are also generators of Markov chains. Let P~ be the stationary distribution matrix corresponding to the generator Ao. We define: 2Recall that a generator of a Markov chain is obtained by subtracting the identity matrix from its probability transition matrix
262
MOHAMMED ABBAD AND JP_,RZYA. PILAR
qk := (qk(s), s = 1, 2, ..., m0)T; k = 1,2, ..., m I where qk(s) is the probability (with respect to the Markov chain defined by the generator A0) to be absorbed in the class k starting from s. We define an m0 x m1 matrix W1, and an mx x m0 matrix V1 by: ~V1 "-- [qk(8 I l k = l - m 1 \ /is=l-too
(5)
ls=l-mo ksJk-l-ml
(6)
and rm
Vl "--[
where, inks "-
{ [P0k]s, 0,
if s is in the ergodic class k otherwise
(7)
where P0k is one of the identical rows of Po corresponding to the ergodic class k. Note that from the definitions of W1 and Vl, it follows that:
Po = W1V1 and VI W1 =
I m ~x m , .
(8)
Let Ho be the matrix defined by: Ho := (Po - Ao)
- Po.
(9)
A "reduced series" is constructed by: A I ( c ) - A~ + cA I + e2A{ + ..., where A~ := VI[PoAIPo]W1
AI "- VI[P~A2Po + PoA,HoA,P~]W1
(i0)
SINGULARLYPERTURBEDMARKOVCONTROLPROBLEMS
263
n
AI-1 :-- V I [ E p=l
P~ (Akl HoAk= ) ... (Ak,_, HoAk, ) P~ ]W1 E kl +...kp=n
Now, repeat the construction above with the series A(e) replaced by Al(e). A new series A2(e) and new matrices Pi*, H1, 1/2, W2 are constructed. By induction a sequence of series of mi x mi matrices is defined, where mo>_ml >_m2>_ .... Now, the following results were established by Delebecque[10]. T h e o r e m 2.1 (i) Every series in the construction above is a generator of a Markov chain, for e small enough. (ii) The process of reduction described above has only finitely many steps; say, k steps. (iii) Let P* (e) be the stationary distribution matrix corresponding to the generator A(e). Then, lim P*(e) e.-+o
-
W1W2...WkP~VkVk_I...V1.
/ 1~ o/ / ~
E x a m p l e 2.2 let,
0100 0010 0 0 0 1
P(e)-
O0 0 10-1 0 1 0
+
o / / -lo 1 o/
0 0 -1
0 -1
+e
1
--101 1 00 -I 0 0
"
Hence,
/000 0/ Ao-
0 0 0 1 0 -1 0 1 0
0 0 -1
A1 =
/10 10/
'
0 -1 1
-1 1 -1
0 1 0 0 0 0
,
Ak--0,
From (5) and (6), we have that,
V1--
1
0
0100
0
O)
and W I =
I
i 0 01 I 0 01
o
k>2.
264
M O H A M M E D ABBAD AND JERZY A. FILAR
From (8), we have that,
P o ~"
I 1 00 010 0
I
0
1 0 0 0
"
0 1 0 0
From (9), it follows that,
I
H0
O 0 00 O 0 00 1 -i 0 10 " 0
-101
The coefficients of the reduced series (10) are: A~=
00
1
-1
....
Now, new matrices are constructed:
(~0 1 o I ' ~1_(o0 0 o I "
PI=P~=V2=W2=
The coefficients of the next reduced series are:
-11
A2~ =
- 11
i
- Ai2 - """
The matrices that correspond to the above series are:
~_(Ol ol)
,P~=
~
, H~=
- ~, ~
Also, we have that:
V3=
(,1~ ~
~
, l~V3=
(1) i
"
-i ~
.
SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS
265
The process of Delebecque stops at this stage, therefore k = 2. Hence:
limP*(e) e--+0
-
W1W2 P:~V2 V1 =
/ii~176 / 0 0 ~ ~ 0 0
"
Note that, when the procedure of Delebecque is applied to the case of nearly completely decomposable Markov chain, it gives exactly the method of Courtois. Let P be a transition probability matrix for some Markov chain. Suppose that the perturbation is such that P(E) - P + eA is a transition probability matrix for some irreducible Markov chain for all 0 < e < e~ Let 7r(e) be the unique stationary distribution of P(e). Schweitzer [37] showed that for all e > 0 small enough, or
i
i=O
(ii)
where 7r(O = 7r(o)Ui for some fixed matrix U. Since 7r(e) satisfies, ~r(e) = 7r(e)(P + cA),
(12)
it follows from (11)and (12)that,
(I - P ) = 7r(j+l)(i_ P) - ~(J)A = 0 j _ 0 , 1 , . . . Since P may contain more than one ergodic class, the set of equations, { 7r(~ P) = 0 7rOe = 1 where e is the vector of ones, is not sufficient to determine 7r(0) uniquely. Hassin and Haviv [19] developed an algorithm for computing k" such that:
266
MOHAMMED ABBAD AND JERZY A. FILAR 7r(~ e - 1 P) =
7r(~
0
vr(J+')(I- P) - 7r(J)A = O, j - O, 1,..., k" - 2 is the minimal set of equations needed to determine rr (~ uniquely. If k" = 1, then the only set of needed equations is: rr(~ e = 1 7r(~ P) - 0 Let M(e) be the matrix with entries Mij(e), where :YIij(e) is the mean passage time from state i to state j when transitions are governed by P(e). From the theory of Markov chains, it is well known (e.g see [17], [28]) that,
( I - P(e))M(e) = J - P(E)Md(e),
(13)
where Md(e) is a diagonal matrix whose diagonal coincides with the diagonal of M(e), and all entries of J are one. From (13), it is clear that M(e) admits Laurent series expansion. Let uij be such that M~j(e) = o(e-u'J) which means that u~j is the order of the pole of M~j(e) at zero. The following relation is due to Hassin and Haviv [19],
k" = 1 + max(uij
-- ujj).
The algorithm for the computation of uij for all i and j is stated below (see [19]). Fix a state s. Consider the graph G = (V, Er, E~) where V is the set of states, E r = {(i,J)l pij > 0} is the set of regular arcs, and E~ = {(i,j)[ Aij > 0} is the set of epsilon arcs. For a subset C E V, let 6(C) = { ( i , j ) e E.r U Er e C,j cI C}.
Step1
(Initialization) Construct a graph G'--- (V', E',., Ere) from G by deleting all loops (i, i) e Ee and all edges going out of s. Set u(i) := 0 and S ( i ) : = i for all i E V. S t e p 2 (Condensation of cycles)
SINGULARLYPERTURBED MARKOV CONTROLPROBLEMS
267
If G' contains no directed cycles of regular arcs, go to Step3. Let C be a directed cycle of regular arcs. Condense C into a single vertex c. If ~(C) A E'~ -r q}, set u(c) := max{u(i)li E C}. If ~(C) C E'~, set u(c) = 1 + max{u(i)li E C}, and set E'~ := E'r U ~(C),
E'~:=E'~\6(C). Set S(c) := U,ecS(i). Repeat Step2. Step3 Set T := V'. Let u(j) (i, j) E E'~ where i E T, u(i) := max{u(/), u(j) If T = (s}, go to Step4.
= max{u(i)li E T}, and set T := T \ { j } . For set u(i) := u(j). For (i,j) E E'~ where i E T, set 1}. Otherwise, repeat Step3.
Step4 (Computation of u(i) : uis) {S(vf)lv' E V'} is a partition of V. For each v E
that v E S(v') and set u(v):= u(v'). Set u(~):: m~x{m~• i) e E~},m~•
V\{s},
find v' E V' such
~r(~,i) e E~}}.
oo
The computation of the sequence {~r(0}i=0 by the minimal set of equations is given in [21]. The following example, based on an example given in [19], illustrates the above procedure. Note that the data of this example is the same as Example 2.2 which was used to demonstrate Delebecque's procedure. E x a m p l e 2.3 Let,
P(e)
/ 1000/ / -1010 /
= P + eA
=
0100 1000 0100
+e
0 -1 1
-101 1 00 -100
(i4)
The graph G is defined in Figure 1 where the regular arcs are represented by bold arrows and epsilon arcs by dashed arrows. We proceed through the steps of the algorithm with the fixed state s = 2. In Step1, we consider the graph represented in Figure 2. We have that u(1) = u(3) = u(4) = 0.
268
MOHAMMED ABBAD AND JERZYA. FILAR
s
s
s
"<__@ Figure 1 ~ B
s
s
s
s
Figure 2 The algorithm moves to Step2. There is only one regular cycle" (1). All arcs emanating from the regular cycle (1) are epsilon arcs. Hence u(1) = 1, and the next graph to be considered is Figure 3. There is a regular cycle (1,3).
s s S ,%
Figure 3 It is condensed to a vertex c. All arcs emanating from the cycle (1,3) are epsilon arcs. Hence u(c) = 2. Now we have the graph as shown in Figure 4. There are no regular cycles. The algorithm moves to Step3. Hence u(c) = 2 and T = {2, 4}, only one iteration is needed here (see Figure 4) which ends with u(4)= 1. The algorithm moves to Step4. Hence, u(1) = u(3) = 2 and since u(4) = 1, we get that u(2) = 2.
SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS
269
Figure 4
Figure 5 We now have that u(1) - u(3) - 2, u(4) - 1, and u(2) - 0. Recently, Lasserre [33] considered the more general form of a singular perturbation:
Pd "= P + D,
(15)
where P is any transition probability matrix, and P~ is a transition probability matrix with a single ergodic class. The following formulae were derived (see [33]) for the stationary distribution matrix P~ and the fundamental matrix Zd of the perturbed transition matrix P~:
P~ = P ' ( I + D H -1 Z) ee T z~ = (I + ~
- P~)H-~z n
eer _ ZD) and Z "- ( I - P + p~)-I the fundamental where H := ( I - P* + --7matrix of P.
3
Singularly P e r t u r b e d Controlled Markov Chains
A discrete controlled Markov chain (CMC, for short) is observed at discrete time points t = 1,2, .... The state space is denoted by S = { 1 , 2 . . . , N}. With each state s E S we associate a finite action set A(s) = { 1 , 2 , . . . . m s } .
270
M O H A M M E D ABBAD AND JERZY A. FILAR
At any time point t the system is in one of the states s and the controller chooses an action a E A(s); as a result the following occur: (i) an i m m e d i a t e reward r ( s , a ) is accrued, and (ii) the process moves to a state s' E S with transition probability p(s'ls, a), where p(s'ls, a) > 0 and ~s'esp(s'ls, a) = 1. Henceforth, such an CMC will be synonymous with the four-tuple r' - (S, {A(s) 9 s e S}, { r ( s , a ) ' s e S,a e A(s)}, {p(~'l~,a) " e
s, a
e
A decision rule 7rt at time t is a function which assigns a probability to the event that any particular action is taken at time t . In general ~Tt m a y depend on all realized states up to and including time t, and on all realized actions up to time t. Let ht - (So, a0, s l , . . . , at-l,St) be the history up to time t where ao e A(so),... ,at-y e A(st-1). ~Tt(ht, at) is the probability of selecting the action at at time t given the history hr. A strategy 7r is a sequence of decision rules ,'T = (~0 . r l , . . . , 7rt,...). A Markov strategy is one in which 7rt depends only on the current state at time t. A stationary strategy is a Markov strategy with identical decision rules. A deterministic strategy is a stationary strategy whose single decision rule is nonrandomized. Let C, C(S) and C(D) denote the sets of all strategies, all stationary strategies and all deterministic strategies respictivelv. Let Rt and E,~(Rt, s) denote the random variable representing the immediate reward at time t and its expectation when the process begins in state s and the controller follows the strategy ft. The overall reward criterion in the limiting average CMC F is defined by 1 J (s, ,-r) := lim inf T + I T--,oo
T
~E~(Rt's)'
seS,
,'reC
t--O
A strategy 7r~ is called optimal if
J(s ,'1~ - m a x J ( s , ,'1) for all s e S rt E C
(16)
The overall reward criterion in the discounted CMC F is defined by
J'~ ( s) = m a x J'~ ( s, zr) for all s E S ~rE C
(17)
SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS
271
where,
s~
~) := Z
~'E~(R,, ~) , ~ e S, ~ e C
t=O
A strategy ~r~ is called a-optimal if jO(s ~ o ) = ma• J~(s, ~) for m s e S ~EC
It is well known that there always exist optimal and a-optimal deterministic strategies and there is a number of finite algorithms for its computation (e.g., Derman [14], Howard [25], Kallemberg [26], Ross [35]). We shall now consider the situation where the transition probabilities of F are perturbed slightly. Towards this goal we shall define the disturbance law as the set
D = {d(s'ls, a)ls, s' e S, a e A(s)} where the elements of D satisfy (i) ~s, es d(s']s, a) = 0 f o r all s 6 S, a e A(s) (ii) there exists eo > 0 such that f o r all e E [0, e0] p(s']s, a) + ed(s']s, a) >_ 0 f o r all s E S, a e A(s). Now, we have a family of perturbed finite markovian decision processes F~ for all e E [0, e0] that differ from the original CMC F only in the transition law, namely, in r~ we have p~(s'[s, a) := p(s'ls , a) + ed(s'ls, a) f o r all s, s' e S, a E A ( s ) . The limiting average CMC corresponding to r~ is the optimization problem J~(s) := max 4(s, rr) s 6 S ~EC
(L~)
where J~(s, 7r) is defined from F, in the same way as J(s, 7r) was defined in F.
Similarly, the discounted CMC corresponding to F~ is the optimization problem
272
MOHAMMED ABBAD AND JERZY A. FILAR
J:(s) := maxJ,~(s 7r) s E S rrEC
(D~)
~
For every 7r E C(S) we define: (i) The Markov matrix P(Tr) = (pss,(Tr))s.N,=l, where pss,(Tr) :=
X~ P(S'ls, a)Tr(s,a)
aEA(s) for all s, s ~ E S; (ii) the Markov matrix P,(,'r) = (p]s,(Tr)),~,=l where Pess,(~) :- F-,aEa(s)Pc(St[S, a)7[(s, a) for all s, s' E S; (iii) the perturbation generator matrix D(Tr) - (d~,(,'r))~v~,=, where d~,(Tr) "-EaEa(s) d(stlS, a)Tr(S, a); and (iv) the stationary distribution matrix of P~(,-r): P,'(,'r) -(p;;,(,-r)) ~.,,=1 N := lim 9
t ~
1 t +
1
P~(,'r) k=o
where P~ := IN an N x N identity matrix. The stationary distribution matrix P'(,'r) of P(Tr) is defined similarly. Note that for every ,'r E C(S)
P,(,'r) = P(,'r) + eD(Tr).
(18)
With every 7r E C(S) we associate the vector of single stage expected rewards r(Tr) = (r~ (Tr),..., rN(,'r)) T in which rs(,'r) := ~aeA(s) r(s, a),'r(s, a) for each
sES. It is well known that for each stationary strategy. 7r E C(S)
J,(s, 7r) = [P,'(,-r)r(Tr)]s, s E S
(19)
Just as in the case of uncontrolled, singularly perturbed, Markov chains the nearly decomposable assumption leads to elegant solution methods. Furthermore, this assumption has the following appealing physical interpretation: Imagine that the controlled system consists of a number of, largely
SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS
273
autonomous, components that are coupled by infrequent "disturbance" from a central unit. Clearly this induces the nearly decomposable structure. We now formally define the nearly decomposable controlled Markov chain and devote the remainder of this section and the entire following section to the discussion of the many desirable properties of this special and important class of CMC's. Let F be an (unperturbed) CMC. In addition, we shall assume the following: A1) S = U ~ = l ~ where Si n sj = O i f / n l + n2 + ... + nn = N.
76 j, n > 1, card S/ = ni,
A2) p(s'ls , a) = 0 whenever s E Si and s' E Sj, i -76 j. Consequently, we can think of F as being the ':union" of n smaller CMC's ri, defined on the state space Si, for each i = 1, 2, ..., n, respectively. Note that if 1"Ii is the space of stationary strategies in Pi, then a strategy 7r E C ( S ) in F can be written in the natural way as 7r = (7rl, 7r2, ...TRY), where 7F E Ili. The probability transition matrix in Fi corresponding to 7~; is of course defined by P~(Tri) := (ps,s,(Tri))s,s, es~. The generator G~(rri) and the stationary distribution matrices P/" (Tri) matrices can be defined in a manner analogous to that in the original process F. A3) For every i = 1,2, ...n, and for all 7ri E Hi the matrix Pi(Tr ~) is an irreducible matrix. In view of A3), P~(Tr/) is a matrix with identical rows. We shall denote any row of PT(Tr i) by p~(Tri). We shall now consider the situation where the transition probabilities of F are perturbed slightly. Towards this goal we shall define the disturbance law as the set
D = {d(s'ls , a)ls , s' E S, a E A ( s ) } where the elements of D satisfy (i) E~,~s d(s'ls, a) = 0 f o r all s E S , a E A(s). (ii) - 1 <_ d(s[s, a) <_ 0 f o r all s E S, a E A(s) .
274
MOHAMMED ABBAD AND JERZYA. FILAR
(iii)d(s'ls , a) > 0 for all s r s' E S , a E A(s). A4) For every 7r E II and every e E (0, e0], the transition matrix P~(Tr) is irreducible.
Note that as a result of the assumptions A1), A2), and A3) the rank of P[ (Tr) is 1, which is strictly less than n, the rank of P'(,'r). Consequently, the previous perturbation is indeed a singular perturbation in the sense of Delebecque [10], since it changes the rank of this matrix. The perturbed CMC F~, e E (0, %] defined as:
F( : {S,{A(s) ] s E E}, {r(s,a) ] (s,a) E S x A(i)}, {p((s',a,s) I
e s x
• S}>.
is called the nearly decomposable CMC. Note that the underlying CMC F is decomposable and it is the pertubation parameter e > 0 that induces the infrequent coupling and is responsible for the word "nearly", in the above definition.
4
Decentralized Algorithms
In the case of nearly decomposable CMCs, a number of algorithms have been developed. It is natural to hypothesize that, because of the nearly decomposable structure, it may be possible to appropriately compose optimal controls from the underlying autonomous blocks to form an optimal, or nearly optimal, strategy for the whole system. Often these methods transform the perturbed CMC into a singular perturbation form that appears in the theory of differential equations and then apply singular perturbation methods for control system analysis and design. In this direction we refer to Kokotovic and Phillips [29], Khalil and Aldhaheri [3].
SINGULARLYPERTURBED MARKOV CONTROLPROBLEMS
275
In [3], the authors consider a nearly decomposable CMC with the limiting average criterion. The transformation that has been used is defined by: T := [W W'] where W is defined as in (3), and W' is any matrix such that T is nonsingular. The inverse matrix T -1 can be written as:
T-1 :- [ Yl V2] Then, based on the policy improvement of Howard [25], the authors developed an algorithm for the computation of an optimal deterministic strategy for c fixed which can be stated by the following: S t e p l (Value Determination Step) Choose a starting deterministic strategy 7rk. (i) Compute the aggregated matrix Q~(Trk) and the aggregated immediate reward r~ (zrk) defined by: Q~(Trk) :=
V~(Trk)D(Tck)W
r~(~ ~) := y~(~)~(~ ~) where, V~(Trk):= (V, - V,[G(Trk) + eD(Trk)]W'(V2[G(Trk) + eD(Tck)]W')-'V2}, and G(Trk) := P(Tr k) - I. (ii) Solve for )~k and Clk which are uniquely determined by:
{
~e
= Q ~ ( . ~ ) r ~ + r ~ ( . ~) ~(~) = 0
where s is any state. (iii) Compute, c~ "= -[V2(G(zr k) +
eD(zck))W']-l(V2D(zck)WCkl + V2r(Trk))
Step2 ( Policy Improvement Step) For each state s E S, let:
7rk+'(s) "- argmin~A(~){r(s,a) + D(s,a)Wc~ + [G(s,a) + eD(s,a)]W'c~}
276
MOHAMMED ABBAD AND JERZY A. FILAR
where, D(s, a) := (D(s'ls, a))~,es and G(s, a) := (a(s'l~, ~))~'~s. If for some s, the minimum is achieved at Irk(s), let lrk+'(s) := irk(s). If ~rk+l = ~rk, Stop. Othewise let ~rk := ~rk+l and go to Stepl. Phillips and Kokotovic [29] studied the nearly decomposable CMC with the discounted criterion, and proposed the following algorithm for the computation of an approximate optimal solution. Below we present their approach. The model of the continuous time Markov chain is formulated by: dp
d-T = p ( a + eD),
(20)
where I + G + eD is a nearly decomposable Markov chain, and p is the Ndimensional row vector whose entries denote the probabilities Pi of being in state i at time T. In order to analyse the influence of weak interactions eD, the authors considered the change time scale to t := e7. Therefore, in the t-scale the model (19) becomes: dp
--
-
G
--+D).
dt - p ( e
(21)
In the discrete time, the model (20) has the analog: p(k + 1) = p(k)( -G + I + D) E
(22)
It is well known (e.g see Ross [35]) that the optimality condition with respect to the discounted reward criterion for the model decribed above is: J~=
max {a(G(Tr) + D(1r) + I ) 4 ~ + r(n)}
,~C( O)
e
(23)
From the analysis of (22), the following algorithm is developed. Step1 Choose an N-dimentional vector Vk Step2 For each i = 1 , 2 , . . . , n , compute the optimal value v~+ 1 of the CMC Fi
SINGULARLYPERTURBEDMARKOV CONTROLPROBLEMS
277
restricted to the state space Si, in which the rewards, for any deterministic strategy zri, are defined by:
[I + D(Tri)]wvk + ri(Tri),
(24)
where I + D(Tri) is an ni • n matrix whose entries are defined by: [I + D(~ri)]ss, := 5~, + d(s'[s, 7ri(s)), s e Si, s' e S Set Vk+l = (V~+l,..., V~+l ). If Vk+l -- Vk, Stop. Otherwise, set Vk := Vk+l and go to Step1. Delebecque and Quadrat [11] considered a formulation of the problem in the context of management of hydrodams, and showed that their problem can be formulated with the help of a dynamic programming equation that is similar to the one given in (22). These authors showed that their equation is similar to the optimality equation of some discounted CMC in which the discount factor is perturbed. They proposed an aggregation-disaggregation policy improvement algorithm for the computation of an approximate solution to the problem defined in (22). In fact, they considered the case where the transition matrix P(Tr) may have some transient states, but with an additional assumption that the ergodic structure of P(zr) is unchanged by any stationary strategy 7r. Below, we present their approach restricted to the nearly decomposable case.
T h e o r e m 4.1 (i) For any stationary strategy 7~, the linear system:
1 G(Tr) + D(,x) + I)X + r(Tr) has a unique solution. (ii) The solution X~ admits a power series expansion: (x)
j=O
where, Vo(rr) and V1 (Tr) = (Jl (rr) + ~ (7i) are uniquely determined by:
278
MOHAMMED ABBAD AND JERZY A. FILAR
{ P*(Tr)(D(Tr)- -g-I)Vo 1-e 1-~p, (Tr)r(Tr) = 0 + ---gP'(Tr)Vo = I7o
{ G(~)IT1 + (D(~)-
1-"I)Vo + Cr /
P'(~)v~
=
1-~r(,~) = 0 O~
o
{ P"('a')(DOT)- 1---~I)1~71" t - _ P ' ( ~ ' ) ( D ( ~ ' ) - 1z~-~Z)gl--0 P'(~)VI - 171
(25) (26)
(27)
Based on the theorem above, the computation of Vo(Tr) and Vl(,'r), for any stationary strategy 7r, is given by the procedure:
Computation of Vo(Tr)" From (24), it can be shown that the computation of V0(,'r) can be performed by" i) Compute P'(;r), and let V(,-T) and W be the matrices defined as in (2) and (3) respectively. ii) Compute the aggregated generator: D(,~) : V(,~)D(,~)W. iii) Solve the linear system of n equations: 1 -af. o~
+b(~)2+
1 -a o~
~(~)
-
0
where ~(~r)"- V(~r)r(~). iv) Then V0(,'T) is given by" [Vo(7~)]s = X(i), s ~ s
Computation of Vl (Tr) " From (25), it can be shown that the computation of ~ (~)can be performed by: Let h "= ( D ( ~ ) - ~-~I)Vo(,'r)+--g1-~r(Tr). We define h i to be the truncation of h to set of states Si. ,
SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS
279
1) For each i - 1 , . . . , n solve the linear system: a i ( r c i ) X i q- h i - 0 p'~(rri)X i - 0
2) Then 1/1 (rr) is given by: [91
i=l,...,n.
s E Si,
-
Computation of V1 (Tr) "
From (26) and (24), it ca:: seen that the computation of V: (rr) can be performed in a similar way as for the computation of V0(~) with k~-~P*(Tr)r(rr) replaced bv P ' ( r r ) ( D ( , ' r ) - 1 - a I ~ V 1 "~
T
/
"
Now, we can state the algorithm for the computation of an a p p r o ~ m a t e solution: Stepl Select an arbitrary deterministic strategy 7r.
Step2 Compute V0(rr) and V1 (rr) by using the previous procedure.
Step3 Compute, rr'(s) "- arg min {D~(s a)Vo + G(s a)V: + . aEA(s)
~
'
1met
O~
where, D ~ ( s , a ) := ( D ( s ' l s , a) _ 1-a~a vss')s'ES. Step4 Set rr "- rr' and go to Step2, until convergence of V0 occurs.
280
5
M O H A M M E D ABBAD AND JERZY A. FILAR
Limit Markov Control Problem
The results of this section can be found in [1], [2] and [6]. T h e o r e m 5.1 For any stationary strategy 7r E C ( S ) , the limit stationary matrix, P'(zr) "- lirn~_+0P:(~) exists. The theorem above, suggests the following formulation of what is now called the limit Markov control problem:
max [P'(Tr)r(Tr)]s
s E S.
~C(S)
(L)
It is natural to hope that an optimal strategy, if one exists, to (L) could be approximately optimal for the perturbed problem (L~), when the perturbation parameter e is small. The next result shows that the problem (L) in fact possesses an optimal deterministic strategy. T h e o r e m 5.2 There exist a deterministic strategy 7r~ E C ( D ) and a positive number 6 such that for any e E (0, 6), 7r~ is a maximizer in (n~). Moreover, ,x~ is a maximizer in (L).
The limit control principle is stated in the following theorem. This principle confirms our intuition that any optimal solution in (L) is an approximate optimal solution of the perturbed problem (L~) for e sufficiently small. T h e o r e m 5.3 Let 7r~ be any maximizer in (L). Then, limmax [ J~(s 7r~ - J~(s) [- 0
e-+O s E S
'
The problem of finding an optimal solution of (L), by an efficient method, is still open. In the case of nearly decomposable CMCs two methods for the computation of such an optimal strategy are presented at the end of this section. However, the general case is more difficult. Below, we give only a heuristic for finding an optimal strategy for (L).
SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS
281
Let {~n}n~176be any sequence in (0, e0) converging to O. We define the following sequence of strategies: i) choose zr0 arbitrarily, ii) for n _ 1, we define zr~ as an optimal strategy in the perturbed CMC (L~n) obtained by the policy improvement algorithm with 0rn-1 as the starting strategy. In [1], it was shown that there exists n* such that for any n >_ n', 7r, - 7r,., and 7r,. is optimal in the limit Markov control problem (L). However the problem of estimation of n', or equivalently of e,., is still an open problem. In the discounted case, it was shown in [1] that the perturbation is which means that for any maximizer 7r0 in the original problem (Do), limmax i V~(s, 7r~ - V ~ ( s ) 1 = ~0
s6S
stable,
O.
The result above, shows that any optimal strategy in (Do) is an approximate solution of the perturbed problem (D~) for e sufficiently small, It is also possible to consider the general perturbation namely:
p~(Jl~, a)
-
p(s'l~, a) + d(s'ls, a),
s, s' e S, a e A(s).
The corresponding limiting average Markov control problem by:
Jd(s) := max [P~(~)r(~)]~ ~6C(S)
sES
(Ld')
(28) is defined
(29)
where P~(Tr) is the stationary distribution matrix of the transition matrix Pd(Tr) defined by: [Pd(Tr)]ss, "-- ZaeA(s)P~(*'I*, a)Tr(s, a). In general, P~(n) may not have a limit when I]dll -~ 0 as is illustrated by the following example.
282
MOHAMMED ABBAD AND JERZYA. FILAR
E x a m p l e 5.1 Assume that d = (dl, d2) and that there is only one strategy ~r which induces the perturbed transition matrix
Pd(Tr) -- ( 1-did2 1 -dl
"
(30)
The stationary distribution matrix is now given by d2
P~ (Tr) =
d_.._4z__)
d,~d2 d,~d~ dl +d2
(31)
dl +d2
but ( all+d2) dl has no limit as I[dl[--+ 0. However, in [1] it was shown that the general perturbation is stable in the case of unichain or communicating CMCs. That is, for any maximizer no in the unperturbed problem (L~) lim IIPd(Tro)r(Tro)IIdll-~O
Jd[I = O.
We recall that CMC is unichain if for any deterministic strategy n E C(D), the transition matrix P(Tr) has only one ergodic class. Similarly, an CMC is called communicating if for any s, s I E S, there exist a deterministic strategy ,-v E C(D) a natural number n such that [P"(Tr)]ss, > 0.
In the rest of this section, we consider only the nearly decomposable CMCs. In this case, from Section 2, we have the following formula for the limit stationary matrix/5~(7r) for every stationary strategy 7~ E C(S)" (32)
-
where, P ' ( ~ ) is the stationary distribution matrix of the irreducible transition matrix defined by the generator
V(~)D(~)W.
Based on the formula (29), the following two algorithms were developed in [2], for the computation of an optimal deterministic strategy in the limit Markov control problem.
SINGULARLY PERTURBED MARKOV CONTROL PROBLEMS
5.1
283
Linear Programming Algorithm
Consider the following linear programming problem (P)" maximize Ei~=l Esesi E.c:A(s) r(s, a)z~ subject to:
~
i (5~s, - P(s'ls, a))z~,
- - 01
S' E Si;i = 1 , . . . , n
sESi aEA(s) n
E
E
E
i =0; d(s'ls, a)zs~
E
j-1,...,n
i=l s'6S i s6Si aEA(s)
n
E E E i=1
i =1
sESi sEA(s)
i >0;i=1
Zsa u
~ 99
. n;
seSi, aeA(s).
It can be shown (see [2]) that an optimal strategy in the limit Markov control problem (L) can be constructed as follows.
Theorem 5.4 Let {zs,i li = 1 , . . . , n's E Si; a
E A(s)} be an optimal extreme solution to the linear program (P), then the deterministic strategy defined by:
7r(s) - a,
s E Si, i = l , . . . , n v=v z~i > O
is optimal in the limit Markov control problem (L). The linear program (P) is similar to the one given by Gaitsgory and Pervozvanski [18]. However, these authors used techniques different from those in
[2]. 5.2
Aggregation-Disaggregation Algorithm
The following algorithm is a policy improvement algorithm which converges in a finite number of iterations to an optimal deterministic strategy of the
284
MOHAMMED ABBAD AND JERZY A. FILAR
limit Markov control problem. Step1 Select an arbitrary deterministic strategy ~r in the CMC F, and set: [Tr(i)]~ : : 7r(s);
s 6 S,; i : 1, . . . ; n.
Step2 Compute p~(,'r(i)); qij(,x(i)); and c(i, zr(i)), i = 1,... ,n, j = 1 , . . . ,n. For each i = 1 , . . . , n, the computation of p~(Tr(i)) is done by solving the linear system:
{ x~p,(~(i)) - x ~ i
~-,s6Si X s
1 -
-
Step3 Solve for the unknowns A, Y l , . . . , Yn-1; the linear system:
I
n
A + y~ = c(i, 7r(i)) + E i = I q~j(Tr(i))yj
i = 1,...,n
y~ = 0
Step4 For each i = 1 , . . . , n , compute the deterministic strategy ,x'(i) obtained after one iteration of the simplified policy improvement algorithm, when the starting strategy is zr(i), for the CMC Fi with rewards:
j=l s'ES i
s e s,,
Step5 If 7r'(i) = lr(i) for a l l / = 1 , . . . , n . Stop. Otherwise 7r(i) +-- 7r'(i); i = 1 , . . . ,n and go to Step2.
a e A(s).
SINGULARLYPERTURBEDMARKOVCONTROLPROBLEMS
285
References [1] M. Abbad and J. A. Filar, "Perturbation and Stability Theory/or Markov Control Problems", IEEE Trans. Automat. Contr., Vol. 37, NO. 9, pp. 1415-1420, 1992. [2] M. Abbad, T. R. Bielecki and J. A. Filar, "Algorithms for Singularly Perturbed Limiting Average Markov Control Problems", IEEE Trans. Automat. Contr., Vol. 37, NO. 9, pp. 1421-1425, 1992. [3] R. W. Aldhaheri, H. K. Khalil, "Aggregation of the Policy Iteration Method for Nearly Completely Decomposable Markov Chains ", IEEE Trans. Automat. Contr., Vol. 36, NO. 2, pp. 178-197, 1991. [4] E. Altman and V. A. Gaitsgory, "Stability and Singular Perturbations in Constrained Markov Decision Problems", IEEE Trans. Automat. Contr., Vol. 38, NO. 6, 1993. [5] E. Altman and A. Shwartz, "Sensitivity of Constrained Markov Decision Problems", Annals of Operations Research, Vol. 32, pp. 1-22, 1991. [6] T. R. Bielecki and J. A. Filar, " Singularly Perturbed Markov Control Problem: Limiting Average Cost", Annals of Operations Research, Vol. 28, pp. 153-168, 1991. [7] D. Blackwell, "Discrete Dynamic Programming", Annals of Mathematical Statistics, Vol. 33, pp. 719-726, 1962. [8] M. Cordech, A. S. Willsky, S. S. Sastry and D. A. Castanon, "Hierarchical Aggregation of Singularly Perturbed Finite State Markov Processes", Stochastics, Vol. 8, pp. 259-289, 1983. [9] P. J. Courtois, "Decomposability: Queueing and Computing Systems", Academic Press, New York, 1977. [10] F. Delebecque, "A Reduction Process for Perturbed Markov Chains", SIAM J. App. Math., Vol. 48, pp. 325-350, 1983. [11] F. Delebecque and J. Quadrat, "Optimal Control of Markov Chains Admiring Strong and Weak interactions", Automatica, Vol. 17, pp. 281-296, 1981. [12] F. Delebecque and J. P. Quadrat, "Contribution of Stochastic Control Singular Perturbation Averaging and Team Theories to an Example of LargeScale Systems: Management of Hydropower Production", IEEE Tranc. Automat. Contr., Vol. AC-23, NO. 2, pp. 209-222, 1978. [13] E. V. Denardo, "Dynamic Programming ", Englewood Cliffs, NJ: PrenticeHall, 1982.
286
MOHAMMEDABBADAND JERZYA. FILAR
[14] C. Derman, "Finite State Markovian Decision Processes", Academic Press, New York, 1970. [15] N. V. Dijk, "Perturbation Theory for Unbounded Markov Reward Processes with Applications to Queueing ", Adv. Appl. Prob., Vol. 20, pp. 99-i 11, 1988. [16] N. V. Dijk and M. Puterman, "Perturbation Theory for Markov Reward Processes with Applications to Queueing Systems", Adv. Appl. Prob., Vol. 20, pp. 79-98, 1988. / [17] J. L. Doob, "Stochastic Processes", Wiley, New York, 1953. [18] A. Federgruen, "Markovian Control Problems", Mathematical Centre Tracts 97, Amsterdam, 1983. [18] V. G. Gaitsgori and A. A. Pervozvanskii, Theory of Suboptimal Decisions, Kluwer Academic Publishers, 1988. [19] R. Hassin and M. Haviv, "Mean Passage Times and Nearly Uncoupled Markov Chains", SIAM J. Disc. Math., Vol. 5, NO. 3, pp. 386-397, 1992. [20] M. Haviv, "Block-Successive Approximation for a Discounted Markov Decision Model". Stochastic Processes and their Applications, Vol. 19, pp. 151-160, 1985. [21] M. Haviv and Y. Ritov, "Series Expansions for Stochastic Matrices", Unpublished Paper, 1989. [22] M. Haviv, "An Approximation to the Stationary Distribution of Nearly Completely Decomposable Markov Chain and its Error Analisis", SIAM J. Alg. Disc. Math., Vol. 7, NO. 4, 1986. [23] O. Hernandez-Lerma, "Adaptive Markov Control Processes", in Applied Mathematical Sciences, Vol. 79, Spring Verlag, New York, 1989. [24] D. P. Heyman and M. J. Sobel, " Stochastic Models in Operation Research", Vol. 2, 1984. [25] R. A. Howard, "Dynamic Programming and Markov Processes"~ M. I. T. Press, Cambridge, Massachusetts, 1960. [26] L. C. M. Kallenberg, "Linear Programming and Finite Markovian Control Problems", Mathematical Centre Tracts 148, Amsterdam, 1983. [27] T. Kato, "Perturbation Theory for Linear Operators ", Spring-Verlag, Berlin, 1980. [28] J. G. Kemeny and J. L. Snell, "Finite Markov Chains", Van Nostrand, New York, 1960. [29] P. Kokotovic and R. G. Phillips, '9t Singular Perturbation Approach to Modeling and Control of Markov Chains ", IEEE Trans. Automat. Contr.,
SINGULARLYPERTURBEDMARKOVCONTROLPROBLEMS
287
Vol. AC-26, pp. 1087-1094, 1981. [30] P. Kumer and P. Varaiya, "Stochastic Systems: Estimation, Identification and Adaptive Control", Englewood Cliffs, NJ: Prentice Hall, 1986. [31] H.J. Kushner, "Probability Methods for Approximations in Stochastic Control and for Elliptic Equations", Academic Press, New York, 1977. [32] H.J. Kushner and P.G. Dupuis, "Numerical Methods for Stochastic Control Problems in Continuous Time", Springer Verlag, New York, 1991. [33] J. B. Lasserre, "A Formula [or Singular Perturbations of Markov Chains", Unpublished Paper, 1993. [34] M. L. Puterman, "Markov Decision Processes: Discrete Stochastic Dynamic Programming", John Wiley and Sons, New York, 1994. [35] S. M. Ross, "Introduction to Stochastic Dynamic Programming", Academic Press, New York, 1983. [36] P. J. Schweitzer, "Perturbation Theory and Finite Markov Chains", J. Appl. Probability, Vol. 5, pp. 401-413, 1968. [37] P. J. Schweitzer, "Perturbation Series Expansions for Nearly Completely Decomposable Markov Chains", Teletrafic Analysis, Comput. Performance Evaluation, pp. 319-328, 1986. [38] H. A. Simon and A. Ando, "Aggregation of Variables in Dynamic Systems", Econometrica, Vol. 29, pp. 111-138, 1961. [39] M. Sniedovich, "Dynamic Programming", M. Dekker, New York, 1992. [40] H. C. Tijms, "Stochastic Modelling and Analysis", Wiley, New York, 1986. [41] D. J. White, "Markov Decision Processes", John Wiley and Sons, Chichester [England], New York, 1993.
This Page Intentionally Left Blank
CONTROL OF U N K N O W N SYSTEMS VIA DECONVOLUTION Hsi-Han Yeh Siva S. Banda P . J . Lynch
Fright Dynamics Directorate, Wright Laboratory Wright-Patterson AFB, Ohio 45433
I. INTRODUCTION
Recently, a class of heuristic control schemes which have been widely used in industrial processes [ 1-10] have found applications in the aircraft control systems [2, 4-7]. This class of control schemes include Model Predictive Control [1, 8-9], Model Algorithmic Control [2-3, 6-7], Dynamic Matrix Control [10], and Output Predictive Algorithmic Control [4-5]. These schemes share a common feature of the on-line calculation of future control inputs that will steer the system along a desired reference trajectory, based on a given mathematical model of the plant. The control loop is closed by updating the calculation at each sampling instant with new measurements of the actual input and output of the plant. Generally, the algorithms involved make the system nonlinear and time-varying. They are usually too complicated for the system to submit to analytical treatment [ 1, 3]. However, if the prediction scheme is linear and if the CONTROL AND DYNAMIC SYSTEMS, VOL. 73 Copyright 9 1995 by Academic Press, Inc. All rights of reproduction in any form reserved.
289
290
HSI-HAN YEH ET AL.
C(z) z1 -1
G(z)
( 1- z )G(z)
I Fig. 1. Unconstrained Model Algorithmic Control with Closed-Loop Linear Prediction
control and output variables are unconstrained, the closed-loop system becomes linear [3-4, 8-10]. For Model Algorithmic Control (MAC), the controller for this special case of linear prediction is realizable by cascading an integrator with the inverse of the mathematical model of the plant [3]. (See Fig. 1) In MAC and its variants [1-10], the plant model is identified off-line. Where the unitsample response model (or, loosely speaking, impulse-response model) is used [2-3, 6-7], the inverse of the plant model is implemented as a deconvolution between the incoming signal and the unit-sample response model. Since at the n th step of a deconvolution, only the first n-1 signals in the input sequence and the first n signals in the unit-sample response sequence are needed, it is logical to question whether it is necessary to have the unit-sample response of the plant identified off-line and installed in the memory of the controller in advance. To put it more pointedly: Can the controller compute the unit-sample response sequence of the plant term-by-term while the system is in operation, and use the partially identified unit-sample response to generate the next-step control variable in real time? This chapter answers the above questions affirmatively, and goes on further to show that the output error sequence which contains the information on
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
291
the unit-sample response of the plant may be used directly in a deconvolution to generate the optimal next-step control. Section II shows that the deconvolution of two causal signals can be implemented in real time to obtain the unit-sample response sequence of a system, if a sufficient number of delays are allowed. Section III presents a design of self-adaptive control which computes the unitsample response sequence of the unknown plant on-line, and uses it to generate the next step control. In Section IV, a model reference control of an unknown plant via deconvolution is presented.
Section V gives numerical examples.
Comparison with unconstrained Model Algorithmic Control is made in Section VI. Section VII presents the conclusions.
II. REAL-TIME DECONVOLUTION
Deconvolution of causal signals C(z) and G(z) results in a causal signal U(z) given by
c (z) U(z) = G(z)
( 1)
Therefore, the implementation of a deconvolution involves the realization of 1/G(z). Since G(z) is a causal signal, it is represented by an infinite series
G(z) = gk z k + gk+l z ( k + l ) + gk+2 z(k+2) + ---
(2)
for some k > 0, where m may be called the number of delays in G(z). If G(z) is the output of a stable system then lim gn = g, and G(z) may be approximated n~oo by
292
HSI-HAN YEH ET AL.
N-l G(z) ---
E
.
z-(k+N)
gk+iz'(m+l) + gk+N"''-'--~ 1 - fz
(3)
i=0 for sufficiently large N. The constant f is a forgetting factor, 0 < f <_ 1. While 1/G(z) is not realizable for k > 0 it can be easily verified that z-k/G(z) can be realized as in the diagram of Fig. 2, where the unit delay is represented by a triangle.
gk+N z
+
+
gk+N-1 -1 z
I Q O
O O
O
O
D O O
gk+2 +~
t
~
+
gk+l
z -1
z
1
gk
L
Fig 2. Realization of z-k/G(z)
f
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
293
Thus, the deconvolution of two causal stable signals can be implemented on-line with a shift-register and a memory bank of finite memory storages, by allowing a sufficient number of delays. Since the computation is hardware implemented, the computation time required is negligible.
The
following development explores the use of on-line deconvolution in two approaches to the adaptive control of unknown systems, namely, self-adaptive control and model reference control.
III. SELF-ADAPTIVE CONTROL A(z)
R(z>
_~
+
% (Z> i
E(zk/ !
r- [ B(z)
l ]
U(z~
~ ^ ..- G(z)
[C(z) Unknown System
Fig 3. Simultaneous Identification and Control
Consider the block diagram of Fig. 3 where G(z) is the unknown system, A(z) and B(z) are arbitrary design parameters, and Gc(z) is the adaptive controller to be designed. It is readily seen that
E(z) = B 1- ~ G(z) = (1- G(z)Gc(z))R(z)- G(z)A(z)
(4)
294
HSI-HAN YEH ET AL.
Choosing -1
B(z) =A(z)
(5)
[Gcl(z) - G(z)]Gc(z)R(z) + [(3(z) - G(z)]Afz) = 0
(6)
and rearranging Eq. (4) give
If the controller is built in such a way tllat
Gc(z) - ^
1
G(z)
(7)
in other words, if the output of the controller is the deconvolution of R(z) and Cr(z), then Eq. (6) may be rewritten as
[CJ(z) - G(z)]U(z) = 0
(8)
where U(z) is the control signal to the unknown system. The implication of Eq. (8) under the conditions Eq. (5) and Eq. (7) may be interpreted as follows: (a) the negative of the inverse of the z-transform of the excitation signal A(z) is implemented as the transfer function of the error processor B(z), (b) the inverse of the z-transform of the output of the error processor is implemented as the transfer function of the controller Gc(z) and (c) if the input to the plant or the controlled system is non zero for some t = nT, n > 0, then the output of the error processor is the unit-sample response of the unknown plant or controlled system. Furthermore, in this implementation, the error signal E(z) is given by
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
E(z)
295
B(z) (9)
= -A(z)6(z) = -A(z)G(z)
Since A(z) is arbitrary, it may be chosen to minimize the error signal e(nT). Controller
+ V Et
~(z) Identifier
/A(z)
A
G(z) = G(z) If U(z)~f 0
Fig. 4 Identification and Control via Deconvolution
The above observation leads to a scheme of an adaptive control system which identifies the unit-sample response sequence of an unknown plant while at the same time uses this sequence in the adaptive controller to force the plant output to follow the command signal. The plant transfer function is unknown but its unit-sample response can be measured as the output of the error processor, if the controller transfer function is the inverse of the plant transfer function. Without loss of generality, one may assume that the unknown system has one delay (k=l) in its unit-sample response. An implementable adaptive control system can be drawn from the above analysis. (see Fig. 4) The error processor unit in Fig. 3 is labeled "identifier," because its output is exactly the
296
HSI-HAN YEH ET AL.
unit-sample response of the plant if U(z) r 0, which is easily satisfied by applying A(z) in advance of R(z).
A. REALIZATION OF THE CONTROLLER
Note that an additional delay is assigned to the controller, in order to accommodate the computation time needed for the deconvolution.
The
controller may be realized as in Fig. 2, with the additional delay connected to the output. Since the unit-sample response of G(z) has one delay, the first non zero value at the output of the identifier is available to the controller one sampling period after the application of A(z). Therefore, the furst non zero signal of R(z) must be three sampling periods behind that of A(z), in order for the computation time to be accommodated and the deconvolution to be synchronized (the first non zero signals of~(z) and R(z) arrive at the controller at the same time). This can be verified as follows: (Assume A(z) has no delay).
R(z)_ r3 z-3 + r4z'4 +
...
Cr(z) - g l z" 1 + ~ 2 z - 2 + ...
y0 = y~ = 0
r3
Y2 = 7"gl
= Y(z)
(10)
(11)
(12)
Where Y(z) is the controller output. It is easily seen that if rl * 0, then at t=0 one must have
rl
y o = : -gl
(13)
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
But gl is not available until t-T.
297
Therefore, the controller is not
realizable. If r 0 = r I - 0, one must have, at t=T r2
Yl =7-gl
(14)
This means r2 must arrive the controller at t=T, and the computation time is not accommodated. It should be noted that, since the computation time is small compared to the sampling period, it can be accommodated by staggering the input and output sampling of the block G(z), thus saving one delay. However, this is not the issue of this chapter.
B. CONSTRAINTS ON THE UNKNOWN PLANT
The implementability of the block diagram of Fig. 4 imposes two major constraints on the unknown plant: The plant must (1) be of minimum phase and (2) have a convergent unit-sample response sequence. The minimum phase requirement arises as the controller transfer function is the inverse of the estimated plant transfer function.
Any non
minimum phase zero of the plant therefore is an unstable pole of controller. Since at this stage of the control scheme the control process is open-loop, a non minimum phase system is a major obstacle. This is common to other existing model-reference and self-tuning adaptive control systems. As non minimum phase effects are also major contributors to the low performance of optimal control systems and modem robust designs, the minimum phase requirement for the unknown plant should not be regarded as a restriction unique :to the technique presented here. It is a restriction to design procedure, however,
298
HSI-HAN YEH ET AL.
The requirement of a convergent unit-sample response sequence arises because the controller is implemented from the output sequence
of the
identifier. A finite memory controller can be implemented only if the output sequence, which is the unit-impulse response sequence of the plant, is convergent. This type of plant includes those that are input-output stable, or have at worst a pole at the origin of the s-plane (a "type 1" system).
C. THE STARTER SIGNAL
A system involving a deconvolufion such as the one in Fig. 4 must be started by applying the signal A(z) first. (Otherwise, the two signals involved in deconvolution wait indefinitely for one another to start.) Therefore A(z) may be called the "starter signal." Equation (9) implies that -A(z) may be regarded as a linear operator which operates on the unit-sample response of a transfer function G(z) to generate the error signals of the adaptive control system. Since the plant is unknown, selection of A(z) can only be made on the basis of the general properties of the plant, such as the convergence and smoothness of the unitsample response. The magnitude of the excitation signal A(z) may be made small to give a small error sequence. However, the practical implementability of
-l/A(z) imposes a lower limit to the magnitude of the time signal represented by A(z). One possible choice of A(z) is
A(z) =-Uo(1 - z"l)
(15)
This starter signal has non zero values at t=0 and t=T only. This A(z) being a differentiator, gives
E(z) - Uo(1 - z"l)G(z)
(16)
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
en = uo(gn - gn-l)
=
uoAgn
299
(17)
The error signal is the first order difference of the unit-sample response sequence of the unknown plant. The error sequence vanishes as n ~ 0- whether the controlled system is type one or type zero.
The higher the sampling
frequency, the smaller the output error. The error is also proportional to uo, which can be made as small as realistically possible. One can also choose A(z) to be a double differentiator. The choice of a starter signal that will make the system error small and quickly vanishing is practically unlimited.
D. UPDATING THE CONTROLLER PARAMETERS
After all the memory slots in the controller are filled (n > N + 1, the output of the identifier is simply ignored. The system then operates open-loop and the performance as measured by the magnitude of the error signal begins to deteriorate even if the parameters of the plant remain unchanged.
This is
because a growing number of the parameters in G(z) are being approximated by gN- An update scheme is needed for readjusting the values of~l, g2.... gN after the memory storage is filled so that the finite memory controller with the adjusted fgi's may best keep the output error small. The update scheme shown in Fig. 5 is devised heuristically. It is based upon the following reasons: 1. Increasing each term in the convolution sum between ui and ~ in proportion to the output error decreases the output error. 2. In each feedback loop, there must be at least a time delay to account for the computation time. A salient feature of this scheme is that the complicated controller structure of Fig. 5 can be reduced to a simple adjustable-gain controller of Fig. 6, as shown in the following derivation.
300
HSI-HAN YEH ET AL.
Vn+l
n-N+3
1
Un-N+3 Un-1
En T
Cn~
En
c(t)
Unknown System - U 0 5(n)
Fig. 5 Updating Scheme
U 0 5(n)
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
301
Relating the variables at the adders and subtractors of the diagram of Fig. 5 gives, for 2 < i < N - 1 and n > N
~ i ( n - 1) Un.i+ 1 " gi(n)
~ N ( n - 1)
Un-i+2 = b e n
(18)
fgN(n) Vn = ben
(19)
~'i(n)un+2"i + ~N(n)Vn
(20)
Vn. 1-
N-I rn+2= X 1=1
where 13 and b are design parameters and, ~l(n), ~2(N) ..... ~N(n) now are functions of n as they undergo updating at each sampling instant after the memory storages are filled. Simple algebraic manipulation of these equations give
13r.+l - rn+2 = b(N-1) en + ~l(n-1)Un - ~l(n)un+l
(21)
In view of this equation, it is readily seen that the input-output relation of the system of Fig. 5 is completely determined by the following set of equations:
Cn = Un * gn = Z
giun-i
(22)
i=l
en = rn - ca
en
=
~En.l + b(N-1)% - I]rn+l + rn+2
(23)
(24)
302
HSI-HAN YEH ET AL.
El(n)
= E--an
(26)
Un
~n
(27)
Un+l = ~,l(n)
g
Fig. 6 Reduced Block Diagram of Updating Scheme
These equations are represented by the reduced block diagram of Fig. 6. The stability and convergence of this scheme have not been analyzed. Preliminary simulation results given at the end of this chapter show that the Finite memory adaptive controller with the update scheme performs well on the example.
IV. MODEL REFERENCE CONTROL Fig. 7 shows the configuration of a model reference adaptive control system, in which the unknown plant is denoted by G(z), the model is given by
CONTROLOF UNKNOWNSYSTEMSVIADECONVOLUTION
A.(z) %(z) _
~
303
[ a(z) ~
z
)
Unknown[ H(z) 1-.,, System /
I-" Fig. 7 Model Reference Control Gm(z). The adaptive controller is to be designed so that the plant output follows the model output as closely as possible. The model error Em(z) is given by GeG G Em = 1 + GcGH R + 1 + GcGH A - GmR
(28)
where the z-transform variable is suppressed for brevity's sake. Assume that a Ge is implemented so that the system output would be identical to the model output if A(z) were absent, i.e., GcG = Gm 1 + GcGH
(29)
Then G
Gm
Em=I+GcGH A=~c
A
This means that if Gc can be implemented as a deconvolution given by
(30)
304
HSI-HAN YEH ET AL.
AGm G c - Em
(31)
Then G Em = 1 + GeGH A = (1 - GmH)GA
(32)
I ,z, I zl R(z)
E(~A(z)C~(2) _ z2~(zJ i~(z) ~
A(z) .C(z_ Em(Z) ~(~z)G(z)
i
Fig. 8 Model Reference Control via Deconvolution
When the unknown plant has a convergent unit-sample response, a starter signal A(z) may be chosen so that the model error E m is small and quickly vanishing. The block diagram of the model reference control via deconvolution is given in Fig. 8.
A. REALIZATION OF THE CONTROLLER
The controller in Fig. 8 is drawn on the asstanption that the plant G(z) has one delay, so that the block z -l/Era(z) may be realized as in Fig. 2 with the first non zero signal in Em(z ) arriving at the controller at the end of the first sampling period. The extra delay at the output of the controller is introduced to
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
305
account for the computational delay required for the digital implementation. As in the self-adaptive control case, this delay may be avoided by using staggered sampling. It is important to note that a correct number of delays must be incorporated into the model Gin(z) in order for the deconvolution controller to be realizable. Assume that the feedback has no delay, and the starter signal A(z) and the reference input R(z) are applied without delay. Then the controller output (before the computation delay) is given by (see Fig, 8) Y(z) =
E(z)A(z)Gm(z)z Em(z)
-
X(z)z -1 Em(z)
I (33)
where E(z) and A(z) have no delay, Era(z) has one delay and the number of delays needed in Gm(z) is to be determined. If Gm(z) has two delays, i.e.,
Gm(z) = gm(2)z "2 + gm(3)z"3 + ...
(34)
X ( z ) -- x 0 4- x 1z- 1 4- ...
(35)
with gm(2) ;~ 0, then
for some x0 ;~ 0. Substituting Eq. (35) and the Taylor series expansion of Em(z) into Eq. (33) gives
Y(z) =
(X o + Xl z ' l + ...)Z -1
1 emlZ" + em2z2 + ...
Therefore at t = 0, the initial signal of Y(z) must be
(36)
306
HSI-HAN YEH ET AL.
Yo= xo
(37)
eml
But eml is the time signal of Era(z) at the end of the first sampling period, i.e., at t=T. Therefore the transfer function from E(z) to Y(z) is anticipatory. It is easily verified that the controller is causal only if the model Gin(z) has three or more delays. If the computation time is accommodate~ by staggered sampling, then one less delay is required for Gin(z). As in the self-adaptive scheme, a starter signal is necessary for the deconvolution controller to operate. The plant also must have minimum phase and have a convergent unit-sample response. After all the memory slots in the controller are filled, the model error is ignored but the system still operates closed-loop. It is possible to adopt an update scheme similar to the self-adaptive case but that has not been attempted in this exploratory study. The following section presents numerical examples for the control schemes developed in this chapter.
V. NUMERICAL EXAMPLES
This section presents the simulation results of applying the adaptive control algorithms developed in this chapter to a generic aircraft model. The plant model is a two-degree-of-freedom short-period approximation to the longitudinal dynamics of a delta-winged configuration. At Mach 0.9 sea level flight condition, the longitudinal dynamics are given by
i (t) =
[ -2.544 1004.4 I x ( t ) + 1"'710"39] -.01545 -3.3270 [-43.534 u(t)
CONTROL OF UNKNOWNSYSTEMSVIA DECONVOLUTION
307
c(0 = [0 1]xCt)
(39)
where c(0 is the pitch rate and u(t) is the control surface deflection. The transfer function of the above state equation is CCs) -4.3543(s + 2.2919) U ( s ) - s 2 + 5.8715s + 23.982
.
.
.
.
.
.
.
.
.
.
.
.
.
X
-!
-
X
-2
o - input x - - output
-3
-4
X
-5
-E;
~ , . ,
I, I ,,, ,
.5
I+ , , , , ,
I
, l.., , , ,
1.5
. , I ,I
2
, , ,,
, I , , ,, ,
2.5
,
.
3
TIME
Fig. 9 Open-Loop Response to Reference Input
3.5
.
308
HSI-HAN YEH ET AL.
The system is descretized with a sampling period of T=.04 second and a zeroorder hold. Fig. 9 shows the response of the uncompensated plant to an input which rises exponentially to a normalized final value after an initial two-period delay, with a time constant equal to 0.28854.
A. SELF-ADAPTIVE CONTROL 1.2
.9 .S .3
xO
x o 0 ( 0
R @
E
-.3
-.6 -.9
o-x u
Input output
1.5
Z
-1.2
-1.s -1.8 95
1
2.5
TIME Fig. 10 Adaptive Control Response
3. S
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
309
Fig. 10 shows the response of the adaptive system under the same input. The number of memory locations used in the adaptive controller is N=50. The forgetting factor is arbitrarily chosen at f= 1. The starter signal is chosen as A(z) = u0(1-z-l), with u0 = 1. In the first two seconds of operation, the adaptive controller operates in the identification-and-control phase. (The controller in this phase is given by Fig. 4.) At t=2 seconds, the controller is switched over to the update phase which is represented by Fig. 5 with the design parameters chosen to be b = 1/N and [3 = 1. (The simulation is run using Fig. 6, which is equivalent to Fig. 5.) Significant improvement in the output performance due to the adaptive controller is observed by comparing Fig. 10 with Fig. 9. Also, the smooth transition at the switch-over at t=2 as indicated by the smoothness of the output curve in Fig. 10 is worth noting. Since the starter signal A(z) is chosen to be u0(1 - zl), the error signal is given by -uoAgn (See Eq. (17)). In this example, u0 is arbitrarily selected at u0 -1. A much smaller value could have been used. The lower bound of u0 is only limited by the numerical accuracy of the controller implementation (a gain of -l/uo is implemented in the error processor, or identifier, of Fig. 4). Thus, the error signal may be made arbitrarily small as long as the numerical error generated by computations involving u0 and l/u0 is negligible. Note that at t=T, the error signal is exceptionally large (1.62). It is the response of the plant to the starter signal that is arbitrarily chosen. It can be reduced by selecting a smaller U0.
The performance of the self-adaptive control on a time-varying plant has also been simulated. The damping ratio of the plant of Eq. (40) is assumed to vary at a rate of 0.075 r/sec, where r is a random number between 0 and 1. (If r= 1, the damping ratio of the plant would vary from 0.6 to 0.3 in 4 seconds.) Simulation results show that if the plaiat variation starts after the 38th sampling instant (1.5 sec after the starter signal), the adaptive control output follows the
310
H S I - H A N YEH ET AL.
reference trajectory closely with negligible error. The output plot resembles that of Fig. 10 and therefore is not shown.
B. MODEL REFERENCE CONTROL
A simulation run is made for the model reference control system of Fig. 8 using the plant of Eq. (40) for G(s). The system again is discretized with a sampling period of T = 0.04 second and a zero order hold. The model is chosen tobe
Gm(z ) _ (I- ~)z, 3 -
1
~ Z -l
(41)
Its response to a unit-step input is an exponential saturation with three delays, given by Cm(n) = (1 - otn-2) and Cm(0) = Cm(1) = Cm(2) = 0. The starter signal is chosen to be
A(z) = u0 (1
- o~z " l )
(42)
so that (see Fig. 8)
A(z)Gm(z)z 2 = (1 - tz)u0z-I
(43)
The feedback compensator is arbitrarily taken to be H(z) = 1. Fig. 11 shows the model output and the plant output of this system, when the reference input is a unit step, u0 = 0.1 and ot = 0.87055. Fifty memory slots (N=50) are assigned to the convolution controller. The forgetting factor f is arbitrarily chosen to be f--0.5. The numerical result verifies that the model reference control via deconvolution performs well in controlling an unknown plant, as anticipated in the theory.
CONTROL OF UNKNOWN
SYSTEMS VIA DECONVOLUTION
311
- . . . . . . . . . . :- - - - - - ~ - i~ 9- - - ...... ~ _ ~ _ _,_ _ i ~ . . . . . . . . . . . _. . . . . . . . . . . . . . . . . . . . . . . . ~ --~ . . . . . . . . . . . . . . . . . . . ~ - _ ~- _ '
f
....
B
.8
.S
•
o x o
o~model x--system
output output
X 0
.4
X 0 X 0
.2
.it_
.
r -.2
f•
8
'''
i ' I'
.S
.
! I I ' ''
1
.
! ....
I.S
.
I,,
2
.
! s I,,,,
2. S
.
1,
3
,,
.
, I,,
,
3.S
4
TIME
Fig. 11 Response of Model Response Control
VI. COMPARISON WITH MODEL ALGOR/THMIC CONTROL (MAC)
MAC is one of the variants of the heuristic control schemes which f'mds applications in the aerospace industries [2-3, 6-7]. The MAC concept is based on the prediction of the system output using a unit-sample response model of the plant. The MAC system is designed by setting up an algorithm that computes a sequence of future control inputs that will steer the output along a given path.
312
HSI-HAN YEH ET AL.
Since the control sequence is computed in every sampling period, only the nextstep input is used. The deconvolution control shares with MAC the same design philosophy of treating the plant output as convolution of the input and the unitsample response. The main difference between MAC and deconvolution control is that MAC assumes that the unit-sample response of the plant is known, whereas the deconvolution control either identifies the unit-sample response of the plant on-line when it is needed, or generates the desired control sequence via deconvolution without requiring that the plant model be known. The comparison becomes more meaningful when one puts the modelreference deconvolution control of Fig. 8 in the perspective of MAC design. Substitution of Eq. (32) for Em in the system of Fig. 8 gives a single loop system with a controller Gc(z) equals to
Gm(z) Gc(z) = (-I"- Gm(z)H(z))G(zi
(44)
This compares with the unconstrained MAC with closed-loop linear prediction (See Fig. 1) which has a controller 1 - o~
Gc(z) = ~
1
x C,(z)
(45)
where G(z) is the finite unit-sample response model of the plant. Thus, it is seen that the deconvolution controller is able to implement the inverse of the exact plant model in the controller via on-line deconvolution, whereas the MAC system needs to have a unit-sample response model of the plant as a trade-off for the real-time deconvolution, the deconvolution control system must have a starter signal at the input of the plant, and the system model Gm(z) must have
CONTROL OF UNKNOWN SYSTEMS VIA DECONVOLUTION
313
sufficient number of delays. The output error of the deconvolution control system is contributed by the starter signal (Eq. (32)) which may be selected to make the error small and quickly vanishing. The output error of the MAC system is due to the error of the plant model G(z). Finally, it is noted that both the deconvolution control and the unconstrained, closed-loop linear MAC are subject to the same restrictions, namely, the plant must be minimum phase and have convergent unit-sample response.
VII. CONCLUSIONS
The model-based controller of the unconstrained model algorithmic control may be implemented by on-line deconvolution, without prior knowledge of the unit-sample response sequence of the plant. On-line deconvolution can also be used to generate control signals that will steer the output of an unknown plant to follow a reference trajectory as closely as the numerical accuracy of the instrumentation allows, provided that the unknown plant is stable and has minimum phase.
VIII. REFERENCES
1. J. Richalaet, A.Rault, J. L. Testud, and J. Papon, "Model Predictive Heuristic Control: Application to Industrial Process Control," Automatica, Vol. 14, 1978, p. 413-419. 2. R. K. Mehra, W. C. Kessel, A. Rault, J. Richalet, and J. Papon, "Model Algorithmic Control Using IDCOM for the F-100 Jet Engine Multivariable
314
HSI-HAN YEH ET AL.
Control Design Problem," in Alternatives for Linear Multivariable Control with
Turbofan Engine Theme Problem, edited by Sain, Peczhowski and Mesa, National Engineering Consortium, Inc. Chicago, 1978, pp. 317-350. 3. R. K. Mehra, R. Rouhani, and L. Praly, "New Theoretical Developments in Multivariable Predictive Algorithmic Control," Proceedings of the 1980 Joint
Automatic Control Conference, pp. 387-392. 4.
J. G. Reid, D. E. Chaffin, and J. T. Silvetthorn, "Output Predictive
Algorithmic Control:
Precision Tracking with Application to Terrain
Following," Journal of Guidance and Control, Vol. 4, No. 5, Sept-Oct 1981, pp. 502-509. 5. M. E. B ise, and J. G. Reid, "Further Application of Output Predictive Algorithmic Control to Terrain Following," Proceedings of the 1984 American
Control Conference, pp. 937-942. 6. W. E. Larimore, and S. Mahmood, "Basic Research on Adaptive Model Algorithmic Control," Technical Report, AFWAL-TR-85-3113, Flight Dynamics Laboratory, Wright-Patterson Air Force Base, Ohio, 1985. 7. J. V. Carroll, and R. F. Gendron, "Vectored Thrust Digital Flight Control For Crew Escape," Technical Report, AFWAL-TR-85-3116, Vols. I-II, Flight Dynamics Laboratory, Wright-Patterson Air Force Base, Ohio, 1985. 8. J. G. Reid, and S. Mahmood, "A State Space Analysis of the Stability and Robusmess Properties of Model Predictive Control," Proceedings of the 1986
American Control Conference, pp. 335-338. 9. J. J. Downs, E. F. Vogel, and V. J. Williams, "Tuning and Sensitivity Study for Model Predictive Control," Proceedings of the 1988 American Control
Conference, pp. 2357-2361. 10. O. A. Asbjorsen, "Stability and Robusmess of the DMC Algorithmic as Compared With the Traditional ARMA Model," Proceedings of the 1988
American Control Conference, pp. 278-283.
INDEX
B-polynomial methods, recursive time delay estimations, sampled-data control systems, 162-166 adaptive dead-time estimator, 164-165 basic extended B-polynomial algorithm, 162-163 other algorithms, 165-166 recursive least squares method, 163-164 simulation experiments, 186-189
Catalytic cracker, discrete, 4 Chemical plant, discrete model, 8-9 Controlled Markov chains, singularly perturbed, s e e Markov chains, singularly perturbed Correlation methods, recursive time delay estimations, sampled-data control systems, 167-171 delay estimation based on correlation analysis, 167-169 fixed model variable regression estimator, 170 other algorithms, 171 simulation experiments, 190-192
Deconvolution, control of unknown systems, 289-314 background, 289-291 conclusions, 313 model algorithmic control versus, 311-313
315
model reference control, 302-306 realization of controller, 304-306 numerical examples, 306--311 model reference control, 310-311 self-adaptive control, 308-310 real,time deconvolution, 291-293 self-adaptive control, 293-302 constraints on unknown plant, 297-298 realization of controller, 296-297 starter signal, 298-299 updating controller parameters, 299-302 Discrete-time systems randomly perturbed, robust stability, 89-120 background, 89-91 colored perturbations, 104-105 discussion, 114-116 jump Markov perturbations, 106-109 white and, 110-114 models and mathematical preliminaries, 91-95 white noise perturbations, 95-104 linear structured, 100-104 unstructured, 98-99 nonlinear, 95-98 stochastic parameters, observer design, 121-158 appendices Lemma 3, 150-151 Lemma 4, 152 Lemma 5, 152-153 discussion, 148-149 full order observation, 124-I 28 Lemma 1,131-133 Lemma 2, 133 Lemma 3, 135 appendix, 150-151
316
INDEX
Lemma 4, 135-136 appendix, 152 Lemma 5, 136 appendix, 152-153 mean square detectability, 130-133 mean square stability, 128-130 notation and mathematical preliminaries, 123-124 reduced order observation, 134-145 linear biased estimation arbitrary order, 137-139 with general observer, 139-141 linear unbiased estimation with general observer, 141-145 robust estimation, 145-148 steady state case, 133-134 Theorem 1,127-128 Theorem 2, 128-129 Theorem 3, 129-130 Theorem 4, 132-133 Theorem 5, 133-134 Theorem 6, 138-139 Theorem 7, 141 Theorem 8, 145-146 Theorem 9, 146-147 weakly coupled, stochastic, 1--40 Lemma 1, 28-31 Lemma 2, 31-32 recursive methods, 2-9 algebraic equation Lyapunov, 2-4 case study: discrete catalytic cracker, 4 Riccati, 4-9 case study: chemical plant model, 8-9 reduced-order controller, 9-18 background, 1-2 case study: distillation column, 17-18 Kalman filter, 13-14 new method, 19-36 filtering problem, 26-32 linear-quadratic control problem, 20-26 optimal, Gaussian, 32-36 linear-quadratic Gaussian optimal control problem, case study: distillation column, 35-36 Theorem t, 4 Theorem 2, 17 Distillation column control problem, 17-18, 35-36
Filtering, see also Kalman filtering discrete-time stochastic systems, 26-32; see also Discrete-time systems, weakly coupled
Jump Markov type parameters, s e e also Discrete-time systems, stochastic parameters, observer design randomly perturbed discrete-time systems, 106-114 K
Kalman filtering stability analysis, 207-236 appendices error representations of floating-point computation, 228-230 proof of Theorem 1,231-233 proof of Theorem 2, 233-235 background, 208-210 conclusions, 227-228 definitions, 214-215 Lemma 1,215-219 Lemma 2, 220 numerical example, 224-227 problem formulation, 210-214 Theorem 1,221-223 proof, 231-233 " Theorem 2, 223-224 proof, 233-235 stochastic systems, noisy and fuzzy measurements, 240-247; see also Stochastic systems, noisy and fuzzy measurements background, 240-241 compression of fuzziness, 245-247 fuzzy parameters, 242-245 review, 240-241 weakly coupled stochastic discrete-time systems, 13-14 L Least squares methods, recursive time delay estimations, sampled-data control systems, 163-164, 171-175
INDEX nonlinear time delay identification, 171-173 simulation experiments, 193-196 estimated model variable regression estimator, 194-196 nonlinear time delay identification, 193-194 Linear-quadratic control problem, weakly coupled discrete systems, 20-26, 32-36; s e e a l s o Discrete-time systems, weakly coupled Linear systems, stochastic, noisy input and output measurements, s e e Stochastic systems, linear, noisy input and output measurements Lyapunov algebraic equation, discrete, parallel algorithm, 2--4; s e e a l s o Discrete-time systems, weakly coupled case study: discrete catalytic cracker, 4
M
Markov chains, singularly perturbed, 257-287 background, 257-258 controlled, 269-284 decentralized algorithms, 274-279 limit Markov control problem, 280-284 aggregation'disaggregation algorithm, 283-284 linear programming algorithm, 283 MIMO, linear stochastic systems, noisy input and output measurements, 51-53 Model algorithmic control, 290 deconvolution and, 311-313 Model reference control, unknown systems, 302-306, 310 realization of controller, 304-306
Noisy input and output measurements, linear stochastic systems, s e e Stochastic systems, linear, noisy input and output measurements
Observer design, discrete-time stochastic parameter systems, s e e Discrete-time systems, stochastic parameters
317
Pad6 approximation, recursive time delay estimations, sampled-data control systems, 176-178, 178-181 simulation experiments, 196-197 Parameter systems, stochastic, discrete-time, s e e Discrete-time systems, stochastic parameters
Rational approximations recursive time delay estimations, sampleddata control systems, 175-182 basic concepts, 175-176 Padr-based delay identification algorithm continuous-time, 178-181 discrete-time, 176--178 simulation experiments, 196-197 sampled-data control systems, recursive time delay estimations, other algorithms, 182 Real-time deconvolution, 291-293 Recursive time delay estimations, sampled-data control systems, s e e Sampled-data control systems, recursive time delay estimations Reduced-order control, stochastic discrete-time weakly coupled large scale systems, s e e Discrete-time systems, weakly coupled Riccati algebraic equation, parallel algorithm, 4-9; s e e a l s o Discrete-time systems, weakly coupled case study: chemical plant model, 8-9 Robust stability, randomly perturbed discretetime systems, s e e Discrete-time systems, randomly perturbed, robust stability
Sampled-data control systems, recursive time delay estimations, 159-206 background, 159-160 conclusions, 201-202 correlation methods, 167-171 9delay estimation, correlation analysis, 167-169 fixed model variable regression estimator, 170 other algorithms, 171 extended B-polynomial methods, 162-166 adaptive dead-time estimator, 164--165 basic extended B-polynomial algorithm, 162-163
318
INDEX
other algorithms, 165-166 recursive least squares method, 163-164 model formulation, 160-161 nonlinear least squares methods, 171-175 estimated model variable regression estimator, 173-175 nonlinear time delay identification, 171-173 rational approximations approach, 175-182 basic concepts, 175-176 continuous-time Pad6-based delay identification algorithm, 178-181 discrete-time Pad6-based delay identifica-, tion algorithm, 176-178 other algorithms, 182 simulation experiments, 185-201 adaptive dead-time estimator, 189-190 algorithm, 197-201 basic extended B-polynomial algorithm, 186-189
correlation analysis, 190-192 correlation methods, 190-192 discrete-time PadE-based delay identification algorithm, 196-197 experimental set-up, 185-186 nonlinear least squares method, 193-196 estimated model variable regression estimator, 194-196 nonlinear time delay identification, 193-194 rational approximations approach, 196-197 variable regressor method, 197-201 variable regressor method, 182-184 simulation experiments, 197-201 time delay estimation algorithm, 182-184 Self-adaptive control, unknown systems, 293-302, 308-310 constraints, 297-298 realization of controller, 296-297 starter signal, 298-299 updating controller parameters, 299-302 Stochastic systems large scale, discrete-time, weakly coupled, s e e Discrete-time systems, weakly coupled linear, noisy input and output measurements, 41-88 appendix: Theorem 5, 85-88 background, 41-45 bispectral approaches, 53-59 consistency, 73-74
integrated, 66-69 linear estimator, 59-63 bispectrum and cross-bispectrum, 60-62 overdetermined, 63 nonlinear estimator, 64--65 conclusions, 80 consistency, 71-75 bispectral approaches, 73-74 integrated polyspectral approaches, 74-75 Lemma 1, 64 Lemma 2, 64 Lemma 3, 64 Lemma 4, 68 Lemma 5, 68-69 Lemma 6, 70 Lemma 7, 70-71 Lemma 8, 71-72 Lemma 9, 85-86 Lemma 10, 86 polyspectral techniques, 53-59 integrated, 65-71 bispectral, 66-69 auto- and cross-bispectrum, 66-67 linear estimator, 66 nonlinear estimator, 68-69 overdetermined linear system of equations, 67-68 consistency, 74-75 trispectral, 69-71 integrated auto- and cross-trispectrum, 69-70 nonlinear estimator, 70-71 simulation examples, 75-79 Theorem l, 50 Theorem 2, 51 Theorem 3, 53 Theorem 4, 73-74 Theorem 5, 74 proof, 86-88 Theorem 6, 74 Theorem 7, 74 Theorem 8, 74-75 Theorem 9, 75 Theorem 10, 75 time-domain technique, 45-53 asymptotic properties, 50-51 fourth cumulant criterion, 48-50 MIMO, 51-53 model assumptions, 45-48 noisy and fuzzy measurements, 237-256 background, 237-241
INDEX fuzzy numbers and fuzzy arithmetic, 238-240 Kalman filter, 240-241 discrete filtering, 247-255 conclusions, 251 example, 251-252, 253-255 Kalman filtering, 240-247 background, 240-241 compression of fuzziness, 245-247 fuzzy parameters, 242-245 review, 240-241
319
Time delay estimations, sampled-data control systems, s e e Sampled-data control systems, recursive time delay estimations W
Weakly coupled large scale systems, discretetime, stochastic, see Discrete-time systems, weakly coupled
This Page Intentionally Left Blank
