Rational Matrix Equations in Stochastic Control (Lecture Notes in Control and Information Sciences)

"

Remark 1.4.4 (i) By its definition, the second moment matrix P (t) is nonnegative definite for all t ≥ 0. But if we solve equation (1.16) backwards in time, then P (t) is not necessarily nonnegative definite for t ≤ 0. In fact (as we will show in Section 3.4), P (t) ≥ 0 for all t ≤ 0 is equivalent ˜ such that to the existence of a matrix A, AX + XA∗ +

N O

˜ ˜∗ Ai0 XAi∗ 0 = AX + X A

for all

X ∈ Kn×n .

i=1

In other words, P (t) ≥ 0 for all t ∈ R, if and only if P (t) is the second ˜ This reflects moment matrix associated to a deterministic system x˙ = Ax. the fact that stochastic differential equations in general cannot be solved backwards in time. (ii) If x(t) solves the inhomogeneous linear stochastic differential equation (1.12), then (e.g. [6]) the expectation m(t) = Ex(t) and the second moment P (t) = Ex(t)∗ x(t) satisfy the coupled deterministic differential equations m(t) ˙ = Am(t) + a(t) P˙ (t) = AP (t) + P (t)A∗ + a(t)m(t)∗ + m(t)a(t)∗ +

N O

i i ∗ i ∗ i∗ i i ∗ Ai0 P (t)Ai∗ 0 + A0 m(t)a0 (t) + a0 (t)m(t) A0 + a0 (t)a0 (t) .

i=1

We will need this formula in the numerical examples in Section 5.5. (iii) There is a general formula to transform a Stratonovich differential equation to an Itˆ o equation and vice versa, such that the set of solutions is unchanged. We present this formula here for the linear case. If we interpret (1.12) as a Stratonovich equation, then the equivalent Itˆ o equation E is given by [80] > N D 1 O i= i A0 A0 x + ai0 (t) dx = Ax + a(t) + dt 2 i=1 +

N = O

D Ai0 x + ai0 (t) dwi .

(1.17)

i=1

1.5 Stability concepts One of the central notions in the theory of dynamical systems is stability. An equilibrium is called stable, if for every ball B1 containing the equilibrium

1.5 Stability concepts

11

point, one can find a second, possibly smaller ball B2 ⊆ B1 , such that all solutions starting at time t = 0 in B2 stay in B1 for all t > 0; if additionally the solutions even converge to the equilibrium, then the equilibrium is called asymptotically stable. In the context of stochastic differential equations special care has to be taken in the definition of stability, since the solutions do not only depend on time and the initial conditions, but also on the random parameter ω ∈ Ω. If we think of the solutions as families of paths, parametrized by ω, then a natural definition of stability requires almost all of these paths to be stable in the usual sense. This leads to the concept of stochastic stability (also stability in probability, almost sure stability). If, however, we regard solutions as families of random variables x(t) ∈ L2 (Ω, Kk ), then we rather prefer the norms E<x(t)<2Kn = <x(t) 0 the p-th mean E<x(t)
N O

(i)

f0 (t, x) dwi ,

i=1

and let the conditions of Theorem 1.2.1 hold for all T > 0. Moreover we (i) assume that f (t, 0) = f0 (t, 0) = 0 for all t ≥ 0 and all i = 1, . . . , N . Thus the solution x(·; 0) ≡ 0 is an equilibrium state, and for arbitrary x0 ∈ Kn the corresponding solution x(t; x0 ) exists for all t > 0. To simplify matters, we speak of stability of the equation rather than of the equilibrium solution. Standard references for this topic are [126, 130, 143]. Definition 1.5.1 (Stochastic stability) (i)

The equation (1.8) is called stochastically stable, if there exists a δ > 0 such that for all x0 ∈ Kn with <x0 < < δ it holds that @ G ∀P > 0 : lim µ sup <x(t; x0 )< ≥ P = 0 ; (1.18) x0 →0

0≤t<∞

it is called asymptotically stochastically stable if in addition = D lim µ lim <x(t; x0 )< = 0 = 1 . x0 →0

t→∞

(1.19)

12

1 Aspects of stochastic control theory

(ii) The equation (1.8) is called mean-square stable, if for all ε > 0 there exists a δ > 0, such that for each initial condition x0 ∈ L2 (Ω, Kn ) with <x0 0 we have <x(t; x0 ) 0 we have <x(t; x0 )
"

It follows now from Theorem 1.4.3 that system (1.13) is asymptotically meansquare stable, if and only if the deterministic linear differential equation (1.16) is asymptotically stable. The right-hand side of this equation is given by the linear mapping P L→ AP + P A∗ +

N O

Ai0 P Ai∗ 0 .

i=1

Mappings of this type play a prominent role and will be studied in detail in Section 3.3. At this point we anticipate some simple facts. In particular, we will make use of the conventions and notation introduced in the appendix.

1.5 Stability concepts

13

Let Hn ⊂ Kn×n denote the space of Hermitian n × n matrices endowed with the scalar product 1X, Y 2 = trace XY . Then the mappings LA (X) = A∗ X + XA

and

ΠA0 (X) =

N O

i Ai∗ 0 XA0 ,

(1.20)

i=1

define endomorphisms of Hn . We call LA a Lyapunov operator. The operator ΠA0 is positive, which means that ΠA0 (X) is nonnegative definite, if X is ∗ are given by nonnegative definite. The adjoint operators L∗A and ΠA 0 L∗A (X)

∗

= AX + XA

and

∗ ΠA (X) 0

=

N O

Ai0 XAi∗ 0 .

(1.21)

i=1 2

Sometimes it is useful to write matrices X ∈ Kn×n as vectors in Kn and operators T : Kn×n → Kn×n as (n2 × n2 )-matrices. This is achieved by the well-known vec-operator and the Kronecker product (see [113], or Definition ∗ correspond to the 3.3.1). For the moment, we just note that L∗A and ΠA 0 matrices (where the bar denotes complex conjugation) A¯ ⊗ I + I ⊗ A

and

N O

A¯i0 ⊗ Ai0 ,

(1.22)

i=1

The following theorem gives equivalent criteria for (1.13) to be asymptotically mean-square stable. It constitutes the basis for the analysis of stability and stabilizability in the sequel. The result can be found e.g. in [132, 130, 176]. Further equivalent conditions will be given in Theorem 3.6.1. Theorem 1.5.3 The following are equivalent: (i) (ii) (iii) (iv) (v)

System (1.13) is asymptotically mean-square stable. System (1.13) is exponentially mean-square stable. σ (LA + ΠA0 ) ⊂ C− . ∃X > 0 : LA (X) + ΠA0 (X) < 0. ∀Y < 0 : ∃X > 0 : LA (X) + ΠA0 (X) = Y .

Proof: We first show the equivalence of (i), (ii), and (iii). As mentioned above, it follows from Theorem 1.4.3 that system (1.13) is asymptotically meansquare stable, if and only if the deterministic linear autonomous differential equation ∗ )(X) X˙ = (L∗A + ΠA 0

(1.23)

is asymptotically – and thus exponentially – stable. This holds, if and only if ∗ σ(L∗A + ΠA ) = σ(LA + ΠA0 ) ⊂ C− , 0

14

1 Aspects of stochastic control theory

in which case (1.13) is also exponentially mean-square stable. Now we prove ‘(iv)⇒(ii)’: Let numbers c1 , c2 , c3 > 0 and a positive definite matrix X with c1 I ≤ X ≤ c2 I be given, such that LA (X) + ΠA0 (X) < −c3 I. If x(t) = x(t; x0 ) is an arbitrary solution of (1.13) then we have in analogy to the proof of Theorem 1.4.3 d(x(t)∗ Xx(t)) = (dx(t))∗ Xx(t) + x(t)X(dx(t)) + (dx(t))∗ X(dx(t)) = x(t)∗ (LA (X) + ΠA0 (X)) x(t) dt +

N O

? F i x(t)∗ Ai∗ 0 X + XA0 x(t) dwi .

i=1

In the integral form of this equation the last summand has zero expectation because Ex(t)∗ Xx(t) is bounded on any finite interval. Hence @< t G Ex(t)∗ Xx(t) − Ex∗0 Xx0 = E x(s)∗ (LA (X) + ΠA0 (X)) x(s) ds (1.24) 0

< t ≤ −c3 E x(s)∗ x(s) ds 0 < c3 t E (x(s)∗ Xx(s)) ds . ≤− c2 0 By Gronwall’s Lemma (e.g. [184]) we have E<x(t)<22 ≤

c3 1 ∗ c2 <x0 <22 − cc3 t 1 Ex(t)∗ Xx(t) ≤ Ex0 Xx0 e− c2 t ≤ e 2 , c1 c1 c1

which yields exponential mean-square stability. ∗ ), and It remains to prove ‘(ii)⇒(v)’. Since σ(LA + ΠA0 ) = σ(L∗A + ΠA 0 (ii) and (iii) are equivalent, it follows from (1.21) that the Itˆ o-equation P m dx(t) = A∗ x(t) dt + i=1 Ai∗ 0 x(t) dwi is exponentially stable. If Φ(t) denotes the fundamental solution of this equation then for arbitrary matrices Y > 0 there exists the integral < ∞ E (Φ(t)Y Φ(t)∗ ) dt = X > 0 . 0

Setting P (t) = E(Φ(t)Y Φ∗ (t)) we find (again like in the proof of Theorem 1.4.3) that N O d i Ai∗ P (t) = A∗ P (t) + P (t)A + 0 P (t)A0 dt i=1

= (LA + ΠA0 )(P (t)) . Thus it follows from the linearity of the integral

< (LA + ΠA0 )(X) = =

1.5 Stability concepts ∞

<0 ∞ 0

15

(LA + ΠA0 )(P (t)) dt d P (t) dt = −P (0) = Y , dt

which we wanted to show.

"

Corollary 1.5.4 Let α > 0. System (1.13) is exponentially mean-square stable with decay rate greater α if and only if there exists a matrix X > 0 such that LA (X) + ΠA0 (X) + 2αX < 0 . Proof: It is immediate to see from Itˆo’s formula that x(t) solves (1.13) if and only if eαt x(t) solves dx = (A + αI)x dt +

m O

Ai0 x dw .

(1.25)

i=1

Hence (1.13) is exponentially mean-square stable with decay rate greater α if and only if (1.25) is exponentially mean-square stable. By the previous theorem this is equivalent to the existence of X > 0 such that LA+αI (X) + ΠA0 (X) = 2αX + LA (X) + ΠA0 (X) < 0 , which we needed to show.

"

Remark 1.5.5 (i) There is some ambiguity in condition (iii) of Theorem 1.5.3 concerning the domain of the operator LA + ΠA0 . We have defined LA and ΠA0 as endomorphisms of Hn ⊂ Kn×n , which is consistent with regarding (1.23) as a differential equation on Hn . But from their definition (1.20) we have an immediate extension of LA and ΠA0 to the space Kn×n . The question arises, whether LA + ΠA0 as an endomorphism of Kn×n is stable. This is important, if we work with the Kronecker representation in (1.22). In general, if T : Kn×n → Kn×n is K-linear and T (Hn ) ⊂ Hn , then the restriction of T to Hn is an R-linear map T |Hn : Hn → Hn . For K = R the inclusion σ(T |Hn ) ⊂ C− does not imply σ(T ) ⊂ C− , as can be seen from the simple example X L→ −X T . In the case K = C, however, the spectra of T |Hn and T coincide (see e.g. [180]). On the other hand, if K = R we can still consider complex solutions of the stochastic differential equation (1.13). Since we can separate real and imaginary parts, it follows that the stability of real solutions implies

16

1 Aspects of stochastic control theory

the stability of complex solutions. It follows further that (1.16) defines a stable evolution on the space Hn of Hermitian matrices with complex entries. Hence LA + ΠA0 as an endomorphism of Cn×n is stable. (ii) Since the spectra of an operator and its adjoint are complex conjugate, we can equivalently state Theorem 1.5.3 with LA + ΠA0 replaced by its ∗ . In fact, we have already made use of this observation adjoint L∗A + ΠA 0 in the proof. If a matrix X satisfies condition (iv) of Theorem 1.5.3, then, in analogy to the deterministic case, we refer to the quadratic function φ : Kn → R defined by x L→ φ(x) = x∗ Xx as a Lyapunov function for system (1.13). For linear systems and the analysis of mean-square stability it is sufficient to consider quadratic Lyapunov functions. This parallels the situation in the deterministic case. If stochastic stability is considered, the situation is different. We cite the following result from [6]. For technical reasons we restrict ourselves to the real case. Theorem 1.5.6 Consider the general stochastic differential equation (1.8). Assume there exists a twice continuously differentiable function φ : Rn → R satisfying φ(0) = 0 and φ(x) > 0 for x O= 0, such that n

∀x ∈ R \{0} :

φ3x f (x)

+

n O

(i)

(i)

f0 (x)T φ33x f0 (x) ≤ 0 ,

(1.26)

i=1

where φ3x and φ33x denote the gradient and the Hessian of φ at x. Then the trivial solution of (1.8) is stochastically stable. If, moreover, the inequality (1.26) is strict, then the trivial solution of (1.8) is asymptotically stochastically stable. For the linear system (1.13) the quadratic function φ(x) = x∗ Xx obviously satisfies the assumptions of Theorem 1.5.6 if X satisfies condition (iv) in Theorem 1.5.3. Corollary 1.5.7 If system (1.13) is asymptotically mean-square stable, then it is asymptotically stochastically stable. The converse is not true, as the following example shows (cf. also [6]). Moreover, the example demonstrates that one can come to rather contradictory conclusions about the stability of a given system. Example 1.5.8 For different coefficient matrices A, A0 ∈ Rn×n we analyze stochastic and mean-square asymptotic stability of the linear stochastic Itˆo equations dx = Ax dt + A0 x dw and

(1.27)

1.5 Stability concepts

17

= 1 D dx = A + A20 x dt + A0 x dw . (1.28) 2 Note that (1.28) would be the equivalent Itˆ o equation, if we understood (1.27) in the Stratonovich sense. (i)

Stochastic stabilization of (1.27) by noise: Let n = 1 and A = a, A0 = a0 ∈ R. By Theorem 1.5.3 equation (1.27) is asymptotically mean-square stable, if and only if a + 12 a20 < 0. To analyze stochastic stability of (1.27) we apply Proposition 1.4.2 and write the fundamental solution Φ(t) explicitly as @= G 1 D a − a20 t + a0 w(t) . Φ(t) = exp 2 Since x(t; x0 ) = Φ(t)x0 it follows that (1.27) is stochastically stable, if = ε D =0, (1.29) ∀ε > 0 : lim µ sup Φ(t) > x0 →0 |x0 | t≥0 which is equivalent to µ(supt≥0 Φ(t) < ∞) = 1. Since Φ is almost surely continuous a sufficient condition for (1.29) is µ(limt→∞ Φ(t) = 0) = 1, which is equivalent to asymptotic stochastic stability. Hence (1.27) is asymptotically stochastically stable, if and only if D = 1 (1.30) µ lim (a − a20 )t + a0 w(t) = −∞ = 1 . t→∞ 2 By the strong law of large numbers (e.g. [6]) we have = D a0 w(t) µ lim =0 =1. t→∞ t

Therefore (1.30) holds true if and only if a − 12 a20 < 0. By the same arguments (1.28) is asymptotically mean square stable, if and only if a + a20 < 0 and asymptotically stochastically stable, if and only if a < 0. (ii) Stability of (1.28) independent noise: 0 6 0 1 Let n = 2 and A = aI, A0 = a0 with a, a0 ∈ R. −1 0 We have LA = 2aI and σ(ΠA0 ) = ±a20 . By Theorem 1.5.3 equation (1.27) is asymptotically mean-square stable, if and only if a + 12 a20 < 0. Since A20 = a20 I, the fundamental solution Φ(t) is (by Proposition 1.4.2) @= G 1 2D Φ(t) = exp a + a0 t + A0 w(t) . 2 The condition a + 12 a20 < 0 therefore is also necessary for stochastic asymptotic stability in this case. Similarly, (1.28) is both asymptotically mean square stable and asymptotically stochastically stable, if and only if a < 0.

18

1 Aspects of stochastic control theory

(iii) Stability of (1.27) and (1.28) independent of noise: 0 6 01 If n = 2, A = aI, and A0 = a0 with a, a0 ∈ R then LA = 2aI and 00 σ(ΠA0 ) = 0. Hence (1.27) is asymptotically mean-square stable if and only if a < 0. Since A20 = 0 this is also necessary for stochastic stability. The same holds for (1.28). (iv) Mean-square of 6 0 stabilization 6 by noise: 0 (1.28) 1 0 0 1 , A0 = a0 Let A = with a0 ∈ R. 0 −2 −1 0 Since A (and thus LA ) is unstable, (1.27) is not mean-square stable for any a0 . Equation (1.28), however, is asymptotically mean-square stable if and only if the matrix I ⊗ (A −

a0 a0 I) + (A − I) ⊗ I + A0 ⊗ A0 2  2  2 − a20 0 0 a20  0 −1 − a20 −a20  0  =  0  −a20 −1 − a20 0 0 0 −4 − a20 a20

is stable. The eigenvalue problem for this matrix decomposes in an obvious way, and all eigenvalues are negative, if and only if 0 6 2 − a20 a20 det = −8 + 2a20 > 0 , a20 −4 − a20 i.e. |a0 | > 2. Remark 1.5.9 The previous example is both surprising and alarming. Whether the noise term acts stabilizing or destabilizing depends essentially on the interpretation. For Itˆ o equations the noise term always tends to destroy mean-square stability whereas it can be advantageous for stochastic stability; for Stratonovich equations the noise term can even enforce both stochastic and mean-square stability (see e.g. [7]). While this effect of stochastic stabilization by noise is an interesting topic in the qualitative theory of dynamical systems, we conclude that stochastic stability is not an appropriate concept for robustness analysis. Similarly the effect of the noise term does not match with our intuition, if we consider the Stratonovich interpretation. We conclude that the Itˆ o interpretation of stochastic differential equations together with the concept of mean-square stability offers an acceptable model for robustness analysis of linear systems with random parameter vibrations. In the following we will sometimes use the abbreviated term stable instead of exponentially mean-square stable.

1.6 Mean-square stability and robust stability

19

1.6 Mean-square stability and robust stability It lies in the nature of uncertainty that one can never be sure about it. If we model parameter uncertainty as white noise, interpret the formal stochastic differential equation (1.1) in the Itˆ o sense and study mean-square stability, then we have already made three rather specific assumptions, which can only be satisfied in an idealized set-up. Hence we have to check, whether our stability results are in some sense robust with respect to these assumptions. We have already seen that the notion of mean-square stability is well fit for a worst-case analysis. Let us now discuss some aspects of parameter uncertainty. If we consider a linear differential equation x˙ = Ax , as a model for some physical system, then this nominal model describes the true dynamics of the system only approximately and locally. Three important sources of error are the following. Firstly it is often impossible to determine the values of the system parameters exactly; secondly nonlinear effects have been neglected; thirdly the physical system is subject to exogenous disturbances, which might lead to changes in its dynamics. In order to cope with these problems one can allow for parameter uncertainties in the nominal system. Corresponding to the different error sources, the nature of the uncertainties can be different. If a linear time-invariant model is sufficient, but the true values of some parameters cannot be determined exactly, then one can consider a whole class of systems x˙ = (A + A0 )x,

A0 ∈ A0 ⊂ Rn×n .

The set A0 represents all possible deviations of the true system parameters from those of the nominal system. If, however, we wish to cope with nonlinear or exogenous effects, it is, in general, not sufficient to model the parameter uncertainties independent of t and x. We are rather led to a model of the form E > N O i δi (t, x)A0 x , (1.31) x˙ = A + i=1

where the Ai0 represent the uncertain parameters and the δi are unknown scalar functions. Of course, in general we have to impose some restrictions on the functions δi ; on the one hand we have to ensure that the differential equation can be solved in some sense; on the other we can not allow for arbitrarily large δi in general, if we expect (1.31) to be stable. One possibility is to assume the δi to be measurable (deterministic) functions bounded in norm, i.e. max(t,x)∈R×Rn |δi (t, x)| ≤ di for some given di ≥ 0. Then a typical question is whether (1.31) is stable for all δi satisfying this constraint.

20

1 Aspects of stochastic control theory

Or one might – as we basically did in the previous sections – regard the δi as stochastic processes with certain statistical properties. In fact we modelled δi = w˙ i which is the formal derivative of the Wiener process, called white noise. This led us to the Itˆo-equation dx = Ax dt +

N O

σi Ai0 x dwi .

(1.32)

i=1

which allows for arbitrarily large deviations from the nominal system if they occur with sufficiently small probability. But on the other hand this uncertainty model requires that in the mean the nominal system is exact. We show now that if system (1.32) is exponentially mean-square stable with a given decay rate, then system (1.31) can tolerate uncertainties up to a certain bound. This was observed in [16], and we will extend the idea in Section 2.3.2 to the disturbance attenuation problem. The following simple binomial inequality is needed. n , V, W ∈ Kn×k and a, b ≥ 0 be arbitrary. Then Lemma 1.6.1 Let X ∈ H+

a2 V ∗ XV + b2 W ∗ XW ≥ c¯ V ∗ XW + c W ∗ XV

for all c ∈ C with |c| ≤ ab .

Proof: The assertion is obvious if ab = 0. Hence, assume b > 0. For X ≥ 0 and |c| ≤ ab we have G D∗ = c D @ =c |c|2 2 V +bW X V + b W + a − 2 V ∗ XV 0≤ b b b c V ∗ XW + c W ∗ XV ) , = a2 V ∗ XV + b2 W ∗ XW − (¯ which proves the assertion.

"

Proposition 1.6.2 Let system (1.32) be exponentially mean-square stable PN 2 with decay rate greater α > 0, and let 2α ≥ i=1 di for given numbers d1 , . . . , dN ≥ 0. Then (1.31) is asymptotically stable for arbitrary measurable functions δi : R × Rn → C satisfying |δi (t, x)| ≤ di σi for all t ∈ R. Proof: By Corollary 1.5.4, there exists an X > 0 such that 0 > A∗ X + XA + 2αX +

N O

i σi2 Ai∗ 0 XA0

i=1

≥ A∗ X + XA + > ≥

A+

N O

? 2 F i di X + σi2 Ai∗ 0 XA0

i=1 N O i=1

δi (t, x)Ai0

E∗

> X +X A+

N O i=1

E δi (t, x)Ai0

,

1.7 Stabilization of linear stochastic control systems

21

if |δi | ≤ di σi by Lemma 1.6.1. Hence, the quadratic real valued function x L→ x∗ Xx defines a uniform Lyapunov function for (1.31), provided |δi (t, x)| ≤ di σi for all t ∈ R and all i. " We conclude that mean-square stability in the presence of white noise parameter uncertainties guarantees a certain robustness margin with respect to bounded uncertainties. Remark 1.6.3 It is, however, an interesting and to our knowledge open question, whether the assumptions of Proposition 1.6.2 imply a certain robustness margin with respect to parameter uncertainties of e.g. coloured noise type. This question is closely related to the following. Assume that the equation dx = Ax dt + A0 x dw interpreted in the Itˆ o sense is mean-square stable with decay rate α. Can we then name conditions on the noise intensity σ, such that the equation dx = Ax dt + σA0 x dw interpreted in the Stratonovich sense is mean-square (or stochastically) stable? In view of formula (1.17) and Corollary 1.5.4 we might equivalently consider the conditions ∃X < 0 : and ∃X < 0 :

A∗ X + XA + 2X + A∗0 XA0 > 0 ,

(1.33)

@ @ G∗ G 1 1 A + σ 2 A20 X + X A + σ 2 A20 + σ 2 A∗0 XA0 > 0 . (1.34) 2 2

Clearly, for given A, A0 , condition (1.33) implies (1.34) for sufficiently small σ. But what parameters are needed to determine such a margin σ? We leave the question for further research.

1.7 Stabilization of linear stochastic control systems Let there be given a linear stochastic control systems of the general form dx(t) = Ax(t) dt +

N O i=1

Ai0 x(t) dwi (t) + Bu(t) dt +

N O

B0i u(t) dwi (t) . (1.35)

i=1

Here x(t) ∈ Kn is the state vector and u(t) ∈ Km is a control input vector at time t. For a given control u ∈ L2w ([0, ∞[; L2 (Ω, Km )) and an initial value

22

1 Aspects of stochastic control theory

x0 ∈ Kn there exists a solution of (1.35) which we denote by x(·, x0 , u). At a later stage we also consider a disturbance input v(t) ∈ KL and an output vector z(t) ∈ Kq . In this case we write dx(t) = Ax(t) dt +

N O

Ai0 x(t) dwi (t)

i=1

+ B1 v(t) dt +

N O

i v(t) dwi (t) B10

(1.36)

i=1

+ B2 u(t) dt +

N O

i B20 u(t) dwi (t)

i=1

z(t) = Cx(t) + D1 v(t) + D2 u(t) , and use the notation x(·, x0 , v, u) for the solution of (1.36) with a given disturbance-input v ∈ L2w ([0, ∞[; L2 (Ω, KL )), a given control-input u ∈ L2w ([0, ∞[; L2 (Ω, Km )), and an initial value x0 ∈ Kn . Similarly z(·, x0 , v, u) denotes the output. 1.7.1 Stabilizability One of our tasks is to find a feedback control law of the form u(t) = F x(t) with a feedback gain matrix F ∈ Km×n such that system (1.35) is stabilized. This is a straight-forward generalization of the deterministic stabilization problem by static state feedback. In this section we wish to point at some specific features of the stochastic set up. In particular, we introduce a generalized type of Riccati equations which constitutes the central topic of this book. First of all we define stabilizability. Definition 1.7.1 We call system (1.35) open loop stabilizable if for all x0 ∈ Kn there exists a control ux0 ∈ L2w ([0, ∞[; L2 (Ω, Km )) such that x(·, x0 , ux0 ) ∈ L2w ([0, ∞[; L2 (Ω, Kn )). It is called stabilizable by static linear state-feedback if there exists a matrix F ∈ Km×n such that the closed-loop system dx = (A + BF )x dt +

N O

(Ai0 + B0i F )x dwi

(1.37)

i=1

is mean-square stable. In this case, for brevity, we also say that the quadruple (A, (Ai0 ), B, (B0i )) or (if all B0i vanish) the triple (A, (Ai0 ), B) is stabilizable. In Section 2.1 we will see that in fact open loop stabilizability implies stabilizability by static linear state-feedback. The stability of (1.37) can be determined on the basis of Theorem 1.5.3. From this, we can easily derive a necessary criterion for stabilizability, which resembles the so-called Hautus-test [99] for deterministic systems (see also e.g. [135]).

1.7 Stabilization of linear stochastic control systems

23

Lemma 1.7.2 Let (A, (Ai0 ), B, (B0i )) be stabilizable. Assume that X is an eigenvector of the operator LA + ΠA0 , corresponding to an eigenvalue λ O∈ C− . Then B ∗ X O= 0, or B0i∗ X O= 0 for some i ∈ {1, . . . , n}. Proof: By assumption, there exists an F , such that the closed-loop system (1.37) is mean-square stable. Hence, by Theorem 1.5.3, the mapping T : X L→ (A + BF )∗ X + X(A + BF ) +

N O

(Ai0 + B0i F )∗ X(Ai0 + B0i F ) ,

i=0

is stable. Now assume B ∗ X = B0i∗ X = 0 for some eigenvector X of LA + ΠA0 and all i = 1, . . . , N . Then X is also an eigenvector of T corresponding to the same eigenvalue λ. Hence Re λ < 0. " A necessary and sufficient criterion for the triple (A, (Ai0 ), B) to be stabilizable can be stated as a Lyapunov-type linear matrix inequality. Criteria in terms of linear matrix inequalities involving two unknown matrices can be found in [21, 64]. Lemma 1.7.3 Let B ∈ Kn×m . Then (A, (Ai0 ), B) is stabilizable

⇐⇒

∗ )(X) − BB ∗ < 0 . ∃X > 0 : (L∗A + ΠA 0

Proof: ‘⇒’: By Definition 1.7.1, Theorem 1.5.3 and Remark 1.5.5(ii), the triple (A, (Ai0 ), B) is stabilizable, if and only if there exist F ∈ Km×n and ∗ ∗ X > 0, such that (L∗A+BF +ΠA )(X) < 0. This implies x∗ (L∗A +ΠA )(X)x < 0 0 0 ∗ n×n be a unitary matrix, for all nonzero x ∈ Ker B . Let U = [U1 , U2 ] ∈ K ∗ with the columns of U1 spanning Ker B ∗ . If we set Y = (L∗A + ΠA )(X), then 0 for arbitrary α > 0 0 6 = D αU1∗ Y U1 αU1∗ Y U2 ∗ ∗ U = )(αX) − BB U ∗ (L∗A + ΠA , 0 αU2∗ Y U1 αU2∗ Y U2 − U2∗ BB ∗ U2 where U1∗ Y U1 < 0 and U2∗ BB ∗ U2 > 0. Choosing α sufficiently small, we have D = −U2∗ BB ∗ U2 + α U2∗ Y U2 − U2∗ Y U1 (U1∗ Y U1 )−1 U1∗ Y U2 < 0 , ∗ such that U ∗ ((L∗A + ΠA )(αX) − BB ∗ )U < 0 by Lemma A.2. 0 ‘⇐’: If (LA + ΠA0 )(X) − BB ∗ < 0 for some X > 0, then we choose ∗ F = −B ∗ X −1 to obtain (L∗A+BF + ΠA )(X) < 0. " 0

If we think of deterministic systems, then, by the previous Lemma, the condition ∃X < 0 : L∗A (X) − BB ∗ < 0

(1.38)

obviously is equivalent to the pair (−A, B) being stabilizable. Sometimes this is expressed by saying that (A, B) is anti-stabilizable. In this case, there exists

24

1 Aspects of stochastic control theory

a feedback gain matrix F , such that all solutions of the system x˙ = (A+BF )x converge to 0 for t → −∞. Since stochastic differential equations cannot be solved backwards in time, the condition ∗ ∃X < 0 : ΠA (X) + L∗A (X) − BB ∗ < 0 0

(1.39)

does not have an analogous dynamic interpretation. Nevertheless, this condition, which we might call anti-stabilizability (and its dual version, see Remark 1.8.6) plays a role in algebraic solvability criteria for Riccati equations. Obviously, (1.38) implies (1.39). For completeness, let us recall the notions of controllability and observability. Definition 1.7.4 Let (A, B, C) ∈ Kn×n × Kn×m × Kp×n . The pair (A, B) is called controllable, if rk(B, AB, . . . , An−1 B) = n. The pair (A, C) is called observable, if (A∗ , C ∗ ) is controllable. It is well-known e.g. [135] that (±A, B) is stabilizable, if (A, B) is controllable. In particular, controllability of (A, B) implies (1.39). But controllability of (A, B) does not imply stabilizability of (A, (Ai0 ), B), as we will see in the following examples. Example 1.7.5 (i) The scalar case: Consider the scalar stochastic system dx = ax dt + bu dt + a0 x dw + b0 u dw . For f ∈ R the closed-loop system dx = (a + bf )x dt + (a0 + b0 f )x dw is stable if and only if 2(a + bf ) + (a0 + b0 f )2 < 0 (by Thm. 1.5.3). But such an f cannot exist if b0 O= 0 and the discriminant −2ab20 + 2ba0 b0 + b2 is negative. We might ask, whether it could be possible to find a non-real stabilizing f ∈ C for this system. The answer is negative, as we will learn from Lemma 1.7.7. If a real system is stabilizable by complex feedback, then it is stabilizable by real feedback as well. Thus, stabilizability in the presence of control-dependent noise is not a generic property over K. (ii) A non-stabilizable system: Now we consider a system with state-dependent noise only: dx = Ax dt + Bu dt + A0 x dw . One might ask, whether controllability of the pair (A, B), or the necto the system essary condition from Lemma 1.7.2 is sufficient for 0 6 0 be 6 11 0 stabilizable. This is not the case. Let us choose A = ,B = , 01 1

0

6

1.7 Stabilization of linear stochastic control systems

25

01 . The pair (A, B) is controllable, but the system is not stabi10 lizable. To see this, we consider the operator LA+BF + ΠA0 : H2 → H2 as in Thm. 1.5.3, with F = [f1 , f2 ] ∈ R1×2 . It has the Kronecker-product matrix representation       0001 1 0 1 0 1 1 0 0    f1 1 + f2 0  0  0 1   + 0 0 1 0 + 0 1 MF =      0 0 1 1 0 1 0 0 f1 0 1 + f2 0 0 f1 0 1 + f2 0 0 f1 1 + f2 1000   2 1 1 1   f1 2 + f2 1 1  . =  f1 1 2 + f2 1  1 f1 f1 2 + 2f2 A0 =

To simplify the problem, we consider the restriction of the operator MF 2×2 to the0three-dimensional H02 ⊂ R 6 . The symmetric matrices 6 6 0 subspace √ 00 10 01 U1 = constitute an (orthonormal) , U2 = 22 , U3 = 01 00 10 basis of this subspace. Hence, we apply the transformation   2 √0 0   ˆ F = U T MF U with U = [vec U1 , vec U2 , vec U3 ] = 1  0 √2 0  M 2 0 2 0 √   0 0 2 2 √2 √1 =  2f1 3√+ f2 2  . 1 2f1 2 + 2f2 ˆ F − I) = 2(f1 − f2 − 1)2 ≥ 0. This A short calculation yields det(M 2×1 ˆ F (and thus the operator shows that for all F ∈ R the matrix M LA+BF + ΠA0 ) has an eigenvalue λ with Re λ ≥ 1. Though we can move the eigenvalues of LA+BF as far as we wish into the left half plane, we cannot stabilize the operator LA+BF + ΠA0 by any F ∈ R2×1 . Again, it will follow from Lemma 1.7.7 below that the system is not stabilizable by any F ∈ C2×1 either. Nevertheless, the given system (A, A0 , B) satisfies the necessary stability criterion from Lemma 1.7.2. In fact, B ∗ X O= 0 for all eigenvectors of LA + ΠA0 . To see this, we consider MF with F = 0, which represents LA + ΠA0 . For X ∈ K2×2 we have B ∗ X = 0, if and only if vec X = [x1 , 0, x3 , 0]T , with x1 , x3 ∈ K. But the equation M0 vec X = λ vec X obviously requires x3 = 0 and thus also x1 = 0. Hence, X cannot be an eigenvector of LA + ΠA0 , if B ∗ X = 0. This is even stronger, than the condition from Lemma 1.7.2. It can already be guessed from these examples that the characterization of stochastic stabilizability is a difficult problem. To our knowledge there is no

26

1 Aspects of stochastic control theory

simple classification available in the literature. Some special cases have been discussed in [204, 205, 191]. In the following we reformulate the stabilizability problem with the help of rational matrix operators. 1.7.2 A Riccati type matrix equation Stabilizability by static state-feedback is equivalent to the solvability of a Riccati type matrix equation (e.g. [191]). Here we give an ad hoc derivation of this interrelation. Later, in Section 2.1, we will see the connection to the LQ optimal control problem and open loop stabilization. By Theorem 1.5.3 system (1.37) is stable (for a given F ) if and only if there exists an X > 0 satisfying Y ≥ (A + BF )∗ X + X(A + BF ) 6∗ 0 i∗ 60 6 N 0 i O I I A0 XAi0 Ai∗ 0 XB0 + B0∗ XAi0 B0i∗ XB0i F F

(1.40)

i=1

for some Y < 0. It follows from Theorem 1.5.3 that if (1.40) admits a positive definite solution for some Y < 0 then it has a solution X > 0 for arbitrary Y < 0. In particular, we choose 0 6∗ 0 6 0 6 I I P0 S0 Y =− , with some M = M > 0. (1.41) F F S0∗ Q0 (The role of the weight matrix M will become clear in Section 2.1.) Then we can write (1.40) as P (X) + F ∗ S(X)∗ + S(X)F + F ∗ Q(X)F ≤ 0 ,

(1.42)

where P , S and Q are given by ∗

P (X) = A X + XA +

N O

i Ai∗ 0 XA0 + P0 ,

i=1

S(X) = XB +

N O

i Ai∗ 0 XB0 + S0 ,

(1.43)

i=1

Q(X) =

N O

B0i∗ XB0i + Q0 .

i=1

Since Q0 > 0 and X > 0 we have Q(X) > 0. For fixed X we minimize the left hand side of (1.42) with respect to the ordering of Hn . Lemma 1.7.6 Let 0 < Q ∈ Hm and S ∈ Kn×m . Then

1.7 Stabilization of linear stochastic control systems

min

F ∈Km×n

F ∗ S ∗ + SF + F ∗ QF = −SQ−1 S ∗ ;

27

(1.44)

the minimum is attained for F = −Q−1 S ∗ . Proof: The assertion follows from the simple inequality F ∗ S ∗ + SF + F ∗ QF = (F ∗ + SQ−1 )Q(F + Q−1 S ∗ ) − SQ−1 S ∗ ≥ −SQ−1 S ∗ ; equality takes place if and only if F = −Q−1 S ∗ .

"

Hence we have the following result. Lemma 1.7.7 For an arbitrary matrix M > 0 as in (1.41) define P , S, and Q as in (1.43). System (1.35) is stabilizable by linear static state feedback if and only if the Riccati type matrix inequality R(X) = P (X) − S(X)Q(X)−1 S(X)∗ ≤ 0

(1.45)

has a solution X > 0. In this event F = −Q(X)−1 S(X)∗ is a stabilizing feedback gain matrix. Remark 1.7.8 (i) Under the conditions of Lemma 1.7.7, we know that the real-valued function φ(x) = E (x∗ Xx) is a Lyapunov function for system (1.37). For later use we write out the derivative of φ(x(t)) with respect to t, if x is a solution of (1.35) with control u. Like in (1.24) we have 6 O 0 6∗ >0 ∗ 6E 0 6 N 0 i∗ i A X + XA XB x x A0 XAi0 Ai∗ 0 XB0 ˙ + φ(x) = E . B0i∗ XAi0 B0i∗ XB0i 0 u u B∗X i=1

(ii) Let us once more consider the Stratonovich interpretation of (1.35). It is often argued that one can always transform a Stratonovich equation into an Itˆ o equation and apply all the techniques developed for Itˆ o equations. It should, however, be noted that in the case of control-dependent noise this transformation destroys the linear dependence of the closed-loop equation on F . For if we apply a linear static state feedback control to the Stratonovich equation (1.35), then we obtain the closed-loop Stratonovich equation (1.37). The corresponding closed-loop Itˆ o equation is therefore by (1.17) E > N N = D2 D O 1 O= i A0 + B0i F Ai0 + B0i F x dwi . x dt + dx = A + BF + 2 i=1 i=1 Obviously, the dependence on F is much more complicated now, and we cannot apply any standard Riccati techniques to establish the existence of a stabilizing F . This is another practical reason for us to prefer the Itˆ o interpretation.

28

1 Aspects of stochastic control theory

(iii) In the derivation of the Riccati inequality (1.45) we have exploited the properties of the operator LA + ΠA0 . There are other classes of linear systems, whose stability properties can be characterized by means of operators of the same form. • The discrete-time deterministic system xk+1 = Axk is asymptotically stable if and only if ∃X > 0 : A∗ XA − X < 0 (e.g. [184]). • The discrete-time stochastic system xk+1 = Axk + A0 xk wk with independent real random variables wk is mean-square stable, if and only if ∃X > 0 : A∗ XA − X + A∗0 XA0 < 0 (cf. [153]). • The deterministic delay system x(t) ˙ = Ax(t) + A1 x(t − h) is asymptotically stable for all delays h > 0 if ∃X > 0 : A∗ X + XA + X + A∗1 XA1 < 0 (cf. [129]).

1.8 Some remarks on stochastic detectability Once Theorem 1.5.3 is established, it is straightforward to formulate the problem of stabilization by feedback; the situation is similar, though technically more involved, as in the deterministic case. As for the filtering problem, the situation is more complicated. From the practical point of view, we do not have a duality of stabilizability and detectability for systems with multiplicative noise. To see the difference let us consider the deterministic system x˙ = Ax,

y = Cx

with measured output y and the dynamic observer ξ˙ = Aξ + K(y − η),

η = Cξ .

The observer is built as a copy of the system with input y, such that the error equation e˙ = (A − KC)e for e = x − ξ is stable. In this sense the detection problem for the matrix pair (A, C) is equivalent to the stabilization problem for (A∗ , C ∗ ). But if we consider a stochastic system dx = Ax dt + A0 x dw,

y = Cx ,

(1.46)

we cannot use an exact copy of the system in the observer equation, because of the parametric uncertainty. If we set e.g. dξ = Aξ dt + K(dy − dη),

η = Cξ ,

then we find the additive term A0 x dw in the error equation de = (A − KC)e dt + A0 x dw .

1.8 Some remarks on stochastic detectability

29

Hence we can only expect e to converge to zero if x is stable. This is due to the fact that we cannot reproduce the noise process in the observer. If we want to use the estimated state to stabilize the system, then we conclude that unlike in the deterministic case, we cannot separate the estimation problem from the stabilization problem. We have to construct a dynamic compensator directly instead of constructing a dynamic observer and a feedback regulator (see also [1] for uncertain systems). This approach was taken in [107]. It leads to a coupled pair of Riccati equations. While in the deterministic case one obtains one Riccati equation for the observer and one for the controller which can be solved separately, the coupling here reflects the fact that observer and controller cannot be separated. We will come back to this in Section 2.3.3. From the theoretical point of view, however, it can be useful to define detectability as a dual property of stabilizability (e.g. [58, 69]). To this end, we consider the stochastic system dx = Ax dt +

N O

Ai0 x dwi ,

dy = Cx dt +

i=1

N O

C0i x dwi ,

(1.47)

i=1

(C, C0i ∈ Kp×n ) together with the (unrealistic) observer equation dξ = Aξ dt +

N O

Ai0 ξ dwi + K(dy − dη),

dη = Cξ dt +

i=1

N O

C0i ξ dwi .

i=1

Definition 1.8.1 System (1.47) is called (mean-square) detectable if there exists an observer gain matrix K ∈ Kn×p , such that the error equation de = (A − KC)e dt +

N O (Ai0 − KC0i )e dw ,

(1.48)

i=1

is mean-square stable. For brevity, we also say that the quadruple (A, (Ai0 ), C, (C0i )) or (if all C0i vanish) the triple (A, (Ai0 ), C) is detectable. We say that the triple (A, (Ai0 ), C) is β-detectable, if CX O= 0 for any eigen∗ corresponding to an eigenvalue λ O∈ C− . vector X ≥ 0 of L∗A + ΠA 0 Remark 1.8.2 We choose the term β-detectable, since we need this property R to characterize the spectral bound β(T ) = max{Re(λ) R λ ∈ σ(T )} of an operator T (cf. Section 3.2.1). The notion of β-detectability has a simple interpretation. It says that the output of a β-detectable system can only vanish (in mean-square), if the state vector converges to zero (in mean-square). Lemma 1.8.3 Consider system (1.46). The triple (A, (Ai0 ), C) is β-detectable, if and only if for arbitrary x0 the implication

30

1 Aspects of stochastic control theory

∀t ≥ 0 : E
⇒

lim E<x(t, x0 )<2 = 0

t→∞

holds. The proof will be given in Section 3.6, when some useful results on resolvent positive operators are available. It has been noted in [69] that observability of the pair (A, C) and detectability of (A, A0 , C, C0 ) are not comparable. In fact, by duality, it follows immediately from Example 1.7.5(ii) that observability of (A, C) does not imply detectability of (1.47), even if all C0i vanish. The converse does not hold either, since in the deterministic case mean-square detectability (and also β-detectability) coincides with the usual concept of detectability. Moreover, we have the dual versions of the Lemmata 1.7.2 and 1.7.3. Lemma 1.8.4 If the triple (A, (Ai0 ), C) is detectable, then it is β-detectable. Lemma 1.8.5 Let C ∈ Kp×n . Then (A, (Ai0 ), C) is detectable

⇐⇒

∃X > 0 : (LA + ΠA0 )(X) − C ∗ C < 0 .

Remark 1.8.6 The previous lemma shows that detectability can also be understood as a property of the operator LA + ΠA0 and the matrix C ∗ C. This point of view will lead us to a more general notion of detectability in Definition 3.2.8. Naturally, we will call the pair (LA + ΠA0 , C ∗ C) detectable, if the condition in Lemma 1.8.5 holds. But, more generally, we will also call the pair (−LA − ΠA0 , C ∗ C) detectable, if ∃X > 0 :

(−LA − ΠA0 )(X) − C ∗ C < 0 .

(1.49)

This condition is dual to the anti-stabilizability condition (1.38). For the moment, let us call the triple (A, (Ai0 ), C) anti-detectable, if (1.49) holds. If all Ai0 vanish, then (1.49) is equivalent to the pair (−A, C) being detectable. For general Ai0 , however, we do not have a control-theoretic interpretation of the notion of anti-detectability. Nevertheless, the algebraic criterion (1.49) plays an important role later. It is immediate to see that the detectability of (−A, C) implies (1.49). In particular, condition (1.49) holds, if (A, C) is observable. Later, we will need a criterion for a closed-loop system to be observable or (β-)detectable. Lemma 1.8.7 For i = 1, . . . , N let A, Ai0 ∈ Kn×n , B, B0i ∈ Kn×m , and 0 6 P0 S0 n+m M = , with Q0 > 0 . ∈ H+ S0∗ Q0 ∗ m×n set Let F0 = −Q−1 0 S0 and for an arbitrary F ∈ K

1.9 Examples

AF = A + BF,

Ai0,F = Ai0 + B0i F,

0 and MF =

I F

6∗

0 M

I F

31

6 .

(i) If (AF0 , MF0 ) is observable, then (AF , MF ) is observable. (ii) If (AF0 , (Ai0,F0 ), MF0 ) is detectable, then (AF , (Ai0,F ), MF ) is detectable. (iii) If (AF0 , (Ai0,F0 ), MF0 ) is β-detectable, then (AF , (Ai0,F ), MF ) is β-detectable. Proof: Assume that MF X = 0 for some X ∈ Kn×n . By inequality (1.44) with Q = Q0 and S = S0 we have MF ≥ MF0 , MF0 X = 0 and (F − F0 )X = 0. Hence also AF X = AF0 X and Ai0,F X = Ai0,F0 X for i = 1, . . . , N . From this, (i), (ii), and (iii) follow immediately. "

1.9 Examples Let us now produce a number of examples for control systems with multiplicative noise. In doing so we pursue several goals. Firstly, we wish to illustrate the notions introduced so far and make the use of models with stochastic parameter uncertainties plausible. Secondly, we motivate control problems like the disturbance attenuation problem to be introduced in the following chapter. Thirdly, we will use some of the following models as numerical examples in Chapter 5. Finally, we think that a collection of examples is useful to everyone interested in our topic. 1.9.1 Population dynamics in a random environment Let x be the size of a biological population, e.g. fish in a pond or people on a planet. The simplest model for the growth of this population is the differential equation of normal reproduction x˙ = ax . All resource restrictions and problems of crowding neglected, the growth rate is a constant number a ∈ R. Clearly, a is an average number resulting from a magnitude of random events and environmental unpredictability. Taking this into account, it is natural to replace a by a Gaussian white noise process a + wa ˙ 0 (see e.g. [193], [134]). This brings us to the equation from Example 1.5.8 (i) dx = ax dt + a0 x dw . Similarly, it is natural to introduce random parameters in a system with more than one species like e.g. the famous Volterra-Lotka model. So, let us

32

1 Aspects of stochastic control theory

now consider a pond where two species of fish live, say carp and pike. If there were no pike, the carp would multiply exponentially. In the presence of pike one must take account of the number of carp eaten by the pike, which is jointly proportional to the number x of carp and the number y of pike. The pike, however, die out exponentially, if there are no carp, while in the presence of carp their number increases at a rate proportional to both x and y. Hence we obtain a deterministic model of the form (e.g. [8]) x˙ = kx − axy , y˙ = −ly + bxy .

(1.50)

Now let us regard the coefficients as Gaussian white noise processes. Then (see e.g. [134]) we have to replace (1.50) by a stochastic differential equation of the form dx = (kx − axy) dt + k0 x dw1 − a0 xy dw2 , dy = (−ly + bxy) dt − l0 y dw3 + b0 xy dw4 .

(1.51)

We just mention that the transition to a stochastic model after linearization can be problematic. For instance let us linearize equation (1.50) at the equilibrium (x, y) = (k/a, l/b) and set (∆x, ∆y) = (x − k/a, y − l/b). This yields the linear equation 6 60 0 6 0 d ∆x −k ∆x k − al b , (1.52) = ∆y −l + bk l dt ∆y a which describes the local dynamics around the equilibrium. But the stochastic equation (1.51) does not have any non-trivial equilibria. Hence it does not really make sense to consider stochastic coefficient in the linearized equation (1.52). 1.9.2 The inverted pendulum The inverted pendulum is one of the standard examples of linear control theory and has been described in a number of textbooks. It consists of an inverted pendulum attached to the top of a cart through a pivot. The cart is driven along a horizontal rail by a force βu(t), where the function u is regarded as the control input. The task is to stabilize the pendulum in the upper equilibrium position by choosing an appropriate control u. Let l be the length of the pendulum and assume that its mass m = 1 is concentrated at the tip. The angle between the pendulum and the vertical is denoted by φ. By M we denote the mass of the cart and by r the distance from the center of mass of the cart to some reference point. The state of the system is described by the state ˙ T , and the linearized equation of motion is given by (see vector x = [r, r, ˙ θ, θ] [184, 108]) x˙ = Ax + Bu

(1.53)

1.9 Examples

with



01 0 0 0 −mg M A= 0 0 0 0 0 m+M lM g

 0 0 , 1 0



0

33



 1  M  B=  0  . 1 − lM

Here, g denotes the acceleration due to gravity. All friction effects have been neglected. In this model it has been assumed that no external forces act on the system and that the reference coordinate system is at rest. But let us now assume that the whole structure is subject to random vertical vibrations which affect the laws of gravity in our system. In this case, it is customary (e.g. [20, 114, 124]) to model the constant g as time-varying and stochastic. In this spirit we regard g now as a Wiener process. Setting   00 0 0 0 0 −m 0 M  A0 =  0 0 0 0, 0 0 m+M lM 0 we model the randomly perturbed system by the Itˆ o equation: dx = Ax dt + A0 x dw + Bu dt . 1.9.3 A two-cart system In [194, 195, 163] a linear model of the two-mass spring system with uncertain stiffness depicted in Fig. 1.1 was considered. x1 k = k0 + ∆(t) v

m1 = 1

m2 = 1

 x1  x2   x=  x˙ 1  x˙ 2 

x2 u

1111111111111111111111111111111111111111 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 1111111111111111111111111111111111111111 Fig. 1.1. Two carts on a rail, connected by a spring

The system consists of two carts connected by a spring; there is a disturbance v acting on the left cart and a control force u driving the right cart. The control is to be chosen such that the displacements of the carts from their equilibrium position is minimized in some norm, i.e. the effects of v have to be annihilated. This is a typical disturbance attenuation problem. It is assumed that the spring constant k = k(t) has the nominal value

34

1 Aspects of stochastic control theory

k0 = 5/4, but it can vary (e.g. due to nonlinear effects of the spring) and is considered uncertain. In a series of experiments under different conditions it was found out that at each instant of time the values approximately have a Gaussian distribution centred at k0 . The experiments also showed that k ranges over [0.5, 2]. Therefore the difference ∆(t) = k(t)−k0 was modelled as a Gaussian white noise process with intensity σ = 1/4 ensuring that |∆(t)| < 0.75 and hence k ∈ [0.5, 2] with sufficiently high probability. As pointed out before, one might argue whether this model is adequate. It is possible that the Gaussian distribution of the values k was not produced by one individual spring at different instances of time, but by different springs, with time-invariant stiffness each. Or the stiffness of each spring is in fact time-varying but not a Gaussian process. In this case the concept of bounded uncertainties might be preferable. We will come back to this point later. The state-space equations of the stochastic spring system take the general form dx = (Ax + B1 v + B2 u) dt + A0 x dw , z = Cx + Du . 1.9.4 An automobile suspension system Models with uncertain stiffness and damping parameters also occur in more realistic applications than the one presented above. A simple model for an automobile suspension system with stochastic parameter uncertainty has been discussed in [89]. The nominal system, known as the quarter vehicle, is obtained by lumping all four wheels together and considering only the different vertical movements of the car body and the axle. The tyres are modelled by a spring and the suspension system is modelled by a spring and a damper. By y1 and y2 we denote the displacements of the centres of mass of the car body and the axle, respectively, from their equilibrium under the influence of gravity. If v(t) denotes the displacement of the point of contact between tyre and road from the nominal road level, then y1 and y2 are subject to the coupled second-order differential equations [108] m1 y¨1 + c(y˙ 1 − y˙ 2 ) + k1 (y1 − y2 ) = 0 , m2 y¨2 + c(y˙ 2 − y˙ 1 ) + k1 (y2 − y1 ) + k2 y2 = k2 v .

(1.54)

Here m1 and m2 denote the masses, c is the damping coefficient of the dashpot, k1 is the stiffness of the spring in the suspension system and k2 is the stiffness of the tyre, viewed as a spring. It is pointed out in [89], however that the behaviour of a tyre is much more complicated than that of a spring. In particular, the stiffness can vary as the tyre rotates and encounters changing road conditions. These variations are partly of a periodic nature and partly stochastic. While [159] discusses a periodic excitation of the parameter k2 , [89] considers a superposition of periodic excitation and white noise. To keep

1.9 Examples

y1

y2

35

m1

k1

c

m2

k2

Fig. 1.2. Suspension system of a quarter vehicle [89]

the model tractable within our framework, we restrict ourselves to white noise parameter uncertainty only and model k2 in the form k + σ w˙ with white noise w. ˙ The unevenness v of the road is viewed as an external disturbance input. Moreover we assume that we have an active damping system. We model this by setting the left hand side of equation (1.54) equal to u(t), where u is some control input. Thus, setting x = [y1 , y2 , y˙ 1 , y˙ 2 ]T we have the linear stochastic control system     0 0 1 0 0    0 0 0 1   x dt +  0  v dt dx =   −k1 /m1  0  k1 /m1 −c/m1 c/m1  k1 /m2 −(k1 + k)/m2 c/m2 −c/m2 k/m2       0 0 00 0 0 0   0   0 0 0    x dw +  0  v dw . (1.55) +   0   1/m1  u dt +  0 0 0 0 0 −σ/m2 0 0 σ/m2 0 If we define z = [y1 , y˙ 1 ]T as the to be controlled output, then a typical disturbance attenuation problem is the following. Find a feedback control u = F x that stabilizes the system and minimizes the effect of the disturbance input v on the output z.

36

1 Aspects of stochastic control theory

1.9.5 A car-steering model According to [1] the essential features of car-steering dynamics in a horizontal plane are described by the so-called single-track model. It is obtained by lumping the front wheels into one wheel Wf in the center line of the car; the same is done with the two rear wheels (Wr ). This is illustrated in figure 1.3. Here S is the center of gravity and v is the velocity vector. The angle β be-

v S

Wr

δf

β

δr

Wf 5r

5f

ψ x0

Fig. 1.3. Single-track model for car-steering [1]

tween v and the center line is called vehicle sideslip angle, whereas the angle ψ between the center line and some fixed direction x0 in the plane is called yaw angle. The yaw rate r = ψ˙ and the sideslip angle β are the state variables in our model. The front and rear steering angles δf and δr are the control variables (in the case of four-wheel steering). The distances between S and the front and the rear axles are denoted by ^f and ^r . Furthermore let m be the mass of the car, J the moment of inertia, and cr and cf given constants that depend e.g. on parameters like the tyre pressure and temperature. For the dynamics of this system the adhesion coefficient µ between road surface and tyre plays a crucial role. Under different conditions we have  (dry road) , 1 µ = 0.5 (wet road) ,  0.15 (ice) . We allow for different adhesion coefficients µf and µr at the front wheel and the rear wheel. If these values are fixed, we can describe the dynamics of the linearized single-track model in state space form as (1.56) x˙ = Aµ x + Bµ u . 0 0 6 6 0 6 0 6 a a b b δ β , u = f , Aµ = 11 12 , and Bµ = 11 12 with Here x = δr a21 a22 b21 b22 r 1 (cr µr + cf µf ) , mv 1 = (cr ^r µr − cf µf ^f ) − 1 , mv 2

a11 = − a12

1.9 Examples

a21 = a22 = b11 = b12 = b21 = b22 =

37

1 (cr ^r µr + cf ^f µf ) , J F 1 ? 2 − cr ^r µr + cf ^2f µf , Jv c f µf , mv c r µr , mv c f ^ f µf , J −cr ^r µr . J

As the car moves, the value of µ may change. For instance, when it starts raining on a dry road, then µ may vary rapidly between one point and another. The same may happen, if the road is wet and the temperature is close to the freezing point. In these situations it can be appropriate to model µf and µr as time-varying stochastic parameters. Let us assume that Eµf = Eµr = µ0 = 12 , and that µf − µ0 and µr − µ0 can be modelled by independent scaled standard Wiener processes σw1 (t) and σw2 (t), where σ denotes the noise intensity. Clearly, this model is only an approximation, since, for instance, it allows for negative adhesion coefficients. Nevertheless it can be useful to represent the time dependency of µ. Thus we are led to the linear stochastic control system D = (1) (2) dx = Ax dt + σ A0 x dw1 + A0 x dw2 = D (1) (2) + Bu dt + σ B0 u dw1 + B0 u dw2 , where

. A =

cr +cf cr Lr −cf Lf − 2mv 2mv22 cr Lr +cf L2f cr Lr +cf Lf − 2Jv 2J

−

and

. (1) A0

=

c

c L

1

4

0 ,

B =

.

4

cr 2mv r Lr − c2J

c r Lr mv 2 2 2 = , c r Lr c f Lf c f Lf c r Lr − Jv − Jv J J 0 cf 0 cr 6 0 0 mv (1) (2) B0 = cmv , B0 = f Lf 0 crJLr J 0

f f f − mv − mv 2

(2) A0

cr − mv

cf 2mv c f Lf 2J

6 ,

4 , 6 .

Automatic car-steering Based on (1.56) in [1] a model for automatic car-steering is discussed. The problem is the following. Our car has to follow a guiding wire in the street. This reference path is assumed to consist of circular arcs, with reference radius

38

1 Aspects of stochastic control theory

Rref and reference curvature ρref = R1ref . For straight path segments we have ρref = 0. The displacement z of the car from the reference path is defined to be the signed distance of S to the nearest point zref on the path; if |yS | is small, then zref is uniquely determined. To measure the displacement there is a sensor S0 mounted at a distance ^0 in front of S; hence the measured displacement y is the distance from S0 to the path. Finally we define ∆ to be the angle between the tangent to the path at zref and the center line of the car. These variables are illustrated in Figure 1.4. v

S

S0

z

y

β

∆ guiding wire

Fig. 1.4. Vehicle heading and displacement y from the guiding wire (here ρref = 0) [1]

 x With the extended state variable ξ =  ∆  ∈ R4 we have the extended y linearized state space model     0 6 0 Aµ 0 ˙ξ =  0 1 0 0  ξ −  v  ρref + Bµ u , 0 v ^0 v 0 0 / 5 y= 0001 ξ. 

Like above, we obtain a stochastic differential equation, if we replace Aµ and (2) (1) (1) (2) Bµ by A + A0 w˙ 1 + A0 w˙ 2 and B + B0 w˙ 1 + B0 w˙ 2 . For simplicity, we assume that the car has to follow a straight line such that ρref = 0. In this case the trivial solution is an equilibrium. The first problem now is to stabilize this equilibrium by choosing a static linear state feedback control law u = F ξ. Note that the uncontrolled system is not asymptotically stable. Furthermore, we take into account that the guiding line might be slightly distorted. That is, now we interpret ρref as a stochastic disturbance process. A typical disturbance attenuation problem now consists in choosing F such that the influence of ρref on the output y is minimized. We will study problems of this type in Section 2.3. They lead to Riccati equations as in 1.7.7 with an

1.9 Examples

39

indefinite weight matrix M . Of course, it would be of more practical value to have a dynamic compensator of the form ˆ = AK x dξ(t) ˆ(t) dt + BK y(t) dt , u(t) = CK x ˆ(t) + DK y(t) , which does not require the knowledge of the full state ξ. In the case of disturbance-dependent noise instead of control-dependent noise this problem has been studied e.g. in [107]. It leads to a coupled pair of Riccati equations. We will mention this point later. 1.9.6 Satellite dynamics A famous example of a stochastic control system was given by Sagirow. When modelling the motion of a satellite in the upper atmosphere, one has to account for the rapid fluctuations of the atmospheric density ρ and the intensity H of the magnetic field of the earth. In deterministic models based on Newtonian mechanics these quantities occur as constant parameters. Sagirow suggests to model ρ and H in the form ρ = ρ0 (1 + δ w) ˙ and H = H0 (1 + δ w) ˙ with white ˙ noise w. We consider the pitch and the yaw motion of a satellite on a circular orbit. It is assumed that the center of gravity of the satellite always stays in the plane of the orbit. Let x, y, z denote the principal axes of inertia of the satellite; a moving frame is given by the orbital tangent (ξ), the radius vector (η) and the normal to the orbital plane (ζ). Planar pitch motion Following [171] let us first consider the pitch motion of a satellite in the upper atmosphere. We are interested in the dynamics of the pitch angle θ and introduce ˙ With the above stochastic model for the the state variables x1 = θ, x2 = θ. atmospheric density one can derive (see [171], [6] [134]) a stochastic differential equation of the form G @ G @ G @ x2 0 x1 = dt + dw ; d x2 −bx2 − sin x1 − c sin x1 −abx2 − b sin x1 here x2 = x˙ 1 and a, b, c are given constants with b, c > 0. If we linearize this equation at (x1 , x2 ) = (0, 0) we obtain the equation @ G 0 0 6@ G 6@ G x1 0 0 0 1 x1 x1 d dw . dt + = x2 x2 x2 −a −ab 2c − 1 −b

40

1 Aspects of stochastic control theory

y

η θ ξ

x

Fig. 1.5. The plane pitch motion of a symmetrical satellite in a circular orbit [171] ζ

z

x

η=y ψ ξ

Fig. 1.6. The yaw motion of a satellite in a circular equatorial orbit [171]

Stabilization of the yaw motion by the magnetic field Now we consider the oscillations of the yaw angle ψ of a satellite in a circular equatorial orbit. Aerodynamic effects are neglected. The nominal intensity H of the magnetic field of the earth is orthogonal to the orbital plane. A magnetic bar is placed in the satellite-fixed z-axis (i) to produce a restoring torque My = −HIz sin ψ with the constant magnetic moment Iz . A second torque is due to the currents that arise when the satellite (ii) is moving in the magnetic field. It has the form My = −KH 2 ψ˙ with a constant K, depending on the thickness and the electromagnetic properties of the shell as well as the moments of inertia of the satellite. For constant H the yaw oscillations are thus described by B ψ¨ = −KH 2 ψ˙ − HIz sin ψ .

(1.57)

1.9 Examples

41

The quantities H and H 2 , however, depend on the activity of the sun. They can be modelled more correctly in the form H = H0 (1 + δ w) ˙ and H 2 = 2 H0 (1 + 2δ w). ˙ Thus (1.57) becomes a stochastic equation. For convenience we KH 2 H0 Iz ˙ Then the set α = B , β = B 0 and introduce the state variable ω = ψ. linearized stochastic model for the yaw oscillations is given by 60 6 60 6 0 0 6 0 0 ψ ψ 0 1 ψ dw . (1.58) dt + = d αδ −2βδ ω ω −α −β ω In [18] this model is extended by an active controller. It is assumed that some noise is associated with the activation of this controller. For instance, this will be the case if the controller changes the value of K. The controlled model has the form 0 6 60 6 0 0 6 60 6 0 6 0 0 ψ 0 0 ψ 0 1 ψ u dw . dw + u dt + dt + = d δu ω αδ −2βδ 1 ω −α −β ω 1.9.7 Further examples Without going into details, we mention some more examples. An electrical circuit In [195] an electric circuit consisting of a resistor, a capacitor, an inductor, and a voltage source in a serial connection is considered. The differential equations describing the capacitor voltage VC and the current i in this circuit are di 1 R = − i + (V − VC ) , dt L L 1 dVC = i, dt C

(1.59) (1.60)

where R, L and C as usual denote resistance, inductance and capacity. One can control the system by applying the voltage V . The authors claim that nearby electrical devices can induce changes in the inductance value L, such that L−1 should be modelled in the form L−1 = −1 ˙ This turns equation (1.59) into the stochastic system L−1 0 + L1 w. 6 0 R 6 0 1 6 0 60 − L0 − L10 i i L0 V dt = dt + d VC VC 0 − C1 0 6 0 1 6 60 0 R 1 − L1 − L1 i dw + L1 V dw . + VC 0 0 0 Similarly, e.g. in [173], the case of randomly perturbed capacity C has been discussed.

42

1 Aspects of stochastic control theory

An electrically suspended gyro In [79] we find a model for an electrically suspended gyro of the form @0 dx =

6 0 6 G 2 O 0 01 u dt + x+ σi x dwi . 1 β0 i=1

According to [79], one noise term comes from the electronic circuits and the other from thermal expansion and contraction of the gap between the rotor and the electrode cavity. The task is to stabilize the system by an appropriate choice of u. Mathematical finance Another very important area of applications for stochastic differential equations is mathematical finance. A multitude of books on this topic has appeared in recent years. Here we just make the annotation that Itˆ o equations are the central model in this theory. In a few words, the situation is the following. Suppose there is a market in which m + 1 assets are traded continuously. One of the assets is the bond, whose price process P0 (t) is subject to the deterministic differential equation dP0 (t) = r(t)P0 (t) dt,

P0 (0) = p0 > 0 .

(1.61)

The other m assets are stocks whose price processes P1 (t), . . . , Pm (t) satisfy the stochastic differential equations dPi (t) = bi (t)Pi (t) dt +

m O

σij (t) dwj (t),

Pj (0) = pj > 0 .

(1.62)

j=1

The stochastic uncertainty in this model is due to the independent and partly unpredictable trading of a large number of shareholders, who drive the market price of a stock like the molecules of liquid act on a pollen. In [218] one can find a linear quadratic control problem associated to optimal portfolio selection.

2 Optimal stabilization of linear stochastic systems

Having introduced and motivated our concepts of stochastic control systems in the previous chapter we now turn to optimal and suboptimal stabilization problems. A classical problem in optimal control theory is the so-called linear quadratic (LQ-)stabilization problem. Among all controls that stabilize a given system, one determines the one that satisfies a certain quadratic nonnegative semidefinite cost-functional. This will be the topic of the first section. The same techniques used to find an LQ-optimal stabilization can also be applied to identify a worst-case perturbation of a control system. This leads to the Bounded Real Lemma, which is the topic of Section 2.2. The Bounded Real Lemma itself, is the main tool needed to tackle the disturbance attenuation problem in Section 2.3. Throughout this chapter let A, Ai0 ∈ Kn×n , C ∈ Kq×n ,

i ∈ Kn×L , B1 , B10 D1 ∈ Kq×L ,

i ∈ Kn×m , B2 , B20 D2 ∈ Kq×m .

2.1 Linear quadratic optimal stabilization According to [192], the linear quadratic regulator problem (LQ-control problem) and the Riccati equation were introduced by Kalman in [119]. The corresponding problems for stochastic differential equations were first considered by Wonham in [211, 212, 213] and subsequently e.g. in [97, 98, 164, 37, 52, 190, 69, 218]. As in Section 1.7 we consider a linear stochastic control system dx(t) = Ax(t) dt +

N O i=1

Ai0 x(t) dwi (t) + B2 u(t) dt +

N O

i B20 u(t) dwi (t) . (2.1)

i=1

If u is a given control, then the corresponding solution with initial value x(0) = x0 is denoted by x(t; x0 , u). T. Damm: Rational Matrix Equations in Stochastic Control, LNCIS 297, pp. 43–60, 2004. Springer-Verlag Berlin Heidelberg 2004

44

2 Optimal stabilization of linear stochastic systems

Furthermore, as in Section 1.7, let there be given a weight matrix 0 6 P0 S0 M = > 0. S0∗ Q0 This strictness condition will be weakened later (cf. Corollary 5.3.4), but for the moment it simplifies the presentation. We consider the following linear quadratic optimal control problem: For each given initial value x0 ∈ Kn find a control input u = ux0 such that the quadratic cost-functional 6∗ 0 6 < ∞0 x(t; x0 , u) x(t; x0 , u) J(x0 , u) = E M dt (2.2) u(t) u(t) 0 is minimized. By an adequate choice of M , one can punish large deviations of the state from the equilibrium or large values of the control input, which might be too energy-consuming or even destroy the system. But, as e.g. [135] points out, the main benefit of a positive semidefinite quadratic cost-functional is the fact, that it is mathematically easy to handle and provides some criterion to choose a controller. To formulate the solution of this problem we recall the definition of the Riccati operator from Section 1.7 and set R(X) = P (X) − S(X)Q(X)−1 S(X)∗ 0 =

6∗ 0

I

−Q(X)−1 S(X)∗

(2.3)

P (X) S(X) S(X)∗ Q(X)

60

I

−Q(X)−1 S(X)∗

6 ,

with P (X) = A∗ X + XA +

N O

i Ai∗ 0 XA0 + P0 ,

i=1

S(X) = XB2 +

N O

i Ai∗ 0 XB20 + S0 ,

i=1

Q(X) =

N O

i∗ i B20 XB20 + Q0 .

i=1

The operator R is well defined on the set R dom R = {X ∈ Hn R det Q(X) O= 0} . We also introduce R dom+ R = {X ∈ Hn R Q(X) > 0} ⊂ dom R .

(2.4)

2.1 Linear quadratic optimal stabilization

45

Clearly dom R and dom+ R are open; from Q0 > 0 it is obvious that n H+ ⊂ dom+ R. The condition X ∈ dom+ R represents a constraint on X ∈ Hn . While in Section 1.7 we have already discussed the relation between the feedback stabilization problem and the Riccati inequality, we now strengthen this result by considering optimal open-loop stabilization and the Riccati equation. Theorem 2.1.1 Let M > 0 and R be defined according to (2.3). Then the following are equivalent: (i) System (2.1) is open-loop stabilizable. (ii) System (2.1) is stabilizable by static linear state-feedback. (iii) The Riccati-type matrix equation R(X) = 0 has a positive definite solution X > 0. Moreover, if (iii) holds, then the static linear feedback control u = −Q(X)−1 S(X)∗ x stabilizes (2.1) and minimizes the cost-functional (2.2). Proof: Obviously (ii) implies (i) and by Lemma 1.7.7 we know that (iii) implies (ii). It remains to show that (i) implies (iii). Let T > 0 and define the finite-horizon cost-functional 6∗ 0 6 < T0 x(t; x0 , u) x(t; x0 , u) M JT (x0 , u) = E dt . (2.5) u(t) u(t) 0 On dom+ R we consider the differential Riccati-equation X˙ = −R(X) with boundary condition X(T ) = 0 ∈ int dom+ R; we denote the solution by XT . By time-invariance there exists a number ∆ > 0 independent of T , such that XT (t) ∈ dom+ R exists for all t ∈]T − ∆, T ]. For 0 ≤ t ≤ T < ∆ we can therefore consider a control of the form u(t) = −Q(XT (t))−1 S(XT (t))∗ x(t) + u1 (t) ,

(2.6)

where u1 ∈ L2w ([0, T ]) is arbitrary. By some standard technical calculation, which we present below, we have for all T ∈ [0, ∆[ JT (x0 , u) = x∗0 XT (0)x0 + E

< 0

T

u1 (t)∗ Q(XT (t))u1 (t) dt .

(2.7)

Since Q(X) > 0 for X ∈ dom+ R, the cost-functional is minimized if u1 ≡ 0. Obviously XT (0) > 0, since JT (x0 , u) > 0 for all x0 . Moreover, by assumption for each x0 ∈ Kn there exists a stabilizing control ux0 ∈ L2w ([0, ∞[). Hence for all T > 0

46

2 Optimal stabilization of linear stochastic systems

x∗0 XT (0)x0 =

JT (x0 , u) min R D = R ≤ J(x0 , ux0 ) < ∞ . ≤ JT x0 , ux0 R u∈L2w ([0,T ])

[0,T ]

n In particular, XT (0) ∈ int H+ exists for all T > 0. Since JT (x0 , u) increases with T , it follows that XT (t) = XT −t (0) ≤ xT (0) is uniformly bounded for t ∈]T − ∆, T ]. Thus, for each finite ∆, the solution XT can be extended to an open neighbourhood of T − ∆. We conclude that for all T > 0 the solution XT exists in fact on ] − ∞, T ]. For T 3 > T > t we have XT & (t) > XT (t), since JT (x0 , u) increases with T . Therefore XT (t) converges to some X∞ > 0 as T → ∞, where X∞ is independent of t. By continuity X∞ solves R(X) = 0, and the feedback control u = −Q(X∞ )−1 S(X∞ )∗ x stabilizes system (2.1) by Lemma 1.7.7. Moreover, if we let T = ∞ in (2.7), we see that it yields an LQ-optimal feedback control. For completeness, we verify formula (2.7), which can be found e.g. in [107]. We write x instead of x(t; x0 , u) and omit the argument t. Since XT (T ) = 0, we have for all T ∈ [0, ∆[

JT (x0 , u) =

x∗0 XT (0)x0

< +

T

0

R d R E (x∗ XT (t)x) R ds + JT (x0 , u) . dt t=s

The derivative in the integrand is given by x∗ X˙ T x = −x∗ R(XT )x plus the formula in Remark 1.7.8(i) (with XT instead of X). To this formula, we add the integrand in JT according to (2.5). Thus, we obtain the matrices P (XT ), S(XT ), and Q(XT ). Setting F (XT ) = −Q(XT )−1 S(XT )∗ we have altogether 60 6 < T 0 6∗ 0 x P (XT ) S(XT ) x dt JT (x0 , u) = x∗0 XT (0)x0 + E S(XT )∗ Q(XT ) u u 0 < −E

0

T

∗

x

0

I F (XT )

6∗ 0

P (XT ) S(XT ) S(XT )∗ Q(XT )

60

6 I x dt . F (XT )

If now u has the special form u = F (XT )x + u1 from (2.6), then everything cancels out except for those terms involving u1 . Among these the terms u1 S(XT )∗ x and their complex conjugates cancel out as well, such that finally we retain the integrand in (2.7). " We have needed the strict positivity condition M > 0 in order to guarantee that the control u = −Q(X)−1 S(X)∗ x stabilizes system (2.1). If this can be guaranteed by some other argument, then we may weaken the condition M > 0. Corollary 2.1.2 Let M ∈ Hn+m and R be defined according to (2.3). Assume that X+ ∈ dom+ R and R(X+ ) = 0. If the feedback-control u = −Q(X+ )−1 S(X+ )∗ x

2.2 Worst-case disturbance: A Bounded Real Lemma

47

stabilizes (2.1), then it minimizes the cost-functional (2.2) among all stabilizing controls. Proof: Let the control u(t) = −Q(X+ )−1 S(X+ )∗ x(t) + u1 (t) with u1 ∈ L2w ([0, ∞[) stabilize system (2.1) for some given initial state x0 . Then, like in the proof of Theorem 2.1.1, we have < ∞ ∗ J(x0 , u) = x0 X+ x0 + E u1 (t)∗ Q(X+ )u1 (t) dt ≥ x∗0 X+ x0 , 0

with equality if and only if u1 = 0.

"

The idea of the proof of Theorem 2.1.1 is the same as in the deterministic case. But apart from the fact that here we need Itˆ o‘s formula to derive (2.7) there is another important difference: For deterministic linear systems it can easily be seen from the Kalman normal form (e.g. [135]) that open-loop stabilizability implies stabilizability by static state feedback. Under the condition of state feedback stabilizability, however, one can use algebraic or iterative techniques to prove the solvability of the Riccati equation. We will develop such a general framework for a large class of operator equations in Chapters 4 and 5. On the other hand, if feedback stabilizability is not given a priori one really has to invoke an open-loop control to obtain an upper bound for the solution of the differential Riccati equation; this is necessary in order to show that the solution does not escape in finite time. The analogous situation on a more complicated level occurs in the proof of the so-called Bounded Real Lemma in [107].

2.2 Worst-case disturbance: A Bounded Real Lemma A problem opposite to that of finding an optimal stabilizing control is to determine a worst-case disturbance of a stable system. To be more precise, let us consider the system dx(t) = Ax(t) dt +

N O

Ai0 x(t) dwi (t) + B1 v(t) dt +

i=1

N O

i B10 v(t) dwi (t) (2.8)

i=1

z(t) = Cx(t) + D1 v(t) ,

(2.9)

where now we regard v ∈ L2w ([0, ∞[; L2 (Ω, KL )) as an external disturbance. Definition 2.2.1 We call system (2.8) internally stable, if the unperturbed system dx(t) = Ax(t) dt +

N O i=1

Ai0 x(t) dwi (t)

48

2 Optimal stabilization of linear stochastic systems

is asymptotically mean-square stable. We call system (2.8) externally stable, if for all perturbations v ∈ L2w ([0, ∞[; L2 (Ω, KL )) the output z is also in L2w , i.e. z ∈ L2w ([0, ∞[; L2 (Ω, Kq )). If (2.8) is externally stable, then we define the perturbation operator L : L2w ([0, ∞[; L2 (Ω, KL )) → L2w ([0, ∞[; L2 (Ω, Kq )) by Lv(·) = z(·; 0, v). We will see below that L is a bounded operator, if system (2.8) is internally i stable. In the deterministic case (when all Ai0 and B10 vanish) the perturbation operator can be represented in the frequency domain by the transfer function G(s) = C(sI − A)−1 B1 + D1 . Its norm is given by supω∈R 0. At this point one needs the so-called Bounded Real Lemma, to characterize an upper bound for the H ∞ -norm in terms of matrix inequalities. A version of the Bounded Real Lemma for linear systems with stochastic parameter uncertainties has been proven in [107, 62]. It involves a matrix operator of the form (5.13) with 0 6 −C ∗ C −C ∗ D1 M = . (2.10) −D1∗ C γ 2 I − D1∗ D1 The main idea is to look at the cost functional < T ? 2 F γ γ
(2.11)

2.2 Worst-case disturbance: A Bounded Real Lemma

P (X) = A∗ X + XA +

N O

49

∗ i Ai∗ 0 XA0 − C C ,

i=1

S(X) = XB1 +

N O

∗ i Ai∗ 0 XB10 − C D1 ,

i=1

Qγ (X) =

N O

i∗ i B10 XB10 + γ 2 I − D1∗ D1 .

i=1

This operator is well defined on

R dom Rγ = {X ∈ Hn R det Qγ (X) O= 0} .

Moreover, in analogy to (2.4) we set

R dom+ Rγ = {X ∈ Hn R Qγ (X) > 0} .

(2.12)

It is straightforward to prove one implication of the Bounded Real Lemma (cf. [62]). Proposition 2.2.2 If for some γ > 0 there exists a matrix X < 0 such that Qγ (X) > 0 and Rγ (X) > 0, then (2.8) is both internally and externally stable and 0 and Qγ (X) > 0 imply P (X) > 0 and hence the internal stability of (2.8). For arbitrary T > 0, we consider a general perturbation v ∈ L2w ([0, T ]) and write x(t) = x(t; 0, v). Applying the formula from Remark 1.7.8(i) in integral form we have (with Ex(0)∗ Xx(0) = JTγ (0, v)) 6 60 6∗ 0 < T0 x(t) x(t) P (X) S(X) JTγ (0, v) = −Ex(T )∗ Xx(T ) + E dt v(t) v(t) S(X)∗ Qγ (X) 0 >0. The inequality follows from X < 0, Qγ (X) > 0, and Rγ (X) > 0 (cf. Lemma A.2). Hence (2.8) is externally stable and 0. Choosing γ sufficiently large we also have Qγ (X) > 0 and " Rγ (X) > 0 and can apply the first part of the proposition. In fact, if the assumptions of the Proposition hold for some γ > 0, then they remain valid, if we decrease γ slightly; hence, we have even 0 to prove the non-strict inequality 0 there exists an X ∈ dom+ Rγ , such that X ≤ 0 and Rγ (X) ≥ 0, then
50

2 Optimal stabilization of linear stochastic systems

Proof: Like in the proof of Proposition 2.2.2 we have JTγ (0, v) ≥ 0 for all T > 0 and arbitrary v ∈ L2w ([0, T ]). Hence 0 under the assumption that (2.8) is internally stable and 0.

2.3 Disturbance attenuation Disturbance attenuation is an important design object for stabilizing controllers. As the examples in the first chapter illustrate, it is often a natural performance criterion that the effect of some exogenous perturbation be diminished or annihilated. Moreover, the concept of disturbance attenuation can be utilized to achieve robustness properties with respect to parameter uncertainties and modelling errors (compare [87]). The idea is to treat these uncertainties as exogenous perturbations resulting from some feedback connection. In other words, one models the uncertainty as another dynamical system, coupled with the nominal system. By virtue of the so-called Small Gain Theorem (e.g. [70]) one can show that such an interconnection of two systems is stable, if the product of the norms of the corresponding perturbation operators is smaller than 1. Hence, one can guarantee a larger robustness margin by decreasing the norm of the perturbation operator of the nominal system. The same method can be applied to stochastic systems (e.g. [58]). Here, however, we will treat the disturbance attenuation problem only as a means to diminish the effect of exogenous disturbances. For deterministic systems the disturbance attenuation problem is widely known as an optimal or suboptimal H ∞ -control problem. It was originally formulated in [219]. A solution of the static-state feedback H ∞ -control problem involving Riccati equations has been given in [221]. We refer to [87] for more details. For stochastic systems with state-dependent noise, the problem of disturbance attenuation by static state-feedback has been considered e.g. in [57, 58, 194, 163]. The more general problem of disturbance attenuation by dynamic output feedback in the case where the state x and the disturbance input v are corrupted by multiplicative white noise, has been the topic of [107, 62]. Finally,

2.3 Disturbance attenuation

51

in [42] we have discussed the problem of disturbance attenuation by static state-feedback in the case of regular system where the state and all inputs are possibly subject to multiplicative white noise. This case will be discussed in the following. In Section 2.3.2 we consider the same problem for bounded instead of white noise parameter uncertainties and in Section 2.3.3 we review some facts on disturbance attenuation by dynamic output feedback. 2.3.1 Disturbance attenuation by static linear state feedback We consider a linear stochastic system with a disturbance input v ∈ KL , a control input u ∈ Km , and an output z ∈ Kq : dx(t) =

Ax(t) dt +

N O

Ai0 x(t) dwi (t)

i=1

+ B1 v(t) dt +

N O

i B10 v(t) dwi (t)

(2.13)

i=1

+ B2 u(t) dt +

N O

i B20 u(t) dwi (t)

i=1

z(t) = Cx(t) + D1 v(t) + D2 u(t). Our aim is to construct a state-feedback control u = F x, such that (2.13) is internally stabilized and the effect of v on z is reduced. To be more precise, let us consider the closed-loop system ; dx(t) = Ax(t) dt +

N O

;i x dwi + B1 v dt + A 0

N O

i B10 v dwi

(2.14)

i=1

i=1

; + D1 v, z = Cx with ; = A + B2 F, A

i ;i0 = Ai0 + B20 A F,

and

; = C + D2 F . (2.15) C

The solutions of (2.14) with initial value x0 at t = 0 are denoted by x(t; x0 , v); the corresponding output processes are denoted by z(t; x0 , v). Definition 2.3.1 Let γ > 0. The γ-suboptimal stochastic H ∞ -problem consists in finding a feedback-gain matrix F , such that system (2.14) is internally (mean-square) stable and the perturbation operator LF : v L→ z of (2.14) has norm
52

2 Optimal stabilization of linear stochastic systems

z = LF v

v (2.13)

u = Fx

Fig. 2.1. The closed-loop system and the perturbation operator LF

Theorem 2.3.2 System (2.14) is internally (mean-square) stable and the perturbation operator LF has norm X = X ∗ ∈ Kn×n such that Qγ1 (X) > 0

and

RγF (X) := PF (X) − SF (X)Qγ1 (X)SF∗ (X) > 0 . (2.16)

Here the affine linear matrix operators PF , SF and Qγ1 are given by ;∗ X + X A ;+ PF (X) = A

N O

;i∗ ;i ;∗ ; A 0 X A0 − C C ,

(2.17)

i=1

SF (X) = XB1 +

N O

;i∗ XB i − C ; ∗ D1 , A 0 10

i=1

Qγ1 (X) =

N O

i∗ i B10 XB10 + γ 2 I − D1∗ D1 .

i=1

Hence the γ-suboptimal stochastic H ∞ -problem is solvable if and only if there exists a pair of matrices (F, X) such that (2.16) is satisfied. To delete the matrix F from this condition we impose a regularity assumption and apply Lemma 1.7.6. Let us first introduce the following notation. We set     −C ∗ C −C ∗ D2 −C ∗ D1 P0 S20 S10 ∗ Q20 S30  , (2.18) M γ =  −D2∗ C −D2∗ D2 −D2∗ D1  =  S20 ∗ ∗ 2 ∗ ∗ ∗ −D1 C −D1 D2 γ I − D1 D1 S10 S30 Q10  i∗  i i∗ i N A0 XAi0 Ai∗ 0 XB20 A0 XB10 O i∗ i i∗ i i∗ i   B20 XA0 B20 XB20 B20 XB10 , (2.19) Π(X) = i∗ i i∗ i i∗ i i=1 B10 XA0 B10 XB20 B10 XB10 and with the same partition we define   ∗   P (X) S2 (X) S1 (X) A X + XA XB2 XB1  S2 (X)∗ Q2 (X) S3 (X)  =  B2∗ X 0 0  + Π(X) + M γ . γ ∗ ∗ ∗ S1 (X) S3 (X) Q1 (X) B1 X 0 0

2.3 Disturbance attenuation

53

Note that Π(X) ≥ 0 for X ≥ 0, i.e. Π is a positive operator (cf. Def. 3.2.1). Definition 2.3.3 We call system (2.13) regular if D2∗ D2 > 0. Regularity of (2.13) implies Q2 (X) < 0 for all X ≤ 0. Often one also assumes D2∗ C = 0 and D2∗ D2 = I for regular systems, since this can always be achieved by some transformation (compare Section 5.1.7). In general, however, we do not make this assumption. Regularization problems will be mentioned in the discussion of numerical examples in Section 5.5. Theorem 2.3.4 Let system (2.13) be regular. The γ-suboptimal stochastic H ∞ -problem is solvable, if and only if there exists a negative definite matrix X < 0, such that Qγ1 (X) > 0

(2.20)

and the Riccati-type inequality 0

S2 (X)∗ R (X) = P (X) − S1 (X)∗ γ

6∗ 0

Q2 (X) S3 (X) S3 (X)∗ Qγ1 (X)

6−1 0

S2 (X)∗ S1 (X)∗

>0

6 (2.21)

is satisfied. For any matrix X < 0 satisfying (2.20) and (2.21), the feedback-gain matrix F = −(Q2 (X) − S3 (X)Qγ1 (X)−1 S3 (X)∗ )−1 × (S2 (X)∗ − S3 (X)Qγ1 (X)−1 S1 (X)∗ )

(2.22)

solves the γ-suboptimal stochastic H ∞ -problem. Proof: Let us first write RγF in a different form. By definition PF (X) = (A∗ + F ∗ B2∗ )X + X(A + B2 F ) + ∗ + P0 + F ∗ S20 + S20 F + F ∗ Q20 F

SF (X) = XB1 +

N O

∗ i∗ i i (Ai∗ 0 + F B20 )X(A0 + B20 F )

i=1

(2.23)

N O

∗ i∗ i ∗ (Ai∗ 0 + F B20 )XB10 + S10 + F S30 ,

i=1

such that RγF (X) = PF (X) − SF (X)Qγ1 (X)−1 SF (X)∗ ∗ ˆ ˆ ˆ = Pˆ (X) + S(X)F + F ∗ S(X) + F ∗ Q(X)F with

(2.24)

54

2 Optimal stabilization of linear stochastic systems

Pˆ (X) = P (X) − S1 (X)Qγ1 (X)−1 S1 (X)∗ , ˆ S(X) = S2 (X) − S1 (X)Qγ1 (X)−1 S3 (X)∗ , ˆ Q(X) = Q2 (X) − S3 (X)Qγ1 (X)−1 S3 (X)∗ .

(2.25)

ˆ Note that Q(X) is negative definite if X < 0 and Qγ1 (X) > 0. In this event, it follows from Lemma 1.7.6 (with inverted signs) that (for the given X < 0) −1 ˆ ˆ the right hand side of (2.24) is maximized by F = F (X) = −Q(X) S(X)∗ . ∞ In other words, if the γ-suboptimal H -problem is solvable then, by Theorem 2.3.2, there exists a matrix X < 0 such that Qγ1 (X) > 0 and −1 ˆ ˆ ˆ Rγ (X) = RγF (X) (X) = Pˆ (X) − S(X) S(X)∗ > 0 ; Q(X)

(2.26)

vice versa, if such an X exists, then the matrix F in (2.22) solves the γsuboptimal H ∞ -problem. It remains to verify, that Rγ has the form specified in (2.21). To this end we write Rγ (X) as the Schur-complement (see Def. A.1) G 0 6N 6 @0 ˆ ˆ Pˆ (X) S(X) Pˆ (X) S(X) ˆ Q(X) of the matrix Rγ (X) = S ∗ ˆ ∗ ˆ ˆ ˆ S(X) S(X) Q(X) Q(X) ˆ Furthermore we write with respect to the block Q(X).    0 6 P (X) S2 (X) S1 (X) N ˆ ˆ P (X) S(X) = S  S2 (X)∗ Q2 (X) S3 (X)  Qγ1 (X) , ∗ ˆ ˆ S(X) Q(X) S (X)∗ S (X)∗ Qγ (X) 1

3

1

i.e. we interpret Rγ (X) as a double Schur-complement. Applying the quotient formula cited in Lemma A.2 we find    6 P (X) S2 (X) S1 (X) N 0 (X) S (X) Q 2 3  Rγ (X) = S  S2 (X)∗ Q2 (X) S3 (X)  S3 (X)∗ Qγ1 (X) γ ∗ ∗ S1 (X) S3 (X) Q1 (X) 6−1 0 6∗ 0 6 0 Q2 (X) S3 (X) S2 (X)∗ S2 (X)∗ . = P (X) − S3 (X)∗ Qγ1 (X) S1 (X)∗ S1 (X)∗ " From our calculations we can also draw the following conclusions. Corollary 2.3.5 Let system (2.13) be regular. Assume that X ≤ 0 satisfies Qγ1 (X) > 0 and let the feedback-gain matrix F (depending on X) be defined according to (2.22). 3

(i) Rγ (X) = RγF (X) and for the derivatives at X we have RγX = (RγF )3X . i (ii) If (A + B2 F, (Ai0 + B20 F )) is stable and Rγ (X) ≥ 0, then
2.3 Disturbance attenuation

55

−1 ˆ ˆ S(X)∗ Proof: (i) The first assertion follows from (2.26). With F = −Q(X) γ we have for the derivative of R at X in direction H 3

RγX (H) = P 3 (H) + S 3 (H)F + F ∗ Q3 (H)F + F ∗ S 3 (H)∗ = (RγF )3X (H) . Here we have applied the product rule for Rγ , whereas RγF was evaluated according to (2.24). (ii) This follows from Corollary 2.2.3. "

Remark 2.3.6 Together with the constrained Riccati-type inequality (2.21) and (2.20) we study the constrained Riccati-type equation Rγ (X) = 0

with X < 0 and Qγ1 (X) > 0 .

(2.27)

We can recover the Riccati equations from different types of H ∞ -control problems in (2.27). For simplicity let D2∗ [C, D2 ] = [0, I] and D1 = 0 (cf. Section 5.1.7). (i)

If Π = 0, i.e. all stochastic terms vanish, then (2.27) specializes to the indefinite Riccati equation of deterministic continuous-time H ∞ -control (e.g. [75, 145]) A∗ X + XA − C ∗ C − X(−B2∗ B2 + γ −2 B1∗ B1 )X = 0

(ii) If A = − 21 I, B1 = 0, B2 = 0, equation (2.27) turns into its counterpart from discrete-time stochastic control (compare [61]). (iii) If A = − 12 I, B1 = 0, B2 = 0, and  ∗  A0 XA0 A∗0 XB20 A∗0 XB10 ∗ ∗  , XA0 B20 XB20 0 Π(X) =  B20 ∗ ∗ B10 XA0 0 B10 XB10 we obtain the constrained indefinite Riccati equation of deterministic ∗ discrete-time H ∞ -control (e.g. [75, 96]): B10 XB10 + γ 2 I > 0, and ∗ ∗ −X + A∗0 XA0 − C ∗ C − A∗0 XB20 (B20 XB20 − I)−1 B20 XA0 ∗ ∗ 2 −1 ∗ − A0 XB10 (B10 XB10 + γ I) B10 XA0 = 0 . i (iv) If we let γ → ∞ or if we set B1 = 0 and B10 = 0 for all i = 1, . . . , N in either of the Riccati-type equations from H ∞ -control, we end up with the corresponding Riccati-type equation from LQ-control. i = 0 for all i = 1, . . . , N , (v) If on the other hand, we set B2 = 0 and B20 then we recover the Riccati-type equation associated to the Bounded Real Lemma. In other words, the Bounded Real Lemma describes a disturbance attenuation problem without control.

56

2 Optimal stabilization of linear stochastic systems

Of course, we can write the Riccati-operator Rγ in the customary form Rγ (X) = P (X) − S(X)Qγ (X)S(X)∗ , if we set S(X) =

/

5 S2 (X) S1 (X) ,

Qγ (X) =

0

Q2 (X) S3 (X) S3 (X)∗ Qγ1 (X)

6 .

Remark 2.3.7 (i) By our regularity assumption, it is obvious that the constraint (2.27) implies Qγ1 (X) > 0 and Q2 (X) < 0. We set R dom± Rγ = {X ∈ dom Rγ R Qγ1 (X) > 0 and Q2 (X) < 0} . (2.28) In particular, Qγ (X) is indefinite for X ∈ dom± Rγ . Hence we call this constraint indefinite, in contrast to the definite constraints X ∈ dom+ R and X ∈ dom+ Rγ associated to (2.4) and (2.12). Clearly, the indefinite constraint is associated to an indefinite input weight cost. Nevertheless, we prefer to use the terms definite and indefinite constraints rather than costs, since e.g. the definite constraint Q(X) > 0 can be fulfilled even for an indefinite cost matrix Q0 . This particularity of stochastic systems has been discussed e.g. in [26]. (ii) Another remark on our notation is in order. One might expect the indices 1 and 2 in 3 × 3-block matrices like (2.18) to occur in reverse order. It is, however, standard to write B1 for the matrix at the disturbance input v and B2 for the matrix at the control input u. If one views the disturbance attenuation problem as a two-player game, then the disturbance v is regarded as the first player whose action has to be answered by the second player u. This convention already fixes the indices. On the other hand, we have arranged the block-matrices in such a way that we can easily evaluate Schur-complements. If we switched the blocks, we would have to take the Schur-complement with respect to the block in the middle, which is really confusing. 2.3.2 Systems with bounded parameter uncertainty We have seen in Section 1.6 that mean-square stability with a given decay rate α in the presence of white noise parameter uncertainties guarantees a certain robustness margin with respect to bounded uncertainties. The same idea can be extended to the γ-suboptimal H ∞ -problem. In analogy to the systems (2.13) and (2.14) we consider the open-loop system > E N O i x˙ = A + δi (t, x)A0 x > +

i=1

B1 +

N O

E i δi (t, x)B10

i=1

z = Cx + D1 v + D2 u .

v+

> B2 +

N O

E i δi (t, x)B20

u

i=1

(2.29)

2.3 Disturbance attenuation

The corresponding closed-loop system is > E > E N N O O i i ;+ x(t) ˙ = A δi (t, x)A;0 x + B1 + δi (t, x)B10 v i=1

57

(2.30)

i=1

; + D1 v , z = Cx with ; = A + B2 F, A

i ;i0 = Ai0 + B20 A F,

and

; = C + D2 F . C

The δi are arbitrary measurable real or complex functions, which are bounded by given numbers di > 0, i.e. ∀t > 0, x ∈ Rn : |δi (t, x)| < di . We denote the solutions of (2.30) by xδ (t; x0 , v) and the output by zδ (t; x0 , v). System (2.30) is called internally stable, if for all x0 the unperturbed solution xδ (t; x0 , 0) is exponentially stable, i.e. ∃ω > 0, M > 1 : ∀x0 ∈ Kn , t > 0 : <xδ (t; x0 , 0)< < M eωt . For a given γ > 0 let the matrix operators Qγ1 and Rγ be given like in (2.20) and (2.21). We will give a sufficient stabilization criterion with guaranteed disturbance attenuation bound γ for system (2.29) in terms of these matrix operators. Again we make use of the simple Lemma 1.6.1. Theorem 2.3.8 Let γ > 0, α = matrix X < 0, such that Qγ1 (X) > 0

PN

i=1

and

d2i and assume that there exists a

Rγ (X) > −αX .

(2.31)

If F is given by (2.22) then system (2.30) is internally stable; moreover zδ (·; 0, v) ∈ L2 (R+ , RL ) if v ∈ L2 (R+ , Rq ) and −αX implies P (X) > −αX, that is ;∗ X + X A ; + αX + A

N O

;i∗ ;i A 0 X A0 > 0 .

i=1

Hence the internal stability follows from Corollary 1.5.4 and Proposition 1.6.2. Now we consider the finite-horizon cost functional < T F ? 2 JT (0, v) = γ 0. With X < 0 we have < T d ∗ JT (v) ≥ JT (v) + x(T ) Xx(T ) = JT (v) + (x(t)∗ Xx(t)) dt . 0 dt

58

2 Optimal stabilization of linear stochastic systems

Computing the derivative in the integrand and using the previous expression for JT (v), we find that the right-hand side is nonnegative, if . 4 4∗ . 4 0 6 0 6 . ∗ ; −C ; ∗ D1 ;∗ C ;∗ ;∗ −C A A I I (2.32) + X ;∗ + 0 < ∆0 + ; ∗ X ; γ 2 I − D ∗ D1 0 0 B1 B1 −D1∗ C 1 where ∆0 =

N O i=1

>

4 0 6 0 6 . i∗ 4∗ E ∗ ; ;i∗ A A I I . + δi (t, x) X ; 0i∗ δ¯i (t, x) ; 0i∗ X 0 0 B10 B10 .

By Lemma A.2 condition (2.31) is equivalent to (2.32) if we replace ∆0 by 0 6 0 6∗ . i∗ 4 . i∗ 4∗ N O ; ; A A I 2 I ∆1 := . di + ; 0i∗ X ; 0i∗ X 0 0 B10 B10 i=1

By Lemma 1.6.1, the i-th summand in ∆0 exceeds the i-th summand in ∆1 , if " |δi (t, x)| < di , whence (2.32) holds. For T → ∞ we get
Remark 2.3.9 (i) If condition (2.31) holds for some value γ then it also holds for all γ˜ in a neighbourhood of γ. Hence for v O= 0, we even have
2.3 Disturbance attenuation

dx(t) =

Ax(t) dt +

N O

59

Ai0 x(t) dwi (t)

i=1

+ B1 v(t) dt +

N O

i v(t) dwi (t) + B2 u(t) dt , B10

(2.33)

i=1

z(t) = Cx(t) + D1 v(t) + D2 u(t), y(t) = C2 x(t) + D21 v(t) . Here C2 ∈ Kp×n , D21 ∈ Kp×L , and y is the measured output. It is to be used in a dynamic compensator of the form dˆ x(t) = AK xˆ(t) dt + BK y(t) dt ,

AK ∈ Knˆ ׈n , BK ∈ Knˆ ×p , CK ∈ Km׈n , DK ∈ Km×p ,

u(t) = CK xˆ(t) + DK y(t) ,

(2.34)

to produce a γ-suboptimal stabilizing control u for (2.33). The dimension n ˆ > 0 of the compensator is arbitrary. If we feed the output u of (2.34) back into (2.33), then we obtain a closedˆT ]T , the disturbance loop system with the extended state-vector x ¯ = [xT , x input v, and the output z. The Bounded Real Lemma can now be applied to characterize the performance specification of our disturbance attenuation problem in terms of a matrix inequality involving the design parameters AK , BK , CK , and DK . Following the ideas in [75] one can use the so-called Projection Lemma to eliminate the design parameters from this matrix inequality. The method resembles a little bit the completion of the square as we have applied it above, but it does not require any regularity assumptions at this stage. After further technical manipulations one ends up with a coupled pair of linear matrix inequalities for matrices 0 > Y > X ∈ Hn . Under the regularity conditions D1 = 0,

D2∗ D2 = Im ,

D2∗ C1 = 0,

∗ D21 D21 = Ip ,

D21 B1∗ = 0

these linear matrix inequalities take the form of the Riccati-type inequalities Rγ1 (X) =

P1γ (X) − S1 (X)Qγ1 (X)−1 S1 (X)∗

>0,

Rγ2 (X, Y ) = P2 (X, Y ) − S2 (X, Y )Qγ1 (X)−1 S2 (X, Y )∗ > 0 with P1γ (X) = XA + A∗ X +

N O

i ∗ 2 ∗ Ai∗ 0 XA0 − C1 C1 + γ C2 C2 ,

i=1

S1 (X) = XB1 +

N O

i Ai∗ 0 XB10 ,

i=1

Qγ1 (X) = γ 2 I +

N O i=1

i∗ i B10 XB10 ,

(2.35)

60

2 Optimal stabilization of linear stochastic systems

P2 (X, Y ) = Y A + A∗ Y +

N O

∗ ∗ i Ai∗ 0 XA0 − C1 C1 + Y B2 B2 Y ,

i=1

S2 (X, Y ) = Y B1 +

N O

i Ai∗ 0 XB10 .

i=1

Both inequalities in (2.35) can be seen as special cases of (2.21). But the coupling makes them much harder to solve. We will come back to these inequalities later and name some of the specific difficulties. In the main, however, we will restrict our attention to the solution of one isolated generalized Riccati equation. Dealing with coupled equations will certainly be a topic for future research. Remark 2.3.10 Note that, in contrast to (2.13), the model (2.33) does not incorporate parameter uncertainties at the control input. In fact, an extension of the model in this direction leads to much more complicated matrix inequalities in the Bounded Real Lemma and makes the problem hardly tractable. Similarly, it would be interesting to consider uncertain parameters in the output equations as well. This might also be a topic of further research.

3 Linear mappings on ordered vector spaces

Before we can start to analyze the generalized Riccati operators derived in the previous chapter, we have to deal with generalized Lyapunov operators of the type occuring e.g. in Theorem 1.5.3. It is crucial to observe that these Lyapunov operators possess certain positivity properties, which will be discussed in the following. The present chapter is split in two parts. In the first part, we introduce the general notions of ordered Banach spaces and resolvent positive operators, which is the set-up to be used in Chapter 4. In the second part, we focus on the ordered vector space Hn of Hermitian matrices, which is relevant for our further investigations in Chapter 5.

3.1 Ordered Banach spaces In this section, we summarize some basic concepts and results from the theories of ordered vector spaces and resolvent positive linear operators. Let X be a real Banach space. Following the terminology in [14] we say that a nonempty subset C ⊂ X is a convex cone if C + C = C, αC ⊂ C for all real numbers α ≥ 0. If the cone is pointed (i.e. C ∩ −C = {0}) such a cone C induces an ordering on X. For x, y ∈ X we write x ≥ y, if x − y ∈ C. If C has interior points R and x − y ∈ int C, then we write x > y. If x ≤ y, the set [x, y] = {z ∈ X R x ≤ z ≤ y} is called the order interval between x and y. We will need the following definitions [139]. Definition 3.1.1 Given a closed convex cone C in the real BanachR space X, the dual cone C ∗ in the dual space X ∗ is given by C ∗ := {y ∗ ∈ X ∗ R ∀x ∈ C : 1x, y ∗ 2 ≥ 0}. (i) C is reproducing if C − C = X. (ii) C is solid if int C O= ∅. (iii) C is normal if ∃b > 0 ∀x, y ∈ C : x ≤ y ⇒ <x< ≤ b
62

3 Linear mappings on ordered vector spaces

(iv) C is regular if every monotonically decreasing sequence x1 ≥ x2 ≥ . . ., which is bounded from below by some element x ˆ ∈ X, converges in norm. A vector x ∈ ∂C is called an extremal of C, if 0 ≤ y ≤ x implies that y is a nonnegative multiple of x. Example 3.1.2 (i) The positive orthant Rn+ of n-dimensional real vectors with nonnegative entries is a pointed convex cone in the Euclidean space Rn ; it has the properties (i)–(iv) from Definition 3.1.1. (ii) Let Hn ⊂ Kn×n , K = R or K = C, denote the real Hilbert space of real or complex n × n Hermitian matrices, endowed with the Frobenius inner product 1X, Y 2 = trace (XY ∗ ) = trace (XY ) and the corresponding R n := {X ∈ Hn R X ≥ 0} (Frobenius) norm <·< (compare Section A). By H+ we denote the subset of nonnegative definite matrices. The space Hn is canonically ordered by this pointed closed convex cone which satisfies all n the conditions (i)–(iv) from Definition 3.1.1. The interior of the cone H+ n is the set of positive definite matrices in H , while its boundary consists n are the nonof all singular nonnegative matrices. The extremals of H+ ∗ negative matrices X ≥ 0 with rk X = 1, i.e. X = xx for some x ∈ Kn (see [14]) The following lemma collects some elementary facts about convex cones. Lemma 3.1.3 Let C be a closed convex cone in a real Banach space X. (i) (ii) (iii) (iv) (v) (vi)

If C is solid, then it is reproducing. In finite dimensions the converse is also true. If C is normal, then it is pointed. In finite dimensions the converse is also true [13, Ex. (2.13)]. C is normal if and only if C ∗ is reproducing [139, Thm. 4.5]. C is reproducing if and only if C ∗ is normal [139, Thm. 4.6]. If C is regular, then it is normal [139, Thm. 5.1]. If C is pointed and finite-dimensional then it coincides with the convex hull of its extremals. (This is a special case of the Krein-Milman Theorem, e.g. [139, Thm. 3.1].)

In Chapter 4 our main results will be derived for real Banach spaces X ordered by a closed solid, regular convex cone C. It then follows from Lemma 3.1.3 that both C and C ∗ are pointed, reproducing, and normal.

3.2 Positive and resolvent positive operators Definition 3.2.1 Let X be a normed vector space ordered by a pointed convex cone C. A bounded linear operator T : X → X is called positive if T (C) ⊂ C. It is called inverse positive if it has a bounded positive inverse and resolvent positive if there exists an α0 ∈ R, such that for all α > α0 the operators αI − T are inverse positive.

3.2 Positive and resolvent positive operators

63

If T is positive and invertible then T (int C) ⊂ int C. Hence if T is inverse positive then T −1 (int C) ⊂ int C, i.e. int C ⊂ T (int C). Example 3.2.2 (i) Let Rn be ordered by the cone Rn+ . A matrix A ∈ Rn×n , regarded as a mapping A : Rn → Rn , is positive, if and only if all its entries are nonnegative. It is resolvent positive, if and only if all offdiagonal entries of A are nonnegative, i.e. A is a Metzler matrix. This is, by definition, equivalent to saying that −A is a Z-matrix. We call A stable, if σ(A) ⊂ C− . Hence, again by definition, A is resolvent positive and stable, if and only if −A is an M -matrix (see [158, 113, 14]). General properties of stable resolvent positive operators are discussed in Theorem 3.2.10. (ii) Let A ∈ Kn×n , then the operator ΠA : Hn → Hn , ΠA (X) = A∗ XA is positive with respect to the canonical ordering of Hn , whereas both the continuous-time Lyapunov operator LA : Hn → Hn , LA (X) = A∗ X + XA, and the discrete-time Lyapunov operator (also Stein operator) SA : Hn → Hn , SA (X) = A∗ XA − X, are resolvent positive but, in general, not positive (see Section 3.4). 3.2.1 Spectral properties For a bounded linear operator T : X → X we denote the spectrum by σ(T ) and set β(T ) = max{Re(λ); λ ∈ σ(T )} for the spectral abscissa, ρ(T ) = max{|λ|; λ ∈ σ(T )} for the spectral radius of T . It is well-known that σ(T ) = σ(T ∗ ) and that the adjoint operator T ∗ is (resolvent) positive with respect to the positive cone C ∗ if and only if T is (resolvent) positive with respect to cl C. The spectrum of positive operators on Rn ordered by the cone Rn+ was analyzed first by Perron and Frobenius, who showed that the spectral radius of such an operator is an eigenvalue corresponding to a nonnegative eigenvector (e.g. [78, 112]). This result was extended by Krein and Rutman in [140] to more general spaces and cones. For instance the following holds: If T is a positive linear operator in a real Banach space X ordered by a closed, normal and reproducing convex cone C, then ρ(T ) ∈ σ(T ) (see [139], Thm. 8.1). Moreover, if, for instance, T is compact, or even X is finite-dimensional then T x = ρ(T )x for some non-zero x ∈ C. In general, however, it is not true that ρ(T ) is an eigenvalue of T . But if one considers the adjoint operator T ∗ instead of T , the existence of an eigenvector in C ∗ corresponding to the eigenvalue ρ(T ∗ ) = ρ(T ) is guaranteed under fairly general conditions. Theorem 3.2.3 Let X be a real Banach space, ordered by a closed, normal, solid convex cone C, and T : X → X a bounded linear operator.

64

3 Linear mappings on ordered vector spaces

(i) T is positive ⇒ ∃v ∈ C ∗ , v O= 0: T ∗ v = ρ(T )v. (ii) T is resolvent positive ⇒ ∃v ∈ C ∗ , v O= 0: T ∗ v = β(T )v. Proof: (i) A proof of (i) can be found in [177] App. 2.6, or [139] Thm. 9.11. (ii) Let α > β(T ). First we note that λ ∈ σ(T ) if and only if 1/(α − λ) ∈ σ((αI − T )−1 ). For sufficiently large α the operator (αI − T )−1 is positive. By (i) there exists a v ∈ C ∗ \{0}, such that (αI − T ∗ )−1 v = ρα v, where ρα = ρ((αI − T ∗ )−1 ). Multiplying this equation from the left by αI−T ∗ we obtain v = ρα αv−ρα T ∗ v. Hence λ0 = α − 1/ρα ∈ R is an eigenvalue of T ∗ with eigenvector v. Moreover, λ0 = max(σ(T ) ∩ R), because for λ ∈ σ(T ) ∩ R we have 1/(α − λ) ≤ ρα = 1/(α − λ0 ), whence λ ≤ λ0 . In particular, λ0 and v do not depend on α. To show that in fact β(T ) = λ0 , let λ ∈ σ(T ). Then |1/(α−λ)| ≤ ρα = 1/(α−λ0 ) for all sufficiently large α and hence 0 ≤ (α − Re λ)2 + (Im λ)2 − (α − λ0 )2 = (λ0 − Re λ)(2α − λ0 − Re λ) + (Im λ)2 . Since α can be made arbitrarily large, this inequality implies λ0 ≥ Re λ prov" ing λ0 = β(T ). We use this result to give a simple proof for the monotonicity of ρ and β: Corollary 3.2.4 Let X be a real Banach space, ordered by a closed, normal, solid convex cone C and let S and T : X → X be bounded linear operators, such that T − S is positive. (i) If S is positive, then so is T and ρ(T ) ≥ ρ(S). (ii) If S is resolvent positive, then so is T and β(T ) ≥ β(S). Proof: (i) If S is positive, then clearly T is positive. Let S ∗ v = ρ(S)v for v ∈ C ∗ with 0, such that b
Lemma 3.2.5 Let X be a Banach space ordered by a closed convex F ? pointed cone. If S : X → X is inverse positive and T is positive, such that ρ S −1 T < 1, then S − T is inverse positive.

3.2 Positive and resolvent positive operators

65

? F−1 −1 P∞ ? −1 Fk −1 Proof: By assumption (S −T )−1 = I − S −1 T S = k=0 S T S is a convergent series of positive operators and thus positive (because C is closed). "

Proposition 3.2.6 Let X be a Banach space ordered by a closed solid, normal convex cone C, α ∈ R, and T : X → X be a resolvent positive linear operator. Then αI − T is inverse positive if and only if α > β(T ), i.e. σ(αI − T ) ⊂ C+ . Proof: ‘⇒’ Let v be a positive eigenvector of T ∗ corresponding to β(T ) (Theorem 3.2.3). Then αI − T can only be inverse positive, if (αI − T ∗ )−1 v = (α − β(T ))−1 v ≥ 0, whence α > β(T ). ‘⇐’: Since T is resolvent positive, there exists an α0 ∈ R such that αI − T is inverse positive for all α > α0 . We choose the minimal α0 with this property and show that it coincides with β(T ). From the implication ‘⇒’ it is obvious, that α0 ≥ β(T ). We assume α0 > β(T ). Then (α0 I − T )−1 exists and is positive by continuity. Now we choose α ˜ ∈ [β(T ), α0 [, such that ρ((α0 − α)(α ˜ 0 I − T )−1 ) < 1. It follows from ˜ )I is inverse positive, conLemma 3.2.5, that αI ˜ − T = (α0 I − T ) − (α0 − α tradicting the minimality of α0 . "

Remark 3.2.7 Since we frequently consider strict inequalities, it is natural to assume C to be solid. In some of the previous results, however, this assumption can be avoided. For instance in Theorem 3.2.3 one could also assume C to be an arbitrary closed convex pointed cone such that C ∗ is locally compact, see [139, Thm. 9.6]. Alternatively, if in (i) T is compact, or in (ii) (αI − T )−1 is compact for α > β(T ), it suffices if C is reproducing instead of solid; see [139, Thm. 9.2]. In this case, also T (not only T ∗ ) possesses a positive eigenvector. Corresponding changes can be made in the assumptions of Proposition 3.2.6. Similarly, the assertions of Corollary 3.2.4 hold if C is only reproducing instead of solid; see [149], [23]. For the spectral theory of resolvent positive operators see also [65], [5] and [68]. The paper [197] gives a nice overview of the spectral theory of positive operators in finite-dimensional ordered vector spaces. Before we formulate the main theorem in this section, we introduce a generalized notion of detectability in analogy to Definition 1.8.1. This definition is motivated by Lemma 1.8.4 and Remark 1.8.6. Definition 3.2.8 Let X be a real Banach space, ordered by a closed, solid, normal convex cone C. Let y ∈ C∪−C and |y| = ±y, so that |y| ∈ C. Consider an operator T : X → X and assume that either T or −T is resolvent positive. The pair (T, y) is called β-detectable, if 1|y|, v2 > 0 for every eigenvector v ∈ C ∗ of T ∗ , corresponding to an eigenvalue λ with Re λ ≥ 0. If T (x)−|y| < 0 for some x ∈ C, then we call the pair (T, y) detectable.

66

3 Linear mappings on ordered vector spaces

It is easy to see that detectability implies β-detectability. The converse is not true, as we have already noted in Section 1.8. Lemma 3.2.9 Let T or −T be resolvent positive and y ∈ C. If the pair (T, y) is detectable, then it is β-detectable. Proof: If (T, y) is detectable then T (x) − y < 0 for some x ∈ C. If T ∗ v = λv for some eigenpair (λ, v) ∈ C × C ∗ \{0} satisfying 1y, v2 = 0 then ¯ v2 . 0 > 1T (x) − y, v2 = 1x, T ∗ (v)2 = λ1x, Since 1x, v2 ≥ 0, we have λ < 0.

"

The following theorem is an infinite-dimensional generalization of a result by H. Schneider and plays a central role. The proof is adapted from [180]. Theorem 3.2.10 Let X be a real Banach space ordered by a closed, solid, normal convex cone C. Suppose R : X → X to be resolvent positive and P : X → X to be positive, and set T = R + P . Then the following are equivalent: (i) (ii) (iii) (iv) (v) (vi) (vii)

T is stable, i.e. σ(T ) ⊂ C− . −T is inverse positive. ∀y ∈ int C : ∃x ∈ int C : −T (x) = y ∃x ∈ int C : −T (x) ∈ int C. ∃x ∈ C : −T (x) ∈ int C. ∃x ∈ C : y = −T (x) ? ∈ C Fand (T, y) is β-detectable. σ(R) ⊂ C− and ρ R−1 P < 1.

Proof: (i) ⇔ (ii): By Corollary 3.2.4 the operator T is resolvent positive and thus by Proposition 3.2.6 the conditions (i) and (ii) are equivalent. (ii)⇒(iii): If −T is inverse positive then −T −1 maps int C into int C, which implies (iii). The chain (iii)⇒(iv)⇒(v)⇒(vi) is trivial. (vi)⇒(i): Assume that (vi) holds for some x ∈ C, but (i) fails. Then β := β(T ) ≥ 0 and T ∗ has an eigenvector v ∈ C ∗ corresponding to β by Theorem 3.2.3. This implies 0 ≥ 1−y, v2 = 1T x, v2 = 1x, T ∗ v2 = 1x, βv2 ≥ 0 , whence 1y, v2 = 0 in contradiction to the β-detectability; hence (i) holds. It remains to prove that (vii) is equivalent to (i)–(vi). (vii)⇒(ii): Suppose (vii), then −R is inverse positive by Proposition 3.2.6 and ρ((−R)−1 P ) < 1. Applying Lemma 3.2.5 we obtain that −T = −R − P is inverse positive, i.e. (ii). (i), (ii), (iv)⇒(vii): Since β(R + P ) ≥ β(R) by Corollary 3.2.4, condition (i) implies σ(R) ⊂ C− , whence −R is inverse positive and Π := −R−1 P is

3.2 Positive and resolvent positive operators

67

positive. By (iv) there exists a positive vector x ∈ int C, such that −T x ∈ int C and since −R−1 ≥ 0 this implies R−1 T x = (I − Π) x ∈ int C. But by Theorem 3.2.3 there ?exists a vF ∈ C ∗ , such that Π ∗ v = ρ(Π)v. Therefore 0 < 1(I − Π) x, v2 = 1 − ρ(Π) 1x, v2, whence ρ(Π) < 1 because 1x, v2 > 0. " Decompositions of the form T = R + P play a role in iterative methods. We will come back to this in Section 3.5. The following definition is adapted from the theory of M -matrices (see e.g. [198, 14]). Definition 3.2.11 Let T be a resolvent positive operator. A decomposition of T into the sum T = R + P with a resolvent positive operator R and a positive operator P is called a regular splitting. The regular splitting T = R + P is called convergent if σ(R) ⊂ C− and ρ(R−1 P ) < 1. 3.2.2 Equivalent characterizations of resolvent positivity In [66] the following equivalent conditions were given for an operator in a finite dimensional space to be resolvent positive (see also [181], [13]). Theorem 3.2.12 Let X be a finite-dimensional real vector space ordered by a closed, solid, normal convex cone C. For linear operators T : X → X the following are equivalent: (i) (ii) (iii) (iv) (v)

T is resolvent positive. exp(tT ) is positive for all t ≥ 0. ∀x ∈ ∂C : ∃v ∈ ∂C ∗ : 1x, v2 = 0 and 1T x, v2 ≥ 0. x ∈ C, v ∈ C ∗ ,R 1x, v2 = 0 ⇒ 1T x, v2 ≥ 0. T ∈ cl{S − αI R S : X → X positive, α ∈ R}.

Remark 3.2.13 (i) A resolvent positive operator is also called Metzler operator e.g. in [68, 109]. Condition (ii) is often called exponential positivity or exponential nonnegativity, e.g. [13]. In [66] an operator is called quasimonotonic, if it possesses property (iii). Operators satisfying (iv) are called cross-positive in [181] and C-subtangential in [13]. An operator T is called essentially nonnegative in [13] if T +αI is positive for sufficiently large α ∈ R. (ii) Though [66] only treats the finite-dimensional case the proof carries over immediately to the situation where X is a real Banach space. But we will not make use of this generalization. (iii) From (iv) we see that the resolvent positive operators form a convex cone. This cone contains all scalar multiples of the identity operator and hence is not pointed. In the following section we determine the maximal subspace in the cone of resolvent positive operators on Hn .

68

3 Linear mappings on ordered vector spaces

Corollary 3.2.14 The set of resolvent positive operators R : Hn → Hn is a closed convex cone. It contains the convex cone of positive operators and all real multiples of the identity. In particular it is solid, but not pointed. Following [90], we use the term lineality space for the maximal subspace contained in the cone of resolvent positive operators. It is an interesting question, dating back to [185], whether every resolvent positive operator T can be represented as the sum T = L+P

(3.1)

of a positive operator P and an element L from the lineality space. Although in [185] an affirmative answer could be given for important classes of cones, it was shown in [90] that this representation is impossible for almost all cones in a certain categorial sense.

3.3 Linear mappings on the space of Hermitian matrices In the following we consider the ordered vector space Hn of Hermitian matrices (defined in the appendix), which plays an important role in applications. Its structure is surprisingly rich and a number of elementary questions is still unanswered. For instance, up to today it is unknown, how to classify positive operators on Hn . While in the case of the vector space Rn ordered by Rn+ one can easily check whether an operator is positive by inspecting the entries of its matrix representation, there is no analogous result for the space Hn . An early mention of this problem can be found in [180]. Several attempts have been made e.g. in [51, 28, 104, 29, 30, 56, 215, 10, 105, 146] to prove partial or related results, some of which we will recapitulate in this Section. Similarly, we do not have a representation theorem for resolvent positive operators on Hn . In particular we do not know, whether a representation of the form (3.1) is possible for resolvent positive operators T : Hn → Hn , because n the cone H+ falls into neither of the classes considered by [185] and [90]. As a partial result in this direction we show in Section 3.4 that the lineality space of Hn coincides with the set of Lyapunov operators. Hence we conjecture that every resolvent positive operator on Hn can be written as the sum of a Lyapunov operator and a positive operator. Note that for the special class of completely positive operators (see Section 3.3.2) the existence of such a representation has been proven in [147]1 . In Section 3.5 we discuss some numerical approaches to solve linear equations with resolvent positive operators, while in Section 3.6 we recapitulate the relation between resolvent positivity and stability. Finally, Section 3.7 contains some results on minimal representations of the inverse of a Sylvester operator. 1

Unaware of this, I posed the corresponding question as an open problem in [44].

3.3 Linear mappings on the space of Hermitian matrices

69

3.3.1 Representation of mappings between matrix spaces We recall some results on the representation of linear mappings between matrix spaces. Since we wish to point out some further details, that are not stated explicitly in the literature, we include the proofs. In the following we consider mappings T : Km×n → Kp×q . Notice that each mapping T : Hn → Hn can be extended to a K-linear mapping T : Kn×n → Kn×n . The extension is unique if K = C, while in the case K = R the mapping T is not uniquely determined for skew-symmetric matrices. Obviously, a linear mapping T : Km×n → Kp×q can also be regarded as a linear mapping between the vector spaces Kmn and Kpq . To make use of this observation, we recall the definition and some basic properties of the Kronecker product and the vec-operator (e.g. [113]). Definition 3.3.1 Let V = (vjk ) = (v1 , . . . , vn ) ∈ Km×n and U = (ujk ) ∈ Kp×q . Then   v1 = D   V ⊗ U = vjk U ∈ Kmp×nq and vec V =  ...  ∈ Knm . vn Lemma 3.3.2 Let U1 , U2 , V1 , V2 , and X be matrices of appropriate sizes. Then (V1 ⊗ U1 )(V2 ⊗ U2 ) = (V1 V2 ) ⊗ (U1 U2 ) and vec(U1 XV1 ) = (V1T ⊗ U1 ) vec X . (m)

In the following let ej (mn) Ejk

(m) (n) ∗ ej ek

denote the j-th canonical unit vector in Km and

= the m × n matrix with the only nonzero entry 1 in the j-th row and k-th column. By E mn we denote the m2 × n2 -block matrix = Dm,n (mn) Ejk = vec Im (vec In )∗ and with the mapping T we associate the j,k=1

mp × nq matrix = D(m,n) (mn) (Ipq ⊗ T )(E (mn) ) = T (Ejk ) . j,k=1

(3.2)

Remark 3.3.3 Here, we understand the notation (Ipq ⊗ T )(E (mn) ) as an abbreviation. If we really wanted to understand Ipq =⊗ T as Da Kroneckerm,n (mn) product, we would have to arrange the entries of Ejk in the j,k=1 = Dm,n (mn) vector vec vec Ejk and identify T with its matrix representation j,k=1

70

3 Linear mappings on ordered vector spaces

as an operator from K=mn → Kpq .DFinally we would have to rearrange the m,n (mn) product (Ipq ⊗ T ) vec vec Ejk ∈ Kmpnq as an mp × nq-matrix. j,k=1

The following simple identity is useful. Lemma 3.3.4 If V = (v1 , . . . , vm ) ∈ Kp×m and W = (w1 , . . . , wn ) ∈ Kq×n then = Dm,n (mn) V Ejk W ∗ = vec V (vec W )∗ . (3.3) j,k=1

Proof: By Lemma 3.3.2 we have (mn)

vec(V Ejk

Thus,

(mn)

V Ejk

¯ ⊗ V vec E (mn) W ∗) = W jk = D (nm) = w ¯1 ⊗ v1 , w ¯1 ⊗ v2 , . . . , w ¯q ⊗ vm e(k−1)m+j

W∗

=w ¯k ⊗ vj = vec(vj wk∗ ) . Dm,n = (mn) = vj wk∗ and V Ejk W ∗

j,k=1

∗

vec V (vec W ) .

=

Dm,n = vj wk∗

j,k=1

= "

Theorem 3.3.5 Let T : Km×n −→ Kp×q be linear. The matrices V1 , . . . , VN ∈ Kp×m , W1 , . . . , WN ∈ Kn×q yield a representation of T : ∀X ∈ Km×n : T (X) =

N O

Vj XWj∗

(3.4)

j=1

if and only if (Ipq ⊗ T )(E (mn) ) =

N O

vec Vj (vec Wj )∗ .

(3.5)

j=1

In particular, the minimal number of summands is ν = rk(Ipq ⊗ T )(E (mn) ), and one can choose a representation with matrices V1 , . . . , Vν ∈ Kp×m , W1 , . . . , Wν ∈ Kn×q , such that (vec Vj )∗ vec Vk = (vec Wj )∗ vec Wk = 0 for all j, k ∈ {1, . . . , ν} with j O= k. Proof: By Lemma 3.3.4 the identity (3.4) clearly implies (3.5); vice versa (3.5) P m×n mn implies T (X) = N . j=1 Vj XWj for all X = Ejk , and thus for all X ∈ K A representation with a minimal number of summands is given e.g. by a singular value decomposition of (Ipq ⊗ T )(E (mn) ), in which case also the orthogonality property holds. " Now we consider mappings between spaces of quadratic matrices.

3.3 Linear mappings on the space of Hermitian matrices

71

Definition 3.3.6 Let T : Kn×n −→ Km×m be a linear map. Then T is called Hermitian-preserving if T (Hn ) ⊂ Hm . It is immediate to see, that the Lyapunov operator is Hermitian-preserving. The following representation result can be found in [104]; the proof is adapted from [29]. Theorem 3.3.7 For a linear map T : Kn×n −→ Km×m consider the following assertions: (i) T is Hermitian-preserving. (ii) ∀X ∈ Kn×n : T (X ∗ ) = (T (X))∗

= Dn (nn) (iii) The nm×nm matrix (Imm ⊗T )(E (nn) ) = T (Ejk ) (iv) There exist matrices V1 , . . . , VN ∈ K {−1, 1}, such that for all X ∈ Kn×n T (X) =

N O

m×n

j,k=1

is Hermitian.

and numbers ε1 , . . . , εN ∈

εj Vj XVj∗ .

(3.6)

j=1

The minimal number of summands is N = rk(Imm ⊗ T )(E (nn) ). One can choose εj = 1 for all j, if and only if (Imm ⊗ T )(E (nn) ) ≥ 0. If K = C, then all these assertions are equivalent. If K = R, then (ii), (iii) and (iv) are equivalent, and each of them implies (i). Proof: (i)⇒(ii) for K = C: For skew-Hermitian matrices S (i.e. iS Hermitian) we have on the one hand T (iS) = T (iS)∗ = −iT (S)∗ and on the other T (iS) = T ((iS)∗ ) = T (−iS ∗) = −iT (S ∗ ) which yields T (S)∗ = T (S ∗ ). As any matrix A can be decomposed in A = H + S, where H = 12 (A + A∗ ) is Hermitian and S = 12 (A − A∗ ) is skew-Hermitian, the result follows. (ii)⇒ (iii): This follows from G G @= G∗ @= @= Dn Dn Dn (nn) ∗ (nn) (nn) T (Ejk )∗ T (Ejk ) T (Ejk ) = = k,j=1 k,j=1 j,k=1 @= G @= G Dn Dn (nn) (nn) T (Ekj ) T (Ejk ) = = . k,j=1

j,k=1

(iii)⇒ (iv): By Sylvester’s inertia theorem there is a decomposition

72

3 Linear mappings on ordered vector spaces

= Dn (nn) T (Ejk )

j,k=1

N O

=

εL vec VL (vec VL )∗

(3.7)

L=1

for appropriate VL , where N is the rank of the matrix on the left. By equation (3.3) we can also write = Dn (nn) T (Ejk )

>

j,k=1

=

N O L=1

that means (nn)

∀Ejk

:

(nn)

T (Ejk ) =

En (nn) εL VL Ejk VL∗ j,k=1 N O L=1

(nn)

εL VL Ejk VL∗

(nn) R and thus the same holds with replaced by any X ∈ span{Ejk R j, k = 1, . . . , n} = Kn×n . Moreover it follows from Sylvester’s Theorem, that one can choose εj = 1 for all j if and only if (Imm ⊗ T )(E (nn) ) ≥ 0. (iv) ⇒ (i): For each l the matrix VL∗ XVL is Hermitian, if X is Hermitian; hence the sum is Hermitian, too. = D (nn) Ejk

(iv) ⇒ (ii): Since for each l and arbitrary X we have VL∗ XVL it follows that T (X)∗ = T (X ∗ ).

∗

= VL∗ X ∗ VL "

Remark 3.3.8 In the real case, (i) does not necessarily imply (ii). For instance, we can define a linear mapping T : R2×2 → R2×2 via T (e1 e∗1 ) = T (e2 e∗2 ) = 0,

T (e1 e∗2 ) = e1 e∗2 ,

T (e2 e∗1 ) = e1 e∗1 + e2 e∗1 .

It is easy to see that T satisfies (i) but not (ii). Since every linear mapping T : Hn → Hn can be extended to a linear mapping T : Cn×n → Cn×n such that (ii) holds, we can draw a simple conclusion. Corollary 3.3.9 Every linear mapping T : Hn → Hn has a representation of the form (3.6); i.e. there are matrices Vj ∈ Kn×n and numbers εj ∈ {−1, 1}, such that (3.6) holds for all X ∈ Hn . The trace of an extension T : Kn×n → PN Kn×n is then given by trace T = j=1 εj |trace Vj |2 . PN ¯ Proof: As T corresponds to the matrix l=1 Pi Vl ⊗ Vl , the trace formula follows from the easily verified identity, trace V ⊗ W = trace V trace W . "

3.3 Linear mappings on the space of Hermitian matrices

73

3.3.2 Completely positive operators Let T : Kn×n → Kn×n be a Hermitian-preserving operator with the repren n ) ⊂ H+ , if all εj are sentation (3.6). Then clearly T is positive, i.e. T (H+ nonnegative. Vice versa, let us verify that for the operator T : X L→ X ∗ which 2 to itself, there are no matrices Vj ∈ Kn×n such that maps H+ T (X) =

N O

Vj XVj∗

(3.8)

j=1

for all X ∈ K2×2 . In view of Theorem 3.3.7 (iv) it suffices to observe that   1000 0 0 1 0  (I22 ⊗ T )(E (22) ) =  0 1 0 0 0001 is indefinite, which is obviously the case. But, perhaps more surprisingly, there also exist positive operators T : Hn → Hn , that do not allow a representation of the form (3.8) for all X ∈ Hn . The first example was given in [30]; it has the form   ax11 + bx22 + cx33 Ta,b,c : X = (xjk ) L→ diag  ax22 + bx33 + cx11  − X ax33 + b11 + cx22 with a = b = c = 2. We do not repeat the proof here. More generally, it was shown in [27] that Ta,b,c is positive if the following conditions are satisfied D = a ≥ 1, a + b + c ≥ 3, and 1 ≤ a ≤ 2 ⇒ bc ≥ (2 − a)2 . Further examples can be found in [189, 146]. Of course, the representation (3.8) is very convenient. It corresponds to a special type of positivity which was introduced for general C ∗ -algebras in [187]. Definition 3.3.10 An operator T : Kn×n → Km×m is called completely positive, if for any ^ ∈ N and any nonnegative definite ^n × ^n-block-matrix L L (Xjk )j,k=1 with Xjk ∈ Cn×n the matrix (T (Xjk ))j,k=1 is nonnegative definite. If T : Kn×n → Km×m is completely positive, then in particular (Imm ⊗ T )(E (nn) ) ≥ 0, since E (nn) = vec In (vec In )∗ ≥ 0. By Theorem 3.3.7 this implies that T can be written in the form (3.8). Conversely, it is immediate to see that any operator of the form (3.8) is completely positive. We will not explore the subtle distinction between positive and completely positive maps much further. But it is worth noting that our abstract results

74

3 Linear mappings on ordered vector spaces

in Chapter 4 only require the notion of positivity, while in our concrete applications to Riccati equations we make use of special representations in the form (3.8), i.e. we work with completely positive maps. Similarly, we will make use of complete positivity in the Sections 3.5 and 3.6. The increasing generality of the concepts of complete positivity, positivity and resolvent positivity can be seen from the following nice – though almost trivial – characterization. Lemma 3.3.11 Let T : Kn×n → Kn×n be a linear operator and consider the (n2 × n2 )-matrix X = (Inn ⊗ T )(E (nn) ). Then 2

(i) T is completely positive ⇐⇒ ∀z ∈ Kn : z ∗ X z ≥ 0. (ii) T is positive ⇐⇒ ∀x, y ∈ Kn : (x ⊗ y)∗ X (x ⊗ y) ≥ 0. (iii) T is resolvent positive ⇐⇒ ∀x, y ∈ Kn with x⊥y: (x ⊗ y)∗ X (x ⊗ y) ≥ 0. Proof: (i): We have already noted that T is completely positive, if and only if it has a representation of the form (3.8). By Theorem 3.3.7 this is equivalent to X ≥ 0. n (ii): By definition, T is positive if T (X) ≥ 0 for all X ∈ H+ . Since T is linear n it suffices to consider the extremals of H+ . Hence, T is positive if and only if y ∗ T (xx∗ )y ≥ 0 for all x, y ∈ Kn . Writing T in the form (3.6), we have y ∗ T (xx∗ )y = vec(y ∗ T (xx∗ )y) =

n O

εj vec(y ∗ V x) vec(y ∗ V x)∗

j=1

=

n O

εj (¯ x∗ ⊗ y¯∗ vec V )(¯ x∗ ⊗ y¯∗ vec V )∗

j=1

=x ¯∗ ⊗ y¯∗

n =O

D ¯ ⊗ y¯ εj (vec V )(vec V )∗ x

j=1

= (¯ x ⊗ y¯)∗ X (¯ x ⊗ y¯) .

(3.9)

Since x, y ∈ Kn were arbitrary, we can replace them by their complex conjugates to prove our assertion. (iii): By Theorem 3.2.12 the operator T is resolvent positive, if 1T (X), Y 2 ≥ 0 n with 1X, Y 2 = 0. Again it suffices to consider extremals for all X, Y ∈ H+ ∗ n . Since 1xx∗ , yy ∗ 2 = trace xx∗ yy ∗ = y ∗ xx∗ y = 0 X = xx and Y = yy ∗ of H+ if and only if x⊥y, it follows that T is resolvent positive if and only if 1T (xx∗ ), yy ∗ 2 = y ∗ T (xx∗ )y ≥ 0 for all x⊥y. Now the assertion follows from equation (3.9). " It follows immediately that, like positive and resolvent positive operators, the completely positive operators form a closed convex cone. Corollary 3.3.12 The set of completely positive operators T : Kn×n → Kn×n is a closed convex pointed solid cone.

3.4 Lyapunov operators and resolvent positivity

75

In the following sections we will discuss an important class of resolvent positive operators on Hn that are not positive, namely Lyapunov operators.

3.4 Lyapunov operators and resolvent positivity A linear operator T : Hn → Hn is said to be a (continuous-time) Lyapunov operator, if there exists an A ∈ Kn×n , such that ∀X ∈ Hn :

T (X) = A∗ X + XA .

In this case we also write T = LA . Note that LA can be written in the form (3.6) as D 1= (A + I)∗ X(A + I) − (A − I)∗ X(A − I) . LA = 2

(3.10)

(3.11)

The role of Lyapunov operators in stability theory is well-known. If for instance, we consider a homogeneous linear deterministic differential equation X˙ = A∗ X,

A, X ∈ Kn×n , X(0) = X0 ,

(3.12)

with the solution X(t), then P (t) = X(t)∗ X(t) ≥ 0 satisfies the equation P˙ = LA (P ) .

(3.13)

As an obvious conclusion, one obtains that A is stable if and only if LA is stable. This is useful, since the stability analysis of LA leads to matrix equations like in Theorem 1.5.3. Furthermore, we conclude that LA generates a positive group. For arbitrary P (0) ≥ 0 the solution P (t) of (3.13) is nonnegative for all t ∈ R. This follows from the fact that we can solve (3.13) forward and backward in time (which, in general, is not the case for stochastic differential equations). Hence, we conclude that ±LA is exponentially and thus resolvent positive. We formulate this result as a Lemma, and give another proof. Lemma 3.4.1 If T is a Lyapunov operator, then both T and −T are resolvent positive. Proof: Let A ∈ Kn×n . By Lyapunov’s Theorem (e.g. [113]), T = LA is inverse positive if σ(LA ) ⊂ C+ . Thus αI − LA = L−A+ α2 I is inverse positive for sufficiently large α ∈ R. Since −T = L−A is a Lyapunov operator, too, it is also resolvent positive. " It is not so clear that in fact Lyapunov operators are the only operators with this property. In other words, each positive linear group on Hn is associated to a differential equation of the form (3.12). This is the content of Theorem 3.4.3 below. To the best of the author’s knowledge, this result is documented in the literature only for the case, where in the assertion ’positive’ is replaced

76

3 Linear mappings on ordered vector spaces

by ’completely positive’, e.g. [147, 4]. Before stating the result, we verify that the situation is different, if T is a discrete-time Lyapunov operator or Stein operator. This means that there exists an A ∈ Kn×n such that ∀X ∈ Hn :

T (X) = A∗ XA − X .

(3.14)

In this case we also write T = SA . It follows e.g. from Lemma 3.2.5 or Theorem 3.2.12 (v) that SA is resolvent positive. But this is not necessarily the case for T = −SA as the following example shows. 6 0 00 and consider the family of indefinite maExample 3.4.2 Let A = 10 0 6 01 with t > 0. For all α > 0 we have αXt + SA (Xt ) = trices Xt = 61 t 0 t α−1 , which is positive for large t though Xt is always indefinite. α − 1 αt Hence αI + SA is not inverse positive for any α, and hence −SA is not resolvent positive. Theorem 3.4.3 Let T : Hn → Hn be a linear operator. The following are equivalent: (i) T is a Lyapunov operator, i.e. ∃A ∈ Kn×n : T = LA . (ii) T and −T are resolvent positive. Proof: ‘(i)⇒(ii)’ follows from Lemma 3.4.1. ‘(ii)⇒(i)’: We begin with the real case K = R. Let T and −T be resolvent positive. First we note that by Theorem 3.2.12 this is equivalent to the following criterion: = D X, Y ≥ 0 and 1X, Y 2 = 0 ⇒ 1T X, Y 2 = 0 . (3.15) If ej denotes the j-th canonical unit vector in Rn , then the set R n B := {ej eTk + ek eTj R j, k = 1, . . . , n} ⊂ H+

(3.16)

forms a basis of Hn ⊂ Rn×n . Thus we have to find an A ∈ Rn×n , such that T (X) = LA (X) for all X ∈ B. Let X = ej eTj (i.e. 2X ∈ B). To apply criterion (3.15) we characterize all matrices n Y ∈ H+ such that 1X, Y 2 = 0 .

(3.17)

Let Y ≥ 0 and 1X, Y 2 = yjj = 0. It is well known, that a diagonal entry of a positive semidefinite matrix can only vanish if both its whole row and column vanish. Hence, (3.17) is true if and only if in Y ≥ 0 the j-th row

3.4 Lyapunov operators and resolvent positivity

77

and column vanish. Criterion (3.15) in turn implies that in T (X) everything vanishes except for the j-th row and column. Otherwise we could choose some Y satisfying (3.17) and 1T (X), Y 2 O= 0. For j = 1, . . . , n we thus have T (ej eTj ) = aj eTj +ej aTj with vectors a1 , . . . , an ∈ Rn . If we build the matrix A = (a1 , . . . , an ), then T (X) = AX + XAT for all X = ej eTj . In other words, we have found a unique candidate for the Lyapunov operator. It remains to show, that also for Xjk = ej eTk + ek eTj with j < k we have T (Xjk ) = AXjk + Xjk AT = aj eTk + ak eTj + ej aTj + ek aTk 

a1k .. .

···

a1j .. .

ank

···

anj

(3.18)



       a1k · · · 2ajk · · · ajj + akk · · · ank      .. .. =  . . .    a1j · · · ajj + akk · · · 2akj · · · anj      .. ..   . . Let j and k be fixed. A matrix Y satisfies condition (3.17) with X = Xjk + Xjj + Xkk ≥ 0 if in Y the j-th and k-th row and column vanish. As above we conclude from criterion (3.15), that in T (X) and hence also in T (Xjk ) everything vanishes except for the j-th and k-th row and column. Thus T (Xjk ) is of the general form T (Xjk ) = bj eTj + ej bTj + bk eTk + ek bTk , 

b1j .. .

···

b1k .. .

bnj

···

bnk

with bj , bk ∈ Rn

(3.19)



       b1j · · · 2bjj · · · bjk + bkj · · · bnj      .. .. =  . . .    b1k · · · bjk + bkj · · · 2bkk · · · bnk      .. ..   . . Now we consider matrices of the form X = xxT with x = xj ej + xk ek where xj , xk ∈ R are arbitrary real numbers. Writing X = xj xk Xjk + x2j ej eTj + x2k ek eTk , and exploiting the linearity of T we have the decomposition T (X) = xj xk T (Xjk ) + x2j T (ej eTj ) + x2k T (ek eTk )

78

3 Linear mappings on ordered vector spaces

= xj xk (bj eTj + ej bTj + bk eTk + ek bTk ) + x2j (aj eTj + ej aTj ) + x2k (ak eTk + ek aTk ) . Let y⊥x, for instance   y1  ..  y =  .  ∈ Rn yn

(3.20)

  yj = xk , yk = −xj , with  yL arbitrary for ^ O∈ {j, k} .

(3.21)

Then the matrix Y = yy T satisfies condition (3.17), and by (3.15) we have 1T (X), Y 2 = 0. If we write T (X) like in equation (3.20) we obtain: 1 1 1 1T (X), Y 2 = trace (T (X)Y ) = y T T (X)y 2 2 2 E > n n n n O O O O = xj xk yj bLj yL + yk bLk yL + x2j yj aLj yL + x2k yk aLk yL

0=

L=1

=

L=1

L=1

L=1

xj x3k (bjj − ajk ) + x3j xk (bkk − akj ) + x2j x2k (−bkj − bjk + ajj + akk ) O O + xj x2k yL (bLj − aLk ) + x2j xk yL (−bLk + aLj ) . L0∈{j,k}

L0∈{j,k}

The right hand side is a homogeneous polynomial in the real unknowns xj , xk , and yL for ^ O∈ {j, k}. Since these unknowns can be chosen arbitrarily, all the coefficients of the polynomial necessarily vanish, i.e. bjj = ajk , bkk = akj , bkj + bjk = ajj + akk , bLj = aLk , bLk = aLj . Inserting these data into (3.19), we see that (3.18) holds. In the complex case K = C the computations are a little bit more involved, because we have to deal with real and imaginary parts. It has to be noted that now dim Hn = n2 while in the real case the dimension was n(n + 1)/2. In particular B must be completed to a basis by R (3.22) Bi = {(ej + iek )(ej + iek )∗ R 1 ≤ j < k ≤ n} . Like in the real case we obtain a candidate for the Lyapunov operator LA with A = (a1 , . . . , an )∗ ∈ Cn×n from the relations T (ej e∗j ) = aj e∗j + ej a∗j . But this candidate is not unique in the complex case; we can add an arbitrary diagonal matrix with purely imaginary entries to A without changing (ej e∗j )A + A∗ (ej e∗j ). In particular, for arbitrarily given real numbers β1 , . . . , βn−1 ∈ R we can choose the ajj such that

3.4 Lyapunov operators and resolvent positivity

Im(ajj − ann ) = βj ,

j = 1, . . . , n − 1 .

79

(3.23)

This will be needed at the end of the proof. i = iej ek − iek e∗j we have to verify that Setting Xjk = ej e∗k + ek e∗j and Xjk T (Xjk ) = AXjk + Xjk A∗ = aj e∗k + ak e∗j + ej a∗j + ek a∗k 

a1k .. .

···

a1j .. .

(3.24)



      a ¯kk · · · a ¯nk   ¯1k · · · 2 Re ajk · · · ajj + a    .. .. =  . .   a  · · · a ¯ + a · · · 2 Re a · · · a ¯ ¯ jj kk kj nj   1j   . . .. ..   ··· anj ank and i i i T (Xjk ) = AXjk + Xjk A∗ = iaj e∗k − iak e∗j + iej a∗k − iek a∗j (3.25)   ··· ia1j −ia1k   .. ..   . .    i¯ · · · −ia + i¯ a · · · ia + i¯ a · · · i¯ a a jk jk jj kk nk    1k   .. .. =  . (3.26) . .    −i¯ ajj − iakk · · · −i¯ akj + iakj · · · a ¯nj    a1j · · · −i¯   .. ..   . . ··· ianj −iank

Let j and k be fixed. Like in the real case we conclude that T (Xjk ) is of the general form T (Xjk ) = bj e∗j + ej b∗j + bk e∗k + ek b∗k , 

b1j .. .

with bj , bk ∈ Rn

 b1k  ..  .  · · · bjk + ¯bkj · · · ¯bnj    ..  . .  ¯ · · · 2 Re bkk · · · bnk    ..  . ···

    ¯b1j · · · 2 Re bjj   .. = .   ¯b1k · · · ¯bjk + bkj   ..  . ··· bnj

bnk

i ) has the same form with bj , bk replaced by some cj , ck ∈ Cn . Clearly T (Xjk Now we consider matrices of the form X = xx∗ with x = xj ej + xk ek where xj , xk ∈ C are arbitrary complex numbers. Writing

80

3 Linear mappings on ordered vector spaces i X = Re(¯ xj xk )Xjk − Im(¯ xj xk )Xjk + |xj |2 ej e∗j + |xk |2 ek e∗k ,

we have the decomposition T (X) = Re(¯ xj xk )(bj e∗j + ej b∗j + bk e∗k + ek b∗k ) − Im(¯ xj xk )(cj e∗j + ej c∗j + ck e∗k + ek c∗k ) + |xj |2 (aj e∗j + ej a∗j ) + |xk |2 (ak e∗k + ek a∗k ) .

(3.27)

Similarly as in (3.21) we choose Y = yy ∗ with    y1 ¯k ,  yj = x   y =  ...  ∈ Cn and yk = −¯ xj ,  arbitrary for ^ O∈ {j, k} , y L yn such that y⊥x. Then 1 1 1 1T (X), Y 2 = trace (T (X)Y ) = y ∗ T (X)y 2 2 E > 2 n n O O bLj y¯L + yk bLk y¯L = Re(¯ xj xk ) Re yj

0=

L=1

>

− Im(¯ xj xk ) Re yj

n O L=1

> 2

+ |xj | Re yj >

n O

aLj y¯L

L=1

cLj y¯L + yk

n O

E cLk y¯L

L=1

>

E 2

+ |xk | Re yk

L=1

n O

E aLk y¯L

L=1

O

¯k = Re(¯ xj xk ) Re |xk |2 bjj − x¯k xj bkj + x

bLj y¯L

L0∈{j,k}

E

O

2

− x¯j xk bjk + |xj | bkk − x¯j

bLk y¯L

L0∈{j,k}

>

O

¯k xj ckj + x ¯k − Im(¯ xj xk ) Re |xk |2 cjj − x

cLj y¯L

L0∈{j,k}

−x ¯j xk cjk + |xj | ckk − x ¯j 2

¯k xj akj + x ¯k + |xj | Re |xk | ajj − x > + |xk |2 Re

cLk y¯L

L0∈{j,k}

> 2

E

O

2

O

E

aLj y¯L

L0∈{j,k}

− x¯j xk ajk + |xj |2 akk + x ¯j

O L0∈{j,k}

E aLk y¯L

.

3.4 Lyapunov operators and resolvent positivity

81

We distinguish between several special cases, where the variables xj , xk and yL are of the form xj = ξj zi , xk = ξk zk , yL = ηL zL with fixed complex numbers zj , zk , zL and real variables ξj , ξk , ηL . In these cases the right hand side is a homogeneous, identically vanishing polynomial in ξj , ξk , ηL . By inspecting the coefficients at different monomials we obtain relations between the entries of A, bj , bk , cj and ck . For each case considered, we provide a list of some monomials and the corresponding vanishing coefficients. (i)

xj = ξj , xk = ξk . ξj ξk3 ξj3 ξk ξj2 ξk2 ξj ξk2 ηL ξj2 ξk ηL

yL = ηL yL = iηL Re(bjj − ajk ) = 0 Re(bkk − akj ) = 0 Re(−bkj − bjk + ajj + akk ) = 0 Re(bLj − aLk ) = 0 Im(bLj − aLk ) = 0 Re(−bLk + aLj = 0 Im(−bLk + aLj ) = 0

Hence bjj , bkk and bLj , bLk for all ^ O∈ {j, k} have the required form. (ii) xj = ξj , xk = iξk . ξj ξk3 ξj3 ξk ξj2 ξk2 ξj ξk2 ηL ξj2 ξk ηL

yL = ηL yL = iηL Re(cjj ) − Im(ajk ) = 0 Re(ckk ) − Im(akj ) = 0 Im(cjk − ckj ) − Re(ajj + akk ) = 0 Re(cLj ) − Im(aLk ) = 0 Im(cLj ) − Re(aLk ) = 0 Im(cLk ) + Re(aLj ) = 0 Re(cLk ) + Im(aLj ) = 0

Hence cjj , ckk and cLj , cLk for all ^ O∈ {j, k} have the required form. (iii) xj = ξj , xk = (1 + i)ξk , yL = 0. Considering the coefficient at ξj2 ξk2 we obtain = 0 = Re − (1 − i)bkj − (1 + i)bjk + 2ajj D + (1 − i)ckj + (1 + i)cjk + 2akk = Re(−bkj − bjk + ajj + akk ) + Im(−bkj + bjk ) − Re(−ckj − cjk ) − Im(−ckj + cjk ) + Re(ajj − akk ) = Im(−bkj + bjk ) − Re(ckj + cjk ). Here we have made use of the corresponding relations in (i) and (ii). The proof would be complete, if we could show that Im(−bkj + bjk ) = Im(ajj − akk ) = − Re(ckj + cjk ) . But without any further specification of the Im ajj this need not be true. Hence, to finish the proof, let us consider the difference S = T − LA , which

82

3 Linear mappings on ordered vector spaces

also satisfies (3.15). Setting sjk = Im(−bkj + bjk − ajj + akk ) we know from the cases (i), (ii), and (iii) that i S(Xjk ) = sjk Xjk

and

i S(Xjk ) = sjk Xjk .

(3.28)

Since j and k were arbitrary we conclude that for all j < k there exist real numbers sjk , such that (3.28) holds. By the construction of A we have S(Xjj ) = 0 for all j = 1, . . . , n. As noted in (3.23) we can assume that Im(ajj − ann ) = Im(−bnj + bjn ) for 1 ≤ j < n, i.e. s1n = s2n = . . . = sn−1,n = 0 . Now let j < k < n be fixed again and set x = ej + iek + en and y = ej + ek − (1 − i)en , such that x∗ y = 0. Taking into account that S(Xjj ) = S(Xkk ) = S(Xnn ) = 0 and sjn = skn = 0 we find = D 0 = y ∗ S(xx∗ )y = y ∗ sjk Xjk y = 2sjk . Hence S = 0 and therefore T = LA .

"

We conclude this section with a simple observation. Proposition 3.4.4 Let A ∈ Kn×n . −1 (i) If ρ(A) < 1, then −SA is completely positive. (ii) If β(A) < 0, then −L−1 A is completely positive.

Proof: (i) Set P (X) = A∗ XA. If ρ(A) < 1 then ρ(P ) < 1 and hence −1 −SA =

∞ O

Pk .

k=0

This is a convergent series of completely positive operators. Since by Corollary 3.3.12 the set of completely positive is closed −SA is completely positive. (ii) Write −LA = R− − R+ with R± (X) =

1 (A ± I)X(A ± I)∗ . 2

−1 R− ) < 1 if σ(A) ⊂ C− . One readily verifies that R− is invertible and ρ(R+ By (i) −1 −1 −1 −1 −1 ◦ R− = S(A+I)(A−I) −L−1 −1 ◦ R− A = (I − R− R+ )

is a composition of completely positive operators and thus completely positive. " The complete positivity of −L−1 A can also be obtained from the following well-known result (e.g. [22, 113]), which we cite for completeness.

3.5 Linear equations with resolvent positive operators

83

Theorem 3.4.5 Let A ∈ Kn×n , such that σ(A) ⊂ C− . Then the inverse of the Lyapunov operator LA is given by < ∞ ∗ −1 LA (Y ) = − etA Y etA dt . 0

Other integral representations for L−1 A can be found e.g. in [100, 83].

3.5 Linear equations with resolvent positive operators One important problem we are confronted with, is to solve linear equations X, Y ∈ Hn×n ,

T (X) = Y

(3.29)

where T : Kn×n → Kn×n is Hermitian-preserving and resolvent positive. The difficulty lies in the fact that the complexity increases rapidly with n. 3.5.1 Direct solution 2 If we merely regard (3.29) as a linear equation in Kn×n ∼ = Kn then a direct solution requires O(n6 ) operations. Obviously, we can decrease the number of steps by a certain factor, if we make use of the fact that X and Y are Hermitian. This means that the lower triangle of these matrices does not contain any information independent of the upper triangle. Moreover, in the real case Hn is a proper T -invariant subspace of Rn×n with an orthogonal basis B given in (3.16). Assume that T is given in the form (3.6). We denote the restriction of T to Hn by T |Hn . The matrix representation of T |Hn with respect to the basis B can be obtained easily from the sum the Kronecker product representation

M =

N O

εj Vj ⊗ Vj ;

j=1

we just have to apply the transformation M L→ U T M U with U =

n : n : j=1 k=j

1 (ej ⊗ ek + ek ⊗ ej ) . <ej + ek <

(3.30)

Here, for matrices E1 and E2 with the same number of rows, E1 ⊕ E2 denotes the block matrix [E1 , E2 ]. By this definition, U has orthonormal columns, and it is easily verified that R R 2 {U x R x ∈ Rn } = {vec X R X ∈ Hn } .

84

3 Linear mappings on ordered vector spaces

Since each row and each column of U has only at most two non-vanishing entries, the transformation M L→ U T M U can be performed in O(n4 ) steps, which does not affect the overall complexity of a direct solution. We have applied the same transformation in Example 1.7.5. In the complex case a similar reduction is possible, although Hn is not a subspace of the complex vector space Cn×n . If we represent the operator T and the matrices X and Y in the basis B ∪ Bi given in (3.16) and (3.22), then (3.29) becomes an n2 -dimensional real linear equation. Note that a general 2 linear equation T (X) = Y with arbitrary X, Y ∈ Cn corresponds to a 2n2 dimensional real equation. We omit the explicit construction of this transformation, since it is rather tedious and does not produce any further insight. Similar reductions for the real case have been discussed e.g. in [200]. In our examples we only need the real case. 3.5.2 The case of simultaneous triangularizability The situation is much better in the case, when T has a representation of the form (3.6), where all Vj are simultaneously triangularizable. This means there exists a nonsingular matrix Q, such that Q−1 Vj Q = Rj for all j = 1, . . . , N , where the Rj are upper triangular. In this case, by a Gram-Schmidt orthonormalization of the columns of Q, we can always assume Q to be unitary, such that Q−1 = Q∗ . The linear equation (3.29) is then equivalent to N O

Rj W Rj∗ = Z

with

Z = Q∗ Y Q and X = QW Q∗ .

(3.31)

j=1 (j)

(j)

If we denote the entries of Uj by ukL and take into account that ukL = 0 for k > ^ then we obtain the entrywise equations zkL =

N O j=1

=

N O j=1

εj

n O n O s=1 t=1

(j)

(j)

rkt wts r¯Ls

N n D O O (j) (j) wkL + εj εj rkk r¯ss

=

j=1

n O

s=L+1 t=k+1

(j)

(j)

rkt wts r¯Ls .

Hence we can compute the wkL recursively in O(N n4 ) operations from   N n n O O O 1 (j) (j) zkL − wkL = PN rkt r¯Ls wts  (3.32) εj (j) (j) ε r r ¯ ss j j=1 s=L+1 t=k+1 kk j=1 e.g. for the index pair (k, ^) increasing in lexicographic order. If X and Y are Hermitian, then it suffices to compute wkL for k ≥ ^. The equation is uniquely and universally solvable if and only if

3.5 Linear equations with resolvent positive operators

∀k, s = 1, . . . , n :

λks =

N O j=1

(j)

(j) O= 0 . εj rkk r¯ss

85

(3.33)

From this it is easy to see that the λks are the eigenvalues of T . Of course, one can derive this result also with the help of the Kronecker product, taking into account that R1 ⊗ R2 is upper-triangular if R1 and R2 are. In the special case, where T = LA is a Lyapunov operator as defined in (3.10) we can do even better. Let us assume that A is already upper triangular. Then – in analogy to (3.32) – the solution to A∗ X + XA = Y can be computed recursively from   n n O O 1 yjk − a ¯Lj xLk + xjL aLk  . xjk = a ¯jj + akk L=j+1

L=k+1

This system can be resolved in O(n3 ) operations. Efficient algorithms have been developed in [11, 85, 95, 161]; see also [77] for an overview. The case, where T = SA is a Stein operator can be transformed to the previous case. We assume that 1 O∈ σ(A) and set B = (A − I)(A + I)−1 . Then the equation T (X) = A∗ XA − X = Y is equivalent to 1 ∗ (B X + XB) = (A∗ + I)Y (A + I) . 2 Remark 3.5.1 It is well-known (e.g. [78]) that a family of matrices is simultaneously triangularizable, if the members of this family are pairwise commutative. More generally, a family of matrices is simultaneously triangularizable, if and only if they generate a solvable Lie algebra (see [172]). In [203, 202] stochastic linear systems have been considered, whose coefficient matrices generate a solvable Lie algebra. This situation occurs e.g. in the example of the electrically suspended gyro in Section 1.9.7, where we had Aj0 = I for all j. From our point of view, however, the case of simultaneous triangularizability is degenerate. In general, it is, for instance, not reasonable to assume that the parameter matrices of the nominal system and the perturbed system commute. But, of course, we will make ample use of the fact that Lyapunov and Stein equations can be solved efficiently. The results of the present subsection can e.g. be found in [113]. For further material on simultaneous triangularization we refer to the monograph [165]. 3.5.3 Low-rank perturbations of Lyapunov equations In [169, 33] the case has been considered, where T is a low-rank perturbation of a Lyapunov operator. Below we will discuss how this case arises in our context. If a rank revealing factorization of the perturbation is available, one can reduce the computational effort significantly. Since the idea is not limited

86

3 Linear mappings on ordered vector spaces

to the situation of matrix equations, we describe it in more general terms. For the moment, regard T as an endomorphism of Kp , p ∈ N and consider a linear equation Tx = y .

(3.34)

Assume that T can be decomposed as T = L + P , where L is nonsingular, L−1 is known or easy to compute, and r := rk P < p. Assume further that we can factorize P as P = P1 P2 with linear operators P1 : Kr → Kp and P2 : Kp → Kr . Then the following holds (compare [169, Thm 2.2]. Lemma 3.5.2 Equation (3.34) is uniquely solvable if and only if I +P2 L−1 P1 is nonsingular. If w solves the equation (Ir + P2 L−1 P1 )w = P2 L−1 y ,

(3.35)

then x = L−1 (y − P1 w) solves (3.34). Proof: With our assumptions, equation (3.34) is equivalent to (Ip + L−1 P1 P2 )x = L−1 y . It is well-known (e.g. [186]) that σ(P2 L−1 P1 ) \ {0} = σ(L−1 P1 P2 ) \ {0} .

(3.36)

Hence T is nonsingular if and only if Ir + P2 L−1 P1 is. If w solves (3.35) and x = L−1 (y − P1 w), then T x = (L + P1 P2 )L−1 (y − P1 w) = y − P1 w + P1 (P2 L−1 y − P2 L−1 P1 w) = y − P1 w + P1 w = y , i.e. x solves (3.34).

"

Since (3.35) is an r-dimensional linear equation, it can be solved in O(r3 ) operations. If r is much smaller than p, and if the effort to invert L is small, then it is advantageous to compute x from the solution of (3.35). This way of representing the solution x is also known as the Sherman-Morrison-Woodbury formula, e.g. [86]. In the context of equation (3.29) the situation arises as follows. The operator T : Kn×n → Kn×n can be written as T = LA + P with a Lyapunov operator LA and a perturbation operator P . The inversion of LA is possible in O(n3 ) operations. If r = rk P , then the overall effort to solve (3.34) reduces to max{O(rn3 ), O(r3 )} operations. We give an example, where the rank of P is small. Example 3.5.3 Consider the stochastic control system (1.55) in Section 1.9.4. Let u = 0 and v = 0. By Theorem 1.5.3 the unperturbed and uncontrolled system is mean-square stable if and only if the equation

3.5 Linear equations with resolvent positive operators

87

T (X) = A∗ X + XA + A∗0 XA0 = I has a negative definite solution X. Here    0 0 0 0 1 0 0 0  0 0 0 1     A= and A0 =  k1 /m1 −c/m1 c/m1  0 0 −k1 /m1 k1 /m2 −(k1 + k)/m2 −c/m2 c/m2 0 −σ

0 0 0 0

 0 0  . 0 0

If σ O= 0, then the operator P : X L→ A∗0 XA0 has rank 1, because A0 ⊗ A0 has only one non-vanishing entry. Similar situations arise, whenever one considers linear control systems, where only a few parameters are subject to multiplicative stochastic perturbations. This, however, is not the case for the perturbation ma(1) trix A0 in Section 1.9.5, which has full rank; hence also the mapping (1)∗ (1) X L→ A0 XA0 (viewed as an endomorphism of Kn×n ) has full rank. j j 2 In general, the Kn×n -endomorphism X L→ Aj∗ 0 XA0 has rank rj if rk A0 = rj . PN j Therefore the rank of the mapping P : X L→ j=1 Aj∗ 0 XA0 is bounded above PN 2 by j=1 rj . To obtain a rank-revealing factorization of P = P1 P2 one can j j j n×rj exploit factorizations Aj∗ and Aj2 ∈ Krj ×n . Then 0 = A1 A2 where A1 ∈ K P is represented by the matrix N O

j∗ A¯j∗ 0 ⊗ A0 =

j=1

N = O

A¯j1 ⊗ Aj1

D= D A¯j2 ⊗ Aj2 .

(3.37)

j=1

Hence, the columns of P1 are linear combinations of the columns of the matrices A¯j1 ⊗ Aj1 and the rows of P2 are linear combinations of the rows of the matrices A¯j2 ⊗ Aj2 . To make these ideas precise, we formulate them in algorithmic form. We restrict ourselves to the relevant case, where T is the sum of a Lyapunov operator and a completely positive operator, i.e. ∀X ∈ Kn×n :

T (X) = LA (X) + ΠA0 (X) .

(3.38)

Remember from (1.20) that LA (X) = A∗ X + XA

and

ΠA0 (X) =

N O

j Aj∗ 0 XA0

j=1

with A, Aj0 ∈ Kn×n . An algorithm For fixed n, we regard the vec-operator as a vector-space isomorphism vec : 2 2 Kn×n → Kn and denote its inverse by vec−1 : Kn → Kn×n . The Kroneckerproduct representation of an operator T : Kn×n → Kn×n is denoted by

88

3 Linear mappings on ordered vector spaces

kron T = vec ◦T ◦ vec−1 . This choice of notation is illustrated by the diagram T

Kn×n −→ Kn×n −1 vec ↑ ↓ vec Kn

2

kron T

−→ Kn

2

n×n Proposition 3.5.4 For A, A10 , . . . , AN , with σ(A) ∩ σ(−A∗ ) = ∅ and 0 ∈K n Y ∈ H , the following algorithm determines σ(L−1 A ΠA0 )\{0}. Π ), then the unique solution X ∈ Hn of (LA + ΠA0 )(X) = Y If 1 O∈ σ(L−1 A 0 A is computed.

1. 2.

For j = 1, . . . , N factorize Aj0 = Aj2 Aj∗ 1 , where Aj1 , Aj2 ∈ Kn×rj and rj = rk Aj0 . N 1 1 N N n2 ×r0 Set A1 = [A11 ⊗A11 , . . . , AN , 1 ⊗A1 ], A2 = [A2 ⊗A2 , . . . , A2 ⊗A2 ] ∈ K PN 2 r0 = j=1 rj . 2

2

Compute a matrix Q = [Q1 , Q2 ] ∈ Kn ×rq , Q1 ∈ Kn ×rq1 , rq1 ≤ r0 , rq ≤ 2r0 , with orthonormal columns, such that im A1 = im Q1 and im[A1 , A2 ] = im Q. 4. Compute H = Q∗1 kron(Π)Q ∈ Krq1 ×rq . 0 60 ∗6 Σ0 V1 5. Perform a singular value decomposition H = [U1 , U2 ] = 0 0 V2∗ U1 ΣV1∗ with unitary matrices U = [U1 , U2 ], V = [V1 , V2 ] and diagonal r , r = rk Σ = rk Π. Σ ∈ int H+ 6. Set P1 = (p1 , . . . , pr ) = Q1 U1 Σ and P2 = V1∗ Q∗ . −1 7. For k = 1, . . . , r compute mk = P2 vec(LA (vec−1 (pk ))) and set M = (m1 , . . . , mr ). 8. σ(L−1 A ΠA0 ) \ {0} = σ(M )\{0}. If 1 ∈ σ(M ), stop. −1 (Y )) and w = (I − M )−1 y. 9. Compute y = P2 vec(LA −1 −1 10. X = LA (Y − vec (P1 w)). P j j j j ∗ Proof: By (3.37) we have kron(Π) = N j=1 (A1 ⊗ A1 )(A2 ⊗ A2 ) . ⊥ Hence im kron Π ⊂ im A1 = im Q1 and (Ker kron(Π)) ⊂ im A2 ⊂ im Q. If Q3 completes [Q1 , Q2 ] to a unitary matrix, then   H1 H 2 0 kron(Π)[Q1 , Q2 , Q3 ] = [Q1 , Q2 , Q3 ]  0 0 0  , 0 0 0 3.

with [H1 , H2 ] = Q∗1 kron(Π)[Q1 , Q2 ] = U1 ΣV1∗ . Hence kron Π = Q1 U1 ΣV1∗ [Q1 , Q2 ]∗ = P1 P2 . Since M = P2 kron(L−1 A )P1 , the assertions follow from Lemma 3.5.2 and equation (3.36). "

3.5 Linear equations with resolvent positive operators

89

Remark 3.5.5 The complexity of this algorithm depends mainly on the numbers n, N , r0 and r = rk Π. To see this, it is important to note that, based on the factorizations Aj0 = Aj1 Aj2 , the operator Π can be evaluated in O((r1 + . . . + rN )n2 ) ≤ O(r0 n2 ) operations. Indeed, each summand j j∗ j∗ j j 2 Aj∗ 0 XA0 = (A2 (A1 XA1 )A2 ) requires only O(rj n ) scalar multiplications. In particular, it follows that the matrix H in step 4.) can be computed in O(rq1 n2 rq ) ≤ O(2r02 n2 ) operations. The computational cost for the standard numerical procedures in each step can be estimated as follows (e.g. [54]). 4. 5. 6. 7. 8. 9. 10. Step 1. 2. 3. O N n3 n2 r0 r02 n2 rq1 n2 rq rq31 nrq r rn3 r3 n3 n3 Since rq ≤ r, the algorithm requires not more than O(N n3 + r02 n2 + rn3 ) operations. For small N and r0 , this is much faster than the O(n6 ) operations of a naive solution. 3.5.4 Iterative solution with different splittings As the dimension increases, it is sometimes useful to consider iterative methods. Again let T be given in the form (3.38) as the sum of a Lyapunov operator LA and a completely positive operator ΠA0 , and assume that T is stable. From Theorem 3.2.10 we can immediately derive an iterative strategy to solve (3.29), such that in each step we only have to solve a Lyapunov equation. Proposition 3.5.6 Let T be given by (3.38) and assume σ(T ) ⊂ C− . For an arbitrary X0 ∈ Hn define the sequence Xk+1 = L−1 A (Y − ΠA0 (Xk )) .

(3.39)

Then Xk converges to the unique solution of (3.29). Proof: By Theorem 3.2.10 we have ρ(L−1 A ΠA0 ) < 1, such that the result follows from Banach’s fixed point theorem. " The given splitting T = LA +ΠA0 , however, is only one possible representation of T which leads to an iterative scheme to solve T (X) = Y . Other splittings are advantageous, in particular to avoid the explicit solution of a Lyapunov equation in each step. To construct such splittings, we note that T has the form T (X) = λ0 V0∗ XV0 + λ1 V1∗ XV1 + λ2 V2∗ XV2 + . . . + λν Vν∗ XVν , (3.40) with numbers λ0 < 0, λ1 , . . . , λν > 0 and given matrices Vj . For instance, we may simply write LA (X) = −A∗− XA− + A∗+ XA+ ,

90

3 Linear mappings on ordered vector spaces

where 1 A− = √ (A − αI) , 2α

1 A+ = √ (A + αI) , 2α

α>0.

(3.41)

Thus LA + ΠA0 has the form (3.40). Or we can apply the following lemma. Lemma 3.5.7 If T is given by (3.38) and σ(T ) ⊂ C− , then the n2 ×n2 -matrix X = (Inn ⊗ T )(E (nn) ) has exactly one negative eigenvalue. Proof: Let A+ and A− be defined by (3.41). By Theorem 3.3.7 and by (3.11) we can write X = vec A+ (vec A+ )∗ − vec A− (vec A− )∗ +

N O

vec Aj0 (vec Aj0 )∗ .

j=1

Hence, by Sylvester’s inertia theorem, X has at most one negative eigenvalue. If all eigenvalues were nonnegative then T would be completely positive. But, since T is stable, we know by Theorem 3.2.10 that ∃X > 0 : T (X) < 0 .

(3.42) "

Hence X has at least one negative eigenvalue.

Theorem 3.3.7 therefore yields a representation of the form (3.40), where the λ0 < 0 < λ1 ≤ . . . ≤ λν are the non-zero eigenvalues of X . Now we multiply the equation T (X) = Y with V0−∗ from the left and V0−1 from the right, and devide by |λ0 |, such that T˜(X) = −X +

ν O

V˜j∗ X V˜j = Y˜

(3.43)

j=1

K

and Y˜ = |λ10 | V0−∗ Y V0−1 . The operator T˜ has the obvious regular splittings T˜ = R + P or T˜ = Rk + Pk with

with V˜j =

λj −1 |λ0 | Vj V0

R(X) = −X ,

P (X) =

Rk (X) = −X + V˜k∗ X V˜k ,

Pk (X) =

ν O

V˜j∗ X V˜j

j=1 ν O

V˜j∗ X V˜j .

j=1,j0=k

Here we recognize Rk as a Stein operator, while −R is just the identity operator. Since T is stable so is T˜ , such that by Theorem 3.2.10 these splittings in fact are convergent. From our experience, we favour the first splitting T = R + P , which leads to the particularly simple iteration scheme Xk+1 = P (Xk ) − Y˜ .

(3.44)

3.5 Linear equations with resolvent positive operators

91

Note that this scheme does not require the solution of any standard Lyapunov or Stein equations. Nevertheless, one also might take advantage of the different splittings T = Rk + Pk with the Stein operators Rk . All the splittings discussed in this subsection lead to an iteration scheme Xk+1 = Π(Xk ) − Y˜ ,

(3.45)

where Π is a positive operator with ρ(Π) < 1. Unfortunately, in the relevant applications ρ(Π) often is approximately equal to 1. This, for instance, happens in the context of γ-subotimal H ∞ -problems, when γ is close to the optimal attenuation parameter. Hence, without any additional accelerations, the scheme (3.45) will turn out to be impracticable. In the following subsection we discuss a simple extrapolation method to overcome this problem in many situations. The basic idea can be found in [139, Section 15.4], where it is referred to as ’L.A. Ljusterniks approach for improving convergence’. 3.5.5 Ljusternik acceleration Consider the iteration scheme (3.45), where Π : Hn → Hn is positive, ρ(Π) < 1 and Y˜ < 0. By Theorem 3.2.3 of Krein and Rutman, we have ρ(Π) ∈ σ(Π), and there exists a nonzero matrix Xρ ≥ 0, such that Π(Xρ ) = ρ(Π)Xρ . We assume now, that for all eigenvalues of Π, different from ρ(Π) are strictly smaller than ρ(Π) in the sense that ∃κ < 1 : ∀λ ∈ σ(Π) \ {ρ(Π)} : |λ| ≤ κρ(Π)} .

(3.46)

In [139] the minimal number κ = κ(Π) satisfying this condition is called the spectral margin of Π. Let X0 = 0 and X∗ > 0 denote the solution to −X + Π(X) = Y˜ . A short computation shows that for all k ≥ 1 Xk+1 − Xk = Π(Xk − Xk−1 ) , Xk+1 − X∗ = Π(Xk − X∗ ) . Hence the iteration amounts to an application of the power method to X1 −X0 and X0 − X∗ . It is well-known that under the given assumptions on σ(Π) the Xk+1 − Xk and the Xk+1 − X∗ converge to eigenvectors of Π, corresponding to the eigenvalue ρ(Π). Hence, after a number of iterations the approximate relations Xk+1 − Xk ≈ ρ(Π)(Xk − Xk−1 ) , Xk+1 − X∗ ≈ ρ(Π)(Xk − X∗ )

(3.47) (3.48)

will hold. This may be detected by measuring the angle between Xk+1 − Xk and Xk − Xk−1 . We now apply an appropriate functional to both sides of (3.47) in order to approximate ρ(Π):

92

3 Linear mappings on ordered vector spaces

t1 = trace ((Xk+1 − Xk )V ) ,

t0 = trace ((Xk − Xk−1 )V ) ,

ρ(Π) ≈

t1 . t0

In principle, it suffices to choose an arbitrary positive definite matrix V > 0, e.g. V = I to guarantee that t1 , t0 O= 0. It turns out, however, that not all V > 0 lead to good results here. Experiments confirm that it makes sense to choose V as an approximation to the positive eigenvector of Π ∗ , provided this eigenvector is positive definite. Using the computed numbers t0 and t1 , we can find an approximation to X∗ from (3.48) as X∗ ≈

t1 Xk − t0 Xk+1 =: Xk+2 . t1 − t0

The iteration proceeds further with the scheme (3.45), until again for some ^ > k the angle between XL+1 −XL and XL −XL−1 is small enough and another acceleration step can be taken. The method has been tried on random examples, i.e. for operators Π(X) = PN ∗ j=1 Vj XVj , where the Vj are full random matrices with normally distributed entries, and ρ(Π) = 1. For such examples, assumption (3.46) holds generically, and therefore the results are quite convincing. On a logarithmic scale we have depicted the relative residual errors for the fixed point iteration with and without acceleration step. Moreover, we see the angles αk between two subsequent increments Xk+1 − Xk and Xk − Xk−1 . When this angle is small, an acceleration step is applied. n=450, N=45

logarithmic scale

0

standard Ljusternik angles

−5

−10

−15

0

2

4

2k

6

8

10

Since ρ(Π) = 1, the standard iteration becomes almost stationary, as the angles between the increments converge to zero. We observe two effects when the acceleration step is applied. First, the relative residual error decreases rapidly from about 10−1 to 10−8 in one step. This is – in principle – what we expected to achieve. Secondly, and maybe even more surprisingly, the speed

3.5 Linear equations with resolvent positive operators

93

of convergence is increased significantly in the subsequent steps. This effect can be explained by noting that the error Xk+2 − X∗ (after the acceleration step) will be close to the invariant subspace of Π, which is complementary to the leading eigenvector. On this subspace, the convergence is determined by the second largest eigenvalue λ2 , which (generically for random examples) is much smaller than ρ(Π). So, the speed of convergence of this method depends on two factors, namely the speed of convergence of the angles αk and the modulus of λ2 . It is wellknown (e.g. [139, Thm. 15.4]) that the convergence of the αk is determined by the spectral margin κ(Π) = |λ2 |/ρ(Π) (≈ |λ2 |, if ρ(Π) ≈ 1). We conclude that the method works best, if ρ(Π) and |λ2 | are well seperated, i.e. if κ(Π) is small. Conversely, the method may fail, if there are other eigenvalues of nearly or exactly the same size as ρ(Π). If there is an eigenvalue λ2 O= ρ(Π) with |λ2 | = ρ(Π), then the αk and thus the directions of the Xk − Xk−1 will not converge. If the multiplicity of the eigenvalue ρ(Π) is greater than 1, or if there are eigenvalues λ2 O= ρ(Π) with |λ2 | ≈ ρ(Π), then the αk converge very slowly. In some of these cases, we may take advantage of a shifted iteration Xk+1 =

1 (Π(Xk ) + µXk − Y ) 1+µ

with some µ > 0. Obviously this iteration has the same fixed point X∗ , but 1 (Π + µI). The eigenvalues uses the shifted positive linear operator Πµ = 1+µ

of Πµ are λµ = small µ

λ+µ 1+µ ,

where λ ∈ σ(Π). If |λ − ρ/2| > ρ/2, then for sufficiently |λµ | |λ| |λ + µ| < . = ρµ ρ+µ ρ

Hence, if ρ(Π) is a simple eigenvalue R and the second largest eigenvalue λ2 is not contained in the circle {λ ∈ C R |λ − ρ/2| ≤ ρ/2}, then for appropriate µ > 0 the spectral margin of Πµ will be smaller than that of Π. The concrete choice of the shift parameter µ depends on our knowledge on the spectrum of Π. If |λ| = ρ(Π) for all λ ∈ σ(Π), then it is easy to show that the optimal choice is µ = ρ(Π). Usually we would choose e.g. µ = ρ(Π)/2 or µ = ρ(Π)/3. Obviously, the shift will not help, if there are positive real eigenvalues very close to ρ(Π). 3.5.6 Conclusions Basically, we have identified two cases, when an efficient solution of the equation (LA + ΠA0 )(X) = Y for large n is possible. Namely, the case, when PN j 2 j=1 rk A0 K n and the case, when the spectral margin of Π in (3.45) is small.

94

3 Linear mappings on ordered vector spaces

In the first case, a direct solution can be found relatively cheaply, while in the second an accelerated iterative method could be superior. We just mention that for large linear systems there are modern powerful iterative methods, to be found e.g. in [88]. These methods are particularly efficient, if a certain sparsity structure is given. For standard Lyapunov equations such methods have been applied in [110], but for generalized Lyapunov we are not aware of any particular results in this direction.

3.6 Recapitulation: Resolvent positivity, stability and detectability The analogy between Theorem 1.5.3 and Theorem 3.2.10 is striking. Since these results form the basis of our investigations we state them again in a unified and extended version. (1)

(N )

Theorem 3.6.1 Let A, A0 , . . . , A0 ∈ Kn×n and define LA and ΠA0 according to (1.20). With these data consider the homogeneous linear stochastic differential equation (1.13). The following are equivalent: (a) (b) (c) (d) (e) (f ) (g) (h) (i) (j) (k) (l) (m)

System (1.13) is asymptotically mean-square stable. System (1.13) is exponentially mean-square stable. σ (LA + ΠA0 ) ⊂ C− . max σ(LA + ΠA0 ) ∩ R < 0. σ (LA ) ⊂ C− and ∀τ? ∈ [0, 1] :Fdet (LA + τ ΠA0 ) O= 0. σ (LA ) ⊂ C− and ρ L−1 A ΠA0 < 1. ∀Y < 0 : ∃X > 0 : LA (X) + ΠA0 (X) = Y . ∀Y ≤ 0 with (A, Y ) observable: ∃X > 0 : LA (X) + ΠA0 (X) = Y . ∃X > 0 : LA (X) + ΠA0 (X) < 0. ∃X ≥ 0 : LA (X) + ΠA0 (X) < 0. ∃Y ≤ 0 with (A, Y ) observable: ∃X ≥ 0 : LA (X) + ΠA0 (X) ≤ Y . ∃Y ≤ 0 with (A, (Aj0 ), Y ) β-detectable: ∃X ≥ 0 : LA (X) + ΠA0 (X) ≤ Y . −1 − (LA + ΠA0 ) exists and is completely positive.

In a sense this theorem parallels [14, Theorem 2.3], which provides even fifty equivalent conditions for a Z-matrix to be an M -matrix; see also [67, 199, 155]. One might doubtlessly find further equivalent conditions for the operator LA + ΠA0 to be stable, but we restrict us now to the verification of (d), (e), (h), (k), and (m) which were not explicitly part of the Theorems 1.5.3 and 3.2.10. Proof: ‘(c) ⇐⇒ (d) ⇐⇒ (e)’: Since LA + ΠA0 is resolvent positive, we have β(LA + ΠA0 ) = max σ(LA + ΠA0 ) ∩ R by Theorem 3.2.3. Hence (c) ⇐⇒ (d). Moreover β (LA + τ ΠA0 ) increases monotonically with τ by Corollary 3.2.4. Hence β(LA + ΠA0 ) < 0

⇐⇒

∀τ ∈ [0, 1] : β (LA + τ ΠA0 ) < 0

3.6 Recapitulation: Resolvent positivity, stability and detectability

95

and hence (e). Conversely assume that (e) holds but β(LA + ΠA0 ) > 0. Then for some τ ∈]0, 1[ we have β(LA + τ ΠA0 ) = 0 and hence det(LA + τ ΠA0 ) = 0. This contradicts our assumption. ‘(h)⇒(k)’ is trivial. ‘(k)⇒(e)’ (k) implies A∗ X + XA ≤ Y , and it is well-known (e.g. [113, Thm. 2.4.7]) that this together with the observability of (A, Y ) implies X > 0 and σ(A) ⊂ C− . Suppose LA (X0 ) + τ Π(X0 ) = 0 for some τ ∈ [0, 1] and consider the convex combination Xα := αX + (1 − α)X0 . By our assumptions LA (Xα ) ≥ αY − (1 − α)τ Π(X).

(3.49)

We want to show R X0 ≤ 0 and X0 ≥ 0. Assume first X0 O≥ 0 and set α0 = max{α ∈ [ 0, 1] R Xα O> 0}. Since X1 = X > 0 we have Xα0 ≥ 0, such that α0 > 0 and LA (Xα0 ) ≤ α0 Y . This, however, implies Xα0 > 0, in contradiction to the choice of α0 . Now assume X0 O≤ 0, then the last argument can be repeated with X replaced by −X < 0 and inverted order. Thus Ker (LA + τ Π) = {0}. ‘(g)⇒(h)’ By continuity, the assertion (h) holds with X ≥ 0. But as above, one proves X > 0. ‘(f)⇒(m)’: If σ(LA ) ⊂ C− then −L−1 A is completely positive by Proposition 3.4.4. By definition ΠA0 is completely positive. Since ρ(L−1 A ΠA0 ) < 1 we can write −1 −1 LA = −(LA + ΠA0 )−1 = −(I + L−1 A ΠA 0 )

∞ O

−1 k (−L−1 A ΠA0 ) (−LA ) .

k=0

This is a convergent series of completely positive operators and hence by Corollary 3.3.12 completely positive. −1 n ‘(m)⇒(i)’: As a nonsingular positive operator, − (LA + ΠA0 ) maps int H+ to itself. "

Remark 3.6.2 Some of these assertions are taken from [176, 45, 48]. The implication ‘(l)⇒(c)’ can be found in [69] (with a lengthy proof covering about three pages). Theorem 3.6.1 transforms the stochastic mean-square stability problem for n-dimensional systems to various n2 -dimensional algebraic problems. It was already already asked in [132] whether one can reduce the dimension of the algebraic problem. In [212] the following sufficient stability-criterion has been used. Corollary 3.6.3 If σ(A) ⊂ C− and Q< Q Q Q ∞ tA∗ Q Q −1 tA QL (ΠA0 (I))Q = Q e ΠA0 (I)e dtQ A Q Q < 1, 0

(3.50)

96

3 Linear mappings on ordered vector spaces

then σ(LA + ΠA0 ) ⊂ C− . This condition roughly means that the noise effect on the system is not too n large. It implies that for all X ∈ H+ (not only for the eigenvectors of L−1 A ΠA 0 ) we have −1
whence, in particular ρ(L−1 A ΠA0 ) < 1. Remark 3.6.4 As we have seen in the previous section, the inversion of the Lyapunov operator LA basically amounts to a Schur-decomposition of A and the solution of a triangular n2 -dimensional linear equation, while the evaluation of ΠA0 (I) requires the computation of N products of the j 3 form Aj∗ 0 A0 . Hence, for small N condition (3.50) can be checked in O(n ) operations. Lower dimensional necessary and sufficient stability-criteria apparently can only be obtained in special cases like those considered in the previous section. For the special case, where all A, Aj0 are simultaneously triangularizable we have already mentioned that the eigenvalues of LA + ΠA0 can be read from (3.33) and hence stability can easily be checked. Moreover, classical inertia results [157, 206] can be generalized in this case (cf. [180, 24]). It is interesting to note that the case of simultaneous triangularizability is only slightly more general than the case, when all A, Aj0 commute; under this assumption the solution of (1.13) can be written in the explicit form of Proposition 1.4.2. It appears that in some sense the simpler form of the solution corresponds to simple stability criteria. For the even more special case, when all Aj0 are polynomial expressions in A, some assertions of Theorem 3.6.1 have been generalized e.g. in [128]; a survey of these results is given in [50]. The special form of ΠA0 plays a role only in the assertions (a), (b) and (m) of the above theorem. The equivalence of the assertions (c)–(l) remains if we replace ΠA0 by an arbitrary positive operator Π. But we do not have any application for such operators in stability analysis. The question arises, whether we can benefit from the additional knowledge that ΠA0 is completely positive, when we analyze the stability of (1.13). We have already made use of this property in the previous section, where we developed an iterative procedure to solve the equation LA (X) + ΠA0 (X) = Y . Clearly, the solution of this equation also gives an answer to the stability problem: If we set Y = I, for instance, then it suffices to check whether the corresponding solution X is negative definite or not. Hence, all the results in the previous section can also be viewed as means to analyze the stability of

3.6 Recapitulation: Resolvent positivity, stability and detectability

97

system (1.13). We mention another simple result that exploits the complete positivity of ΠA0 . Consider property (e) and set d(τ ) = det (LA + τ ΠA0 ). Then τ0 = min{τ > R 0 R d(τ ) = 0} is the threshold, where LA + τ ΠA0 becomes unstable (e.g. [176]). In [45] we have shown that d is a monotonic function on [0, τ0 ]. Proposition 3.6.5 The determinant d(τ ) is monotonic on [0, τ0 [. Proof: As Tτ := LA + τ ΠA0 is nonsingular on [ 0, τ0 [, Liouville’s formula (e.g. [8]) yields ? F ˙ ) = trace ΠA T −1 d(τ ) . d(τ 0 τ Since −ΠA0 Tτ−1 is F positive by Theorem 3.6.1 (j), Corollary 3.3.9 ? completely " yields that trace ΠA0 Tτ−1 ≤ 0. So d(τ ) is monotonic on [ 0, τ0 ]. Finally, as promised before, we give a proof of Lemma 1.8.3, which, for convenience, we restate here. Lemma 3.6.6 Consider the stochastic system dx = Ax dt +

N O

(j)

A0 x dw,

y = Cx ,

j=1 (j)

and denote the solutions by x(t, x0 ). The triple (A, (A0 ), C) is β-detectable, if and only if for arbitrary x0 the implication ∀t ≥ 0 : E
⇒

lim E<x(t, x0 )<2 = 0

t→∞

(3.51)

holds. Proof: Recall from Theorem 1.4.3 that D d= Ex(t, x0 )x(t, x0 )∗ = (LA + ΠA0 )∗ (Ex(t, x0 )x(t, x0 )∗ ) . dt (j)

Let us first assume that (A, (A0 ), C) is not β-detectable, i.e. there exists a non-zero X0 ≥ 0, such that CX0 = 0 and (LA + ΠA0 )∗ (X0 ) = λX0 with P (j) (j) λ ≥ 0. Consider a decomposition X0 = nj=1 X0 in rank-1 matrices X0 = (j) (j)∗

(j)

x0 x0 with x0 ∈ Kn . If, in general, X(t, Y0 ) denotes the solution of the matrix differential equation X˙ = (LA + ΠA0 )∗ (X) , starting at Y0 , then, by construction, O O (j) (j) (j) eλt X0 = X(t, X0 ) = X(t, X0 ) = Ex(t, x0 )x(t, x0 )∗ .

(3.52)

98

3 Linear mappings on ordered vector spaces (j)

(j)

Since λ ≥ 0, at least one of the summands Ex(t, x0 )x(t, x0 )∗ must not (j) converge to zero for t → ∞, i.e. limt→∞ E<x(t, x0 )<2 O= 0. On the other hand, (j) (j) ∗ since X(t, D x0 ) and CX(t, X0 ) = 0 for all t, it follows = X0 ) ≥ Ex(t, x0 )x(t, (j)

(j)

that E Cx(t, x0 )x(t, x0 )∗

(j)

(j)

= 0, i.e. E
Hence (3.51) does not hold. (j) Let us now assume that (A, (A0 ), C) is β-detectable, and there exists a nonzero solution x(t, x0 ) with E
as t → ∞ .

(3.53)

The second-moment matrix X(t) = Ex(t, x0 )x(t, x0 )∗ satisfies the differential equation (3.52). We define the set R n H+ = cl conv{X(t) R t ≥ 0} ⊂ H+ , which is the closed convex hull of the positive orbit of X(t). Let H = H+ − H+ be the minimal subspace of Hn containing H+ , endowed with the inner product inherited from Hn . Then H+ is a closed solid regular cone in H. By construction, both H+ and H are invariant with respectR to (3.52). That means R (LA + ΠA0 )∗ (H) ⊂ H, and the restriction (LA + ΠA0 )∗ R is resolvent positive H R R with respect to H+ . Let βH be the spectral bound of (LA + ΠA0 )∗ R . We will show that βH < 0, which implies X(t) → 0 and thus also (3.53). By Remark 3.2.7 there exists an eigenvector XH ∈ H+ , such that

H

(LA + ΠA0 )∗ (XH ) = βH XH . Since for all t ≥ 0 0 = E
3.7 Minimal representations We conclude this chapter with a result on a minimal representation of the inverses of Sylvester and Lyapunov operators in the form (3.5). This result will not be needed in the sequel, but it helps to complete the picture. Our principal aim is to determine the minimal number of summands N in a representation of L−1 A . To discuss the problem in more generality, we define the Sylvester operator SA,B : Km×n → Km×n for A ∈ Km×m , B ∈ Kn×n by

3.7 Minimal representations

99

SA,B (X) = AX − XB . Clearly, if n = m and B = −A∗ then SA,B = LA is a Lyapunov-operator. It is assumed that SA,B is nonsingular i.e. 0 O∈ σ(A) − σ(B). Then the following holds. Theorem 3.7.1 Let νA = deg µA and νB = deg µB denote the degrees of the minimal polynomials of A and B, respectively, and ν = min{νA , νB }. (i)

−1 : Km×n → Km×n has a representation of the form The inverse SA,B −1 SA,B (Y ) =

ν O

Vj Y Wj ,

Vj ∈ Km×m , Wj ∈ Kn×n .

i=1

(ii) If B = −A∗ , i.e. SA,B = LA , there exists a symmetric representation L−1 A (Y ) =

νA O

εj Aj Y A∗j ,

εj = ±1, Aj ∈ Kn×n

i=1

with εj = 1 for all j if and only if σ(A) ⊂ C+ . (iii) If an arbitrary representation −1 SA,B (Y ) =

N O

Vj Y Wj ,

Vj ∈ Km×m , Wj ∈ Kn×n

i=1

is given, then necessarily N ≥ ν. Remark 3.7.2 (i) From our proof (and Theorem 3.7.4) it is immediate to see, that an analogous result holds for the discrete Sylvester operator SA,B : X L→ X − AXB. In this case, of course, SA,B is invertible if and only if 1 O∈ σ(A)σ(B), and SA,B specializes to the discrete Lyapunov operator if B = A∗ . Taking this into account, all assertions of Theorem 3.7.1 carry over literally. (ii) In our proof of Theorem 3.7.1 we make use of Theorem 4 in [53]. In fact one can easily use Theorems 1 and 4 in [53] to prove a stronger version of (i), where Vj = pj (A) and Wj = qj (B) with polynomials pj and qj . Our main concern is, however, that one needs at least ν terms in the representation. (iii) Theorem 3.7.1 has appeared in [41]. It answers a question raised by V. Mehrmann at the Oberwolfach meeting 45/2000 on “Nichtnegative Matrizen, M -Matrizen und deren Verallgemeinerungen”. The proof of Theorem 3.7.1 is an application of the Theorems 3.3.5 and 3.3.7. It only remains to determine the rank of the matrix Dm,n = (mn) −1 −1 )(E (mn) ) = SA,B (Eij ) . (3.54) X = (Imn ⊗ SA,B i,j=1

100

3 Linear mappings on ordered vector spaces

To this end we apply the following result from [53, Theorem 4]. For arbitrary k ∈ N and a given matrix vector pair (C, x) ∈ Kk×k × Kk let K(C, x) = (x, Cx, . . . , C k−1 x) denote the reachability matrix. Theorem 3.7.3 Let A ∈ Km×m , B ∈ Kn×n , v ∈ Km , w ∈ Kn , and assume that the Sylvester equation AX − XB = vw∗ has a unique solution X. Then rk X = min{rk K(A, v), rk K(B ∗ , w)}. The analogous result for the discrete Sylvester equation was proven in [209]. Theorem 3.7.4 Let A ∈ Km×m , B ∈ Kn×n , v ∈ Km , w ∈ Kn , and assume that the Sylvester equation X − AXB = vw∗ has a unique solution X. Then rk X = min{rk K(A, v), rk K(B ∗ , w)}. Proof of Theorem 3.7.1: In view of the Theorems 3.3.5 and 3.3.7 we have to show that rk X = ν. = Dm,n conformably with E (mn) , such that If in (3.54) we partition X = Xij i,j=1

(mn)

Xij ∈ Km×n , then we can also write AXij − Xij B = Eij

and thus

(In ⊗ A)X − X (Im ⊗ B) = E (mn) .

(3.55)

In other words X is a solution of the Sylvester equation (3.55) with the right hand side E (mn) . By (3.3) (with V = Im , W = In ), we have E (mn) = vec Im (vec In )∗ . In the following equations let k = m if C = A or k = n if C = B ∗ . It follows from Theorem 3.7.3, that D = 2 rk X = min ∗ rk vec Ik , (Ik ⊗ C) vec Ik , . . . , (Ik ⊗ C)k vec Ik C∈{A,B }  (k)  2 (k) (k) e1 Ce1 . . . C k e1 2 (k)   (k) (k)  e2 Ce2 . . . C k e2    . = min ∗ rk  . .. ..  . C∈{A,B }  .  . . (k)

(k)

2

(k)

ek Cek . . . C k ek

It is clear, that the first νC + 1 columns are linearly dependent, if νC = deg µC denotes the degree of the minimal polynomial of C; hence rk X ≤ νA and rk X ≤ νB . If we consider arbitrary linear combinations of the block rows, we find = D rk X ≥ min ∗ max rk x , Cx , . . . , C k x = min ∗ νC = ν . C∈{A,B } x∈Kk

C∈{A,B }

Hence rk X = min{νA , νB }, which (by the Theorems 3.3.5 and 3.3.7) proves (i), (iii) and the first part of (ii). To prove the second part of (ii) we have to show that the εj can be chosen

3.7 Minimal representations

101

positive if and only if σ(A) ∈ C+ . But it follows e.g. from Theorem 3.6.1 that L−1 A is positive if and only if σ(A) ⊂ C+ , in which case it is even completely positive by Proposition 3.4.4. " The analogous result for the discrete Sylvester operator SA,B (X) = X −AXB follows, if one applies Theorem 3.7.4 instead of Theorem 3.7.3.

4 Newton’s method

This chapter contains some of our main results. It largely follows the presentation in [49]. Our object is to tackle rational matrix equations of the form (2.27), derived in Chapter 2, which we view, very generally, as nonlinear operator equations in an ordered vector space. A standard method to solve nonlinear equations is the Newton iteration. Its applicability to operator equations in normed spaces was first established by Kantorovich in [122]. In the operator-theoretic setup the method is therefore often referred to as the Newton-Kantorovich procedure. Kantorovich specified boundedness conditions on the first and second Fr´echet derivatives of the nonlinear operator, which guarantee convergence of the Newton iteration starting in a small neighbourhood of the actual solution. These results can be simplified and generalized, if the underlying space is ordered and the sequence produced by the iteration can be shown to be monotonic and bounded; this can be the case, for instance, if the nonlinear operator satisfies certain convexity conditions (compare [196] and references therein). In general, however, results on the convergence of the Newton iteration are of a local nature; they require a good initial guess. But in some special cases, e.g. for Riccati operators (cf. [131, 212, 102, 34, 84, 58, 94, 48]), it has been observed that the iteration converges from any point where the derivative of the Riccati operator has its spectrum in the left half plane. This is what we call a non-local convergence result, since the starting point may be far away from any solution of the Riccati equation. In the following, we introduce a class of nonlinear operator equations in ordered Banach spaces that can be solved by a non-local Newton iteration. Moreover, we discuss modifications of Newton’s method that can be used to reduce the computational cost. We make frequent use of the properties of resolvent positive operators proven in Section 3.2. Furthermore, we need the notions of directional concavity and D+ -concavity, which will be defined in the first section of the present chapter.

T. Damm: Rational Matrix Equations in Stochastic Control, LNCIS 297, pp. 103–121, 2004. Springer-Verlag Berlin Heidelberg 2004

104

4 Newton’s method

4.1 Concave maps In the following let X be a real Banach space ordered by a pointed closed convex cone C and consider a (nonlinear) mapping f : X ⊃ dom f → X. Definition 4.1.1 Assume we are given subsets D+ ⊂ D ⊂ dom f . Then we say that f is D+ -concave on D, if we can attach a bounded linear operator Tx : X → X at each point x ∈ D, such that for all x ∈ D ∀y ∈ D+ : f (y) ≤ f (x) + Tx (y − x) .

(4.1)

Remark 4.1.2 (i) Let X = R with the canonical ordering. If e.g. D = dom f = X then f is D+ -concave on D, if at each point of the graph of f we can attach a straight line, such that the whole graph of f above D+ lies below this line. (ii) As a simple, but typical example, consider D = R \ {0}, D+ =]0, ∞[, and f (x) = − x1 with Tx = fx3 . Obviously, f is concave on D+ and convex on −D+ . It is, however, D+ -concave on D, as a short calculation (compare Example 4.4.7(ii)) or an easy geometric argument as in (i) shows. (iii) In the next section, D+ plays the role of a target set, in which we try to find a solution of f (x) = 0, whereas D contains possible initial values for a fixed-point iteration. The D+ -concavity on D is needed, to guarantee the transition from D to D+ . We will also consider the following weaker version of D+ -concavity. Definition 4.1.3 Assume the situation of Definition 4.1.1 and let K ⊂ X. We say that f is D+ -concave on D in direction K, if the inequality in (4.1) holds for all x ∈ D, y ∈ D+ , such that y − x ∈ K. Furthermore, we call f strictly D+ -concave on D in direction K, if y −x ∈ int K \{0} implies that the inequality (4.1) is strict. If D = D+ , we just say that f is (strictly) concave on D in direction K. Obviously, f is D+ -concave on D, if and only if it is D+ -concave on D in direction X. Finally we recall the definition of Gˆ ateaux differentiability. Definition 4.1.4 Let f : X ⊃ dom f → X and x ∈ int dom f . Then we say that f is Gˆ ateaux differentiable at x if, for every h ∈ X the map t L→ f (x+th) defined for x + th ∈ dom f is differentiable at t = 0. In other words, there exists a mapping fx3 : X → X such that for all h ∈ X and all t ∈ R with |t| sufficiently small f (x + th) = f (x) + tfx3 (h) + tφx,h (t)

with lim φx,h (t) = 0 . t→0

4.2 Resolvent positive operators and Newton’s method

105

The following proposition deals with the case, when equality occurs in (4.1) for some pair (x, y) ∈ D×int D+ . Under certain concavity and differentiability conditions we can conclude that then Tx and Ty coincide. We will need this in the analysis of iterative methods, to conclude that σ(Txk ) ⊂ C− implies σ(Txk+1 ) ⊂ C− , where the xk form e.g. a Newton sequence. Proposition 4.1.5 Let D+ ⊂ D ⊂ dom f , K ⊂ X, and assume f : dom f → X to be D+ -concave on D in direction K. Let further x ∈ D and y ∈ int D+ such that y − x ∈ int K and assume f to be Gˆ ateaux differentiable at y. If f (y) − f (x) = Tx (y − x), then fy3 = Tx . In particular, fy3 = Ty if 0 ∈ int K. (ii) If 1f (y) − f (x), v2 = 1Tx (y − x), v2 for some v ∈ C ∗ , then (fy3 )∗ (v) = Tx∗ (v).

(i)

Proof: (i) If f (y) − f (x) = Tx (y − x), we have for all z ∈ D+ with z − x ∈ K f (z) ≤ f (x) + Tx (z − x) = f (x) + Tx (z − y) + Tx (y − x) = f (y) + Tx (z − y) . In particular, it follows for every h ∈ X and z = y ± th with 0 < t < 1 sufficiently small that z ∈ D+ , z − x ∈ K and f (y ± th) = f (y) ± tfy3 (h) ± tφy,h (±t) ≤ f (y) + Tx (±th) = f (y) ± tTx (h) . ≤ ±Tx (h), whence fy3 (h) = Tx (h) for all h ∈ X. As t → 0 we obtain (ii) Applying the functional v to all the expressions in the proof of (i) we " obtain 1fy3 (h), v2 = 1Tx (h), v2 for all h ∈ X, whence (fy3 )∗ (v) = Tx∗ (v). ±fy3 (h)

Let us note a simple consequence of this proposition. Remark 4.1.6 If f is D+ -concave on D+ in direction K and 0 ∈ int K then fy3 = Ty at every point y ∈ int D+ where f is Gˆateaux differentiable. Thus, the Gˆ ateaux differential fy3 : X → X is automatically linear and bounded under these conditions.

4.2 Resolvent positive operators and Newton’s method Throughout this section let X be a real Banach space, ordered by a closed, solid, regular convex cone C and f a continuous mapping from some subset dom f of X to X. Moreover, let there be given subsets D+ ⊂ D ⊂ dom f and attached to each point x ∈ D a bounded linear mapping Tx : X → X. We consider the following assumptions.

106

4 Newton’s method

Assumption 4.2.1 (H1) D+ = D+ + C. (H2) f is D+ -concave on D (with the given Tx ). (H3) For all x ∈ D the operator Tx is resolvent positive. (H4) The Tx are locally bounded on D+ , i.e. ∀x ∈ D+ : ∃ε > 0 :

sup

y∈D+ ,'y−x'<ε


(H5) There exists an x0 ∈ D such that σ(Tx0 ) ⊂ C− . x) ≥ 0. (H6) There exists an xˆ ∈ int D+ such that f (ˆ (H7) f is Gˆ ateaux differentiable on int D+ . If (H5) is satisfied, then we call Tx0 stable and x0 stabilizing (for T ). We address the problem of approximating a solution x ∈ D+ to the equation f (x) = 0 by the Newton-type iteration xk+1 = xk − (Txk )

−1

(f (xk )) .

(4.2)

Theorem 4.2.2 Let Assumption 4.2.1 hold or assume, alternatively, the hypotheses (H1)–(H5) and (H8) There exists an x ˆ ∈ D+ such that f (ˆ x) > 0. Then the iteration scheme (4.2) starting at an arbitrary x0 ∈ D such that σ(Tx0 ) ⊂ C− defines a sequence x1 , x2 , . . . in D+ with the following properties: (i) ∀k = 1, 2, . . . : xk ≥ xk+1 ≥ xˆ, f (xk ) ≤ 0, and σ(Txk ) ⊂ C− . (ii) x+ := lim xk ∈ D+ satisfies f (x+ ) = 0 and is the largest solution of k→∞

the inequality f (x) ≥ 0 in D+ (i.e. f (x) ≥ 0 ⇒ x ≤ x+ for all x ∈ D+ ). Proof: We prove (i) inductively and will only use conditions (H1) to (H5) and xˆ ∈ D+ , f (ˆ x) ≥ 0 in the first few steps of the proof. Suppose that x0 , . . . , xk have been constructed for some k ≥ 0 such that Txi is stable for i = 0, . . . , k, x1 ≥ . . . ≥ xk and f (xi ) ≤ 0 for i = 1, . . . , k. Then −Txk is inverse positive by Theorem 3.2.10 (with P = 0) so that xk+1 is well defined by (4.2) and satisfies Txk (xk − xk+1 ) = f (xk ) .

(4.3)

ˆ. Since xk ∈ D and xˆ ∈ D+ we obtain from the We first prove xk+1 ≥ x D+ -concavity of f on D that x − xk+1 ) = Txk (ˆ x − xk ) + Txk (xk − xk+1 ) Txk (ˆ = Txk (ˆ x − xk ) + f (xk ) ≥ f (ˆ x) ≥ 0 .

(4.4)

ˆ ≤ xk+1 . But we know already that −Txk is inverse positive and so we have x ˆ ∈ D+ and D+ = D+ + C. If k ≥ 1, then it Hence xk+1 ∈ D+ , because x follows by the same argument from (4.3) and f (xk ) ≤ 0 that xk − xk+1 ≥ 0.

4.2 Resolvent positive operators and Newton’s method

107

It remains to show that Txk+1 is stable and f (xk+1 ) ≤ 0. For this we make use of the concavity condition for the pairs (xk+1 , xk ), (ˆ x, xk+1 ) ∈ D+ × D, to obtain f (xk+1 ) ≤ f (xk ) + Txk (xk+1 − xk ) , f (ˆ x) ≤ f (xk+1 ) + Txk+1 (ˆ x − xk+1 ) .

(4.5) (4.6)

By (4.3) the right side in (4.5) vanishes, whence f (xk+1 ) ≤ 0. Therefore (4.6) yields f (ˆ x) ≤ Txk+1 (ˆ x − xk+1 ) = −Txk+1 (xk+1 − x ˆ) .

(4.7)

If now f (ˆ x) > 0 then σ(Txk+1 ) ⊂ C− by Theorem 3.2.10, which completes the proof of (i) under conditions (H1)–(H5) and (H8). To complete the proof of (i) under Assumption 4.2.1 let us suppose that Txk+1 is not stable. By Theorem 3.2.3 (ii) this is equivalent to the condition ∃v ∈ C ∗ \ {0}, β ≥ 0 : Tx∗k+1 v = βv .

(4.8)

Together with the four inequalities x ˆ ≤ xk+1 , f (ˆ x) ≥ 0, f (xk+1 ) ≤ 0 and (4.6), this implies " # x − xk+1 , βv2 = Txk+1 (ˆ x − xk+1 ), v ≥ 1−f (xk+1 ), v2 ≥ 0.(4.9) 0 ≥ 1ˆ Hence 1f (xk+1 ), v2 = 0, which by (4.3) means 1f (xk+1 ) − f (xk ), v2 = 1Txk (xk+1 − xk ), v2 . But xk+1 ≥ xˆ ∈ int D+ and D+ = D+ + C imply xk+1 ∈ int D+ , and so f is Gˆateaux differentiable at xk+1 by condition (H7). From Proposition 4.1.5 (with K = X) we conclude that Txk+1 = fx3 k+1 , and Tx∗k (v) = (fx3 k+1 )∗ (v) = Tx∗k+1 (v) = βv, in contradiction with σ(Txk ) ⊂ C− . Thus Txk+1 must be stable, and this concludes our proof of (i) under Assumption 4.2.1. (ii) By (i) and the regularity of C, the xk converge in norm to some x+ ∈ x ˆ + C ⊂ D+ . Since the Tx are locally bounded on D+ , we can pass to the limit in (4.3) to obtain f (x+ ) = 0. By the first part of the proof the inequality xk+1 ≥ x ˆ holds true for all k ∈ N and all solutions xˆ ∈ D+ of the inequality f (x) ≥ 0 (we only used x ˆ ∈ D+ and not x ˆ ∈ int D+ in the first part of the proof). Therefore x+ is the largest solution of this inequality in D+ . " We say that a solution x˜ ∈ dom f of f (x) = 0 is stabilizing, if σ(Tx˜ ) ⊂ C− . The following lemma shows that there can be at most one stabilizing solution in D+ . Lemma 4.2.3 Suppose that f is D+ -concave on D with resolvent positive Tx and let y ∈ D+ , z ∈ D and f (z) ≤ f (y). If σ(Tz ) ⊂ C− then z ≥ y. In particular, under the assumptions of Theorem 4.2.2, if z ∈ D+ is a stabilizing solution of f (x) = 0 then z = x+ .

108

4 Newton’s method

Proof: By concavity Tz (y − z) ≥ f (y) − f (z) ≥ 0, whence by the stability and resolvent positivity of Tz we have y − z ≤ 0. Now suppose z ∈ D+ is a stabilizing solution of f (x) = 0, then we choose y = x+ and get z ≥ x+ . " On the other hand we have z ≤ x+ by Theorem 4.2.2. The following corollary of Theorem 4.2.2 gives a sufficient condition for the existence of a stabilizing solution. Corollary 4.2.4 If conditions (H1)–(H5) and (H8) hold then x+ in Theorem 4.2.2 is a stabilizing solution of f (x) = 0 and satisfies x+ > xˆ (whence x+ ∈ int D+ ). Proof: We already know x+ ≥ xˆ, and by concavity Tx+ (ˆ x − x+ ) ≥ f (ˆ x) − f (x+ ) = f (ˆ x) > 0 ; ? F hence σ Tx+ ⊂ C− and x+ > x ˆ follows from Theorem 3.2.10.

"

The existence of a stabilizing solution can also be proven under a detectability assumption (cf. Def. 3.2.8) instead of (H8). Corollary 4.2.5 Assume that f is concave and Gˆ ateaux-differentiable on D+ . ˆ ≤ x1 and f (ˆ x) ≥ f (x1 ). If one of the pairs ˆ, x1 ∈ int D+ satisfy x Let x (Txˆ , f (ˆ x) − f (x1 )) or (Tx1 , f (ˆ x) − f (x1 )) is β-detectable, then Tx1 is stable. x)) is βIn particular, if conditions (H1)–(H7) hold and the pair (Txˆ , f (ˆ detectable, then x+ in Theorem 4.2.2 is a stabilizing solution of f (x) = 0. Proof: By concavity, we have Tx1 (ˆ x − x1 ) ≥ f (ˆ x) − f (x1 ) ≥ 0 . We assume that Tx1 is not stable, i.e. β = β(Tx1 ) ≥ 0, and there exists an eigenvector v ∈ C ∗ , satisfying Tx∗1 v = βv. This implies 0 ≥ 1ˆ x − x1 , βv2 = 1Tx1 (ˆ x − x1 ), v2 ≥ 1f (ˆ x) − f (x1 ), v2 ≥ 0 , such that in particular 1Tx1 (ˆ x − x1 ), v2 = 1f (ˆ x) − f (x1 ), v2 = 0. Hence, (Tx1 , f (ˆ x) − f (x1 )) is not β-detectable. Moreover, like in the proof of Theorem 4.2.2 we conclude from Proposition 4.1.5 that Txˆ∗ (v) = Tx∗1 (v) = βv, which implies that (Txˆ , f (ˆ x) − f (x1 )) is not β-detectable either. It follows that each of the detectability assumptions entails the stability of Tx1 . In particular, under the assumptions (H1)–(H7) we can choose x+ from Theorem 4.2.2 in the role of x1 . " We can express this result also as follows. Corollary 4.2.6 Assume (H1)–(H4) and (H7). Then the following are equivalent:

4.2 Resolvent positive operators and Newton’s method

109

(i) ∃x+ ∈ int D+ : f (x+ ) = 0, σ(Tx+ ) ⊂ C− . (ii) ∃ˆ x, x1 ∈ int D+ : f (ˆ x) ≥ 0, (Txˆ , f (ˆ x)) is β-detectable, x1 ≥ xˆ, and f (x1 ) ≤ 0. Proof: If (i) holds, then obviously (ii) holds with x ˆ = x1 = x+ . Conversely, (ii) obviously implies (H6); moreover, by Corollary 4.2.5, (ii) im" plies (H5) with x0 = x1 . Hence, by Theorem 4.2.2, (ii) implies (i). If f is Fr´echet differentiable, condition (H8) is equivalent to the existence of a stabilizing solution of f (x) = 0. Corollary 4.2.7 Assume conditions (H1)–(H5). If f is Fr´echet differentiable on int D+ , then x) > 0) ⇐⇒ (∃y ∈ int D+ : f (y) = 0 and σ(fy3 ) ⊂ C− ) . (∃ˆ x ∈ D+ : f (ˆ Proof: Fr´echet differentiability implies Gˆ ateaux differentiability and by Proposition 4.1.5 we have fx3 = Tx for all x ∈ int D. By Corollary 4.2.4 it only remains to prove ‘⇐’. Let y ∈ int D+ and f (y) = 0, σ(fy3 ) ⊂ C− . Then 0 O∈ σ(fy3 ) and so fy3 is an invertible bounded linear operator on X. By the inverse function theorem, f maps a small neighbourhood U ⊂ D+ of y onto a neighbourhood V of f (y) = 0. Choosing c ∈ V ∩ int C we see that there exists x ˆ ∈ U such that f (ˆ x) = c > 0. " The existence of stabilizing solutions implies quadratic convergence of the sequence (xk ), provided f is sufficiently smooth. This is e.g. a consequence of the following well known result (compare [148, 137, 36]) and also Remark 4.2.15). Proposition 4.2.8 Assume the situation of Theorem 4.2.2, and let f be Fr´echet differentiable in a neighbourhood U of x+ . Moreover, assume that the Tx satisfy a Lipschitz condition
110

4 Newton’s method

(iii) The iteration (4.2) requires the solution of a linear equation of the form Txk x = y in each step. It has been observed in the context of Riccati equations (see e.g. [92]) that it can be advantageous to replace the operators Txk by other operators that are numerically easier to handle. In the sequel we suggest a general framework to take advantage of these observations. 4.2.1 A modified Newton iteration Suppose we have a decomposition of Tx of the form Tx = Rx + Px , where the Rx are resolvent positive and the Px are positive. Replacing Txk by Rxk in (4.2) we define the iteration xk+1 = xk − (Rxk )−1 (f (xk )) .

(4.10)

ˆ ≤ x0 and attached to each point Let there be given x ˆ, x0 ∈ dom f such that x x ∈ [ˆ x, x0 ] a bounded linear mapping Tx : X → X. We consider the following assumptions. Assumption 4.2.10 (H1’) [ˆ x, x0 ] ⊂ int dom f . x, x0 ] in direction K (with the given Tx ) and K ⊃ −C. (H2’) f is concave on [ˆ (H3’) Tx = Rx + Px , where the Rx are resolvent positive and the Px are positive for all x ∈ [ˆ x, x0 ]. (H4’) The Rx are locally bounded on [ˆ x, x0 ]. (H5’) σ(Tx0 ) ⊂ C− and f (x0 ) ≤ 0. (H6’) f (ˆ x) ≥ 0. (H7’) f is Gˆ ateaux differentiable at all x ∈ [ˆ x, x0 ] with fx3 = Tx , and int K ⊃ −C \ {0}. Note that the sets D, D+ do not appear in this framework. The following result has two aspects corresponding to Remark 4.2.9 (ii) and (iii). Firstly, we weaken the concavity conditions in Theorem 4.2.2 at the price of strengthening the requirements imposed on the initial value x0 . Secondly, we allow the operators Tx to be replaced by the Rx , at the price of possibly diminishing the rate of convergence. Theorem 4.2.11 Let Assumption 4.2.10 hold or, alternatively, assume the hypotheses (H1’) - (H6’) with the stronger requirement f (ˆ x) > 0. Then the iteration scheme (4.10) starting at x0 defines a monotonically decreasing sequence in [ˆ x, x0 ] with the following properties: (i) ∀k = 0, 1, . . . : xk ≥ xk+1 ≥ xˆ, f (xk ) ≤ 0, and σ(Txk ) ⊂ C− . (ii) x+ := lim xk ∈ [ˆ x, x0 ] satisfies f (x+ ) = 0 and is the largest solution k→∞

of the inequality f (x) ≥ 0 in [ˆ x, x0 ] (i.e. f (x) ≥ 0 ⇒ x ≤ x+ for all x ∈ [ˆ x, x0 ]).

4.2 Resolvent positive operators and Newton’s method

111

Proof: The proof of (i) proceeds by induction and follows the proof of Theorem 4.2.2. We only elaborate on those points where we must take into account the new assumptions. Let us assume that for some k ≥ 0 we have constructed ˆ and f (xi ) ≤ 0 for i ≤ k. x0 ≥ x1 ≥ . . . ≥ xk such that Txi is stable, xi ≥ x Then also Rxk is stable by Theorem 3.2.10 (with T = Rxk ) and hence −Rxk is inverse positive. Thus xk+1 is well defined by (4.10) and satisfies Rxk (xk − xk+1 ) = f (xk ) ≤ 0 .

(4.11)

We have to show that xk ≥ xk+1 ≥ x ˆ, f (xk+1 ) ≤ 0 and σ(Txk+1 ) ⊂ C− . Since −Rxk is inverse positive, (4.11) implies xk ≥ xk+1 . To prove xk+1 ≥ x ˆ we utilize the concavity of f at xk in direction xˆ − xk ≤ 0 and remember that Txk (y) ≤ Rxk (y) for y ≤ 0. This yields Rxk (ˆ x − xk+1 ) = Rxk (ˆ x − xk ) + Rxk (xk − xk+1 ) x − xk ) + f (xk ) ≥ f (ˆ x) ≥ 0. ≥ Txk (ˆ

(4.12)

Hence x ˆ ≤ xk+1 and so xk+1 ∈ [ˆ x, x0 ]. By the concavity of f at xk and xk+1 in the directions (xk+1 − xk ) ≤ 0 and (ˆ x − xk+1 ) ≤ 0, respectively, we have f (xk+1 ) ≤ f (xk ) + Txk (xk+1 − xk ) ≤ f (xk ) + Rxk (xk+1 − xk ) = 0 f (ˆ x) ≤ f (xk+1 ) + Txk+1 (ˆ x − xk+1 )

(4.13) (4.14)

and so f (xk+1 ) ≤ 0. Up till now we have only made use of the hypotheses x) > 0 then σ(Txk+1 ) ⊂ C− (H1’) - (H6’) with K = −C. If we assume f (ˆ follows as in Theorem 4.2.2. This concludes the proof of (i) for the alternative assumptions. Now assume that Assumption 4.2.10 holds and suppose that Txk+1 is not stable. By Theorem 3.2.3 this is equivalent to the existence of v ∈ C ∗ \ {0} satisfying condition (4.8), which by the inequalities (4.9) implies 1f (xk+1 ), v2 = 0. Hence by (4.11) and (4.13) 1f (xk+1 ) − f (xk ), v2 = 1Rxk (xk+1 − xk ), v2 ≥ 1Txk (xk+1 − xk ), v2 ≥ 1f (xk+1 ) − f (xk ), v2 , where we obviously have equality everywhere. Since xk+1 − xk ∈ int K we ∗ can apply Proposition 4.1.5 to obtain Tx∗k (v) = f 3 xk+1 (v) = Tx∗k+1 (v) = βv, in contradiction to σ(Txk ) ⊂ C− . Thus Txk+1 must be stable. (ii) By (i) and the regularity of C, the xk converge in norm to some x+ ∈ [ˆ x, x0 ]. Since the Tx are locally bounded for x ∈ [ˆ x, x0 ], we can pass to the x, x0 ] satisfy f (x) ≥ 0. limit in (4.11) to obtain f (x+ ) = 0. Now let x ∈ [ˆ Proceeding as in the first part of the proof of (i) with x instead of xˆ, we obtain x ≤ xk+1 and so by induction x ≤ xk for all k ∈ N, whence x ≤ x+ .

112

4 Newton’s method

" Loosely speaking, the alternative in the assumption of Theorem 4.2.11 says that we can trade the smoothness condition (H7’) for the strictness condition f (ˆ x) > 0. But from the proof it is easily seen that we can substitute both these assumptions by another strictness condition. Corollary 4.2.12 Assume the hypotheses (H1’)–(H6’) with the stronger ref (x0 ) > 0 (for inquirement of strict concavity in (H2’). Moreover, let Rx−1 0 stance f (x0 ) < 0). Then the assertions (i) and (ii) of Theorem 4.2.11 hold, with strict inequalities ˆ and f (xk+1 ) < 0 for all k = 0, 1, . . .. in (i), i.e. xk+1 > xk > x Proof: The operator −Rx0 is inverse positive by (H3’) and (H5’). To start ˆ. Since x ˆ ≤ x0 by assumption, this follows an induction, we verify that x0 > x by concavity from f (ˆ x) − f (x0 ) ≤ fx3 0 (ˆ x − x0 ) which implies x − x0 ) ≤ Rx0 (ˆ x ˆ − x0 ≤ Rx−1 (f (x0 )) < 0. (f (ˆ x) − f (x0 )) ≤ −Rx−1 0 0 Now we assume that for some k ≥ 0 we have constructed x0 > x1 > . . . > xk > x ˆ, such that Txi is stable (hence −Rxi inverse positive) for 0 ≤ i ≤ k, and f (x0 ) > 0. f (xi ) < 0 for 1 ≤ i ≤ k. If k = 0 then by assumption x0 − x1 = Rx−1 0 If k ≥ 1 then Rxk (xk − xk+1 ) = f (xk ) < 0. Hence xk > xk+1 in both cases. Applying the strict concavity condition we can derive the formulae (4.12) and (4.13) in a form, where the second respectively the first inequality is strict. ˆ < xk+1 and the strict concavity Hence x ˆ < xk+1 , and f (xk+1 ) < 0. From x we infer strict inequality in (4.14) and so the stability of Txk+1 follows by Theorem 3.2.10. "

Remark 4.2.13 The main effort in the proofs of Theorem 4.2.11 and Corollary 4.2.12 (and similarly Theorem 4.2.2) is made to establish the posifor all k. Here we need the resolvent positivity of the Rx , tivity of −Rx−1 k the stability of Tx0 , and either assumption (H7’), or one of the strictness conditions. Instead of making these assumptions we could, of course, also assume the inverse positivity of −Rx for all x ∈ [ˆ x, x0 ] (compare [196], e.g. Theorem 5.4.). This property does not follow from any combination of our assumptions. In the context of Riccati equations, it is in general too restrictive (cf. Example 4.4.6). It is clear that even under the conditions of Proposition 4.2.8 we cannot expect quadratic convergence of the sequence (xk ) in Theorem 4.2.11. But we have at least linear convergence, i.e. there exists a constant 0 ≤ θ < 1 and an equivalent norm < · <+ on X, such that for sufficiently large k <xk+1 − x+ <+ ≤ θ<xk − x+ <+ . This is the statement of the following proposition.

(4.15)

4.3 The use of double Newton steps

113

Proposition 4.2.14 Assume the situation of Theorem 4.2.11 and let f be continuously Fr´echet differentiable in a small ball U around x+ such that Tx = fx3 . Moreover, assume that the Tx satisfy a Lipschitz condition
such that the Lipschitz condition gives <φx (y − x)< ≤ x, y ∈ U . By the iteration scheme (4.10) we have

1 2 L
− x<2 for all

xk+1 − x+ = xk − x+ = D f (x − Rx−1 ) + (R + P )(x − x ) − φ (x − x ) + x x k + x + k k k k k Pxk (xk − x+ ) + Rx−1 φxk (x+ − xk ) = −Rx−1 k k = −Rx−1 Pxk (xk − x+ ) + O(<xk − x+ <2 ) , k whence <xk+1 − x+ <+ ≤ θ<xk − x+ <+ for sufficiently large k.

"

Remark 4.2.15 Note that for θ we could choose any number ρ(Rx−1 Px+ ) < + θ < 1. In particular, if Rx+ = Tx+ , Px+ = 0, i.e. the iteration is just the Px+ ) = 0 and can make θ Newton iteration in the limit, we have ρ(Rx−1 + arbitrarily small. This reflects the transition to a quadratically convergent sequence. In fact, if e.g.
4.3 The use of double Newton steps We have analyzed the speed of convergence only for the case that fx3 + is stable. But there are important applications, where x+ is almost stabilizing,

114

4 Newton’s method

which means, that β(fx3 + ) = 0. In this case fx3 + is singular and ρ(Rx−1 Px+ ) = + 1; hence the convergence is typically only linear for the Newton iteration and sublinear for the modified Newton iteration. In the context of standard Riccati equations it has, however, been observed in [94] (see also [92]), that the performance of Newton’s method can be improved significantly, if one considers double Newton steps. The basic idea can also be applied to our situation. But we will have to impose a growth bound on the operators fx−1 k (cf. Lemma 4.3.2), which for standard Riccati equations can be derived from the usual assumptions. Numerical experiments suggest that this growth bound also holds for generalized Riccati equations (to be discussed in the following chapter), but we have not yet identified natural conditions which imply such a bound. In the following let f : dom f ⊂ X → X be a Fr´echet differentiable mapping on a convex open domain dom f in a Banach space X. By φx we denote the second order remainder term in the Taylor expansion of f at x, such that for h∈X f (x + h) = f (x) + fx3 (h) + φx (h) . Let there exist numbers L0 , L1 ≥ 0 such that <φx (x − y)< ≤ L0 <x − y<2

for all x, y ∈ dom f ;

(4.16)

and <(φx − φy )(x − y)< ≤ L1 <x − y<3

for all x, y ∈ dom f .

(4.17)

These conditions are satisfied for instance, if the third Fr´echet derivative of f exists on dom f and is bounded. We assume that f (x+ ) = 0 for some x+ ∈ dom f and that there exists a Newton sequence x0 , x1 , . . . ∈ dom f , converging to x+ . The following technical lemma is a variation of a result in [94]. We need it to describe the case, when the Newton sequence is not quadratically convergent. Lemma 4.3.1 Let θ > 0 and set ( R Qθ = k ∈ N R

L0 θ

there exists an index k0 , such that

<xk+1 − x+ < ≤ κ<xk − x+ <2 for all k ∈ Qθ with k ≥ k0 . Proof: Let k ∈ Qθ . Since f (x+ ) = 0 we have the Taylor expansion f (xk+1 ) = fx3 + (xk+1 − x+ ) + φx+ (xk+1 − x+ ) .

4.3 The use of double Newton steps

115

We apply the triangle inequality and the Lipschitz condition to obtain
"

Obviously, if for some θ > 0 the set N\Qθ is finite, then convergence takes place quadratically with factor κ. If, however, fx3 + is singular or almost singular, such a θ does not exist or it might be close to the machine precision. Hence for (almost) arbitrarily small θ > 0 there exists an index k with
(4.18)

In such a situation a double Newton step can be useful. The key observation is, that f (xk ) is almost equal to φxk (x+ − xk ). Namely, by the Taylor expansion at x+ we have
(4.19)

which is close to zero for sufficiently small θ and <xk − x+ <. Thus the double Newton step −1

yk+1 = xk − 2f 3 xk (f (xk ))

(4.20)

is likely to give a very good approximation to x+ at once. This argument of course requires also
116

4 Newton’s method −1

Lemma 4.3.2 Let 0 and all j ∈ N. If (4.18) holds for some k ∈ N, then yk+1 from (4.20) satisfies
4.4 Illustrative applications Our results in the present chapter aim at a solution of the rational matrix equations derived in Chapter 2. This will be the topic of Chapter 5. In the following we sketch some further applications of the Theorems 4.2.2 and 4.2.11. 4.4.1 L2 -sensitivity optimization of realizations In [217] and [101], problems of optimizing the realization of a transfer function were considered. Without going into the details we sketch the basic elements. Given a strictly proper rational matrix G(s) ∈ Rp×m (s) of McMillan degree n and a minimal realization (A0 , B0 , C0 ) ∈ Ln,m,p (R) := Rn×n × Rn×m × Rp×n G(s) = C0 (sI − A0 )−1 B0 , the set of all minimal realizations of G(s) is given as the orbit of (A0 , B0 , C0 ) under the similarity action (S, (A, B, C)) L→ (SAS −1 , SB, CS −1 ) of Gln (R) on Ln,m,p (R). Following [101] we interpret the (A, B, C) ∈ Ln,m,p (R) as discrete time systems and define the L2 -sensitivity measure of a realization (A, B, C) by Q Q Q Q Q Q Q ∂G Q2 Q ∂G Q2 Q ∂G Q2 2 Q Q Q . Q Q Q + + Γ2 (A, B, C) := Q ∂A Q2 Q ∂B Q2 Q ∂C Q2 To simplify the presentation let now m = p = 1 for the moment. We set 0 6 A0 B0 C0 ˆ = −S −1 (B0 B ∗ ), Cˆ = −S −1∗ (C ∗ C0 ) , Aˆ = , B 0 0 A0 A0 0 A0 where SA0 denotes the Stein operator associated with A0 (Example 3.2.2). Moreover let

0 6 I P1 = , 0

4.4 Illustrative applications

0 6 0 P2 = , I

117

and define −1 ∗ Π(X) = −P1∗ SA ˆ (P2 XP2 )P1 .

Note that the operator P ∗ Π is completely positive, i.e. it has a representation ˆ and Cˆ are the controllability and of the form X L→ Ai XAi . Moreover, B the observability Gramians, respectively, of (A0 , B0 , C0 ), and thus positive definite by the minimality of the realization. It was shown in [101] that a realization (A, B, C) = (SA0 S −1 , SB0 , C0 S −1 ) ∈ Ln,m,p (R) minimizes the L2 -sensitivity measure, if and only if X = S ∗ S solves the matrix equation = D ˆ X = 0. R(X) := Π(X) + Cˆ − X Π ∗ (X −1 ) + B (4.21) In the general case, where m, p ≥ 1 one obtains an analogous matrix equation ˆ Cˆ > 0 and completely positive Π. with B, R The operator R is well defined on dom R = {X ∈ Hn R det X O= 0}. We n wish to solve equation (4.21) in D+ := int H+ ⊂ dom R =: D, and verify the conditions of Theorem 4.2.2. By a straightforward calculation we obtain the explicit form of the Fr´echet derivative of R. Lemma 4.4.1 The Fr´echet derivative R3X (H) of R is given by R3X (H) = L−X (Π ∗ (X −1 )+Bˆ ∗ ) (H) + Π(H) + XΠ ∗ (X −1 HX −1 )X .(4.22) As the sum of a Lyapunov operator and positive operators, R3X is resolvent positive. Lemma 4.4.2 The operator R is D+ -concave on D. ˆ are obviously D+ Proof: Both Π and the quadratic mapping X L→ −X BX concave on D. Thus it remains to analyze the operator X L→ F (X) := −XΠ ∗(X −1 )X . For Z ∈ D and Y ∈ D+ we have F (Z) − F (Y ) + FZ3 (Y − Z) = −ZΠ ∗ (Z −1 )Z + Y Π ∗ (Y −1 )Y − Y Π ∗ (Z −1 )Z +ZΠ ∗ (Z −1 )Z − ZΠ ∗ (Z −1 )Y + ZΠ ∗ (Z −1 )Z +ZΠ ∗ (Z −1 Y Z −1 )Z − ZΠ ∗ (Z −1 )Z 0 6∗ 0 ∗ −1 60 6 Y −Π ∗ (Z −1 ) Y Π (Y ) = ≥ 0. Z −Π ∗ (Z −1 ) Π ∗ (Z −1 Y Z −1 ) Z

118

4 Newton’s method

The inequality holds because 0 −1 6 0 −1 6 0 −1 6∗ −Z −1 Y Y Y ≥0 = Y −Z −1 Z −1 Y Z −1 −Z −1 −Z −1 for Y > 0 and Π ∗ has a representation of the form X L→ is D+ -concave on D, which completes the proof.

P

A∗i XAi . Thus F "

ˆ and C, ˆ that R3 is stable for It follows from the positive definiteness of B tI sufficiently large t > 0 and that R(tI) > 0 for sufficiently small t > 0. Thus Theorems 4.2.2 and 4.2.11 can be applied to solve the equation R(X) = 0. As a starting point we can choose X0 = tI for sufficiently large t > 0. If we apply the Newton iteration, we have to solve a linear equation of the form L−Xk (Π ∗ (X −1 )+Bˆ ∗ ) (Hk ) + Π(Hk ) + Xk Π ∗ (Xk−1 Hk Xk−1 )Xk = R(Xk ) (4.23) k

for Hk in each step to obtain the next iterate Xk+1 = Xk − Hk . Since the number of unknown scalar entries in Hk is n(n + 1)/2, a direct solution of this equation would require O(n6 ) operations. As has been pointed out in [92], and in view of Theorem 4.2.11, it might be advantageous to solve the following simplified equation for Hk L−Xk (Π ∗ (X −1 )+Bˆ ∗ ) (Hk ) = R(Xk ) . k

(4.24)

This is just a Lyapunov equation and can be solved efficiently e.g. by the Bartels-Stewart algorithm [11] in O(n3 ) operations. But obviously there is a trade-off between the rate of convergence and the complexity of the linear equations to be solved. Numerical experiments affirm that one can benefit from a combination of both methods. The idea is to use the cheaper fixed point iteration given by (4.24) until the iterates approach the solution when one can really exploit the fast quadratic convergence of Newton’s method. 4.4.2 A non-symmetric Riccati equation In a series of papers (see [93] and references therein) non-symmetric algebraic Riccati equations have been analyzed that arise in transport theory or in the Wiener-Hopf factorization of Markov chains. These equations have the general form R(X) = XCX − XD − AX + B = 0 ,

(4.25)

with (A, B, C, D) ∈ Rm×m × Rm×n × Rn×m × Rn×n . For arbitrary p, q ∈ N we regard Rp×q as an ordered vector space with the

4.4 Illustrative applications

119

p×q closed, solid, pointed convex cone R+ . The operator R thus maps the orm×n dered vector space R to itself. We are looking for the smallest nonnegative solution of the equation R(X) = 0. Like in [93] we make the following assumptions.

Assumption 4.4.3 (i) The linear operator H L→ −(AH + HD) on Rm×n is resolvent positive and stable. (ii) B ≥ 0, and ∃G > 0 : AG + GD = B. (iii) C ≥ 0, C O= 0. By definition, (i) holds if and only if I ⊗ A + DT ⊗ I is an M -matrix (Example 3.2.2). m×n rather than concave, As the condition C ≥ 0 forces R to be convex on R+ we substitute Y = −X and make the transformation f (Y ) = −R(−Y ). Now we are looking for the largest nonpositive solution of the equation f (Y ) = −Y CY − Y D − AY − B = 0,

Y ∈ Rm×n .

The derivative of f at some Y in direction H is given by fY3 (H) = −(A + Y C)H − H(D + CY ) . For Y, Z ∈ Rm×n with Y > Z or Z > Y it follows from (iii) that f (Z) − f (Y ) + fZ3 (Y − Z) = (Z − Y )C(Z − Y ) > 0 . . For Y ≤ 0 the derivative Hence f is strictly concave in direction K = −Rm×n + fY3 is resolvent positive, because H L→ −(AH + HD) is resolvent positive by 4.4.3 (i), and the mapping H L→ −(Y CH + HCY ) is positive, by Y C, CY ≤ 0. Also f03 is stable by 4.4.3 (i) and f (0) = −B ≤ 0. Hence, for any pair Yˆ ≤ Y0 = 0 the conditions (H1’) – (H5’) are fulfilled, with RY = TY = fY3 . Moreover, we have strict concavity in (H2’) and R0−1 f (0) = G > 0. If finally we assume the existence of some solution Yˆ ≤ 0 of the inequality f (Yˆ ) ≥ 0, then also (H6’) is satisfied; hence Corollary 4.2.12 can be applied to rediscover Theorem 2.1 from [93]. Corollary 4.4.4 Consider the operator R from (4.25) and let Assumption −1 ˆ ≤0 4.4.3 hold. Set X0 = 0 and Xk+1 = Xk −R3 Xk (R(Xk )) for k ≥ 0. If R(X) 3 ˆ ≥ 0, then σ(R ) ⊂ C+ and the Xk are well defined for all k. The for some X Xk ˆ and converge Xk are strictly monotonically increasing, bounded above by X, m×n to the smallest solution X∞ of (4.25) in R+ . Furthermore (by continuity) σ(R3X∞ ) ⊂ C+ ∪ iR. Remark 4.4.5 (i) In general, we can not apply Theorem 4.2.2 or 4.2.11 in ˆ < 0 or C = 0. This is because f this framework unless we assume R(X) ˜ ˜ ⊃ Rm×n \{0}. is not concave in any direction K with int K +

120

4 Newton’s method

(ii) In [93] also other fixed point iterations of the form (4.10) have been considered. In terms of the transformed operator f they amount to a splitting fY3 (H) = RY (H) + PY (H) with a resolvent positive, stable operator RY (H) = R0 (H) = −(A0 H + HD0 ), such that A0 ≥ A and D0 ≥ D. The operator PY (H) = (A0 − A − Y C)H + H(D0 − D − CY ) is then positive for all Y ≤ 0, whereas −RY = −R0 does not depend on Y , is resolvent positive and stable by assumption, hence inverse positive for all Y ∈ [Yˆ , 0]. As has been pointed out in [93] we can drop the second part of Assumption 4.4.3 (ii) in this case. This is the situation described in Remark 4.2.13. 4.4.3 The standard algebraic Riccati equation In the following chapter, we will apply our results on Newton’s method to generalized Riccati equations. Now, as an illustration we consider the technically easier case of a standard Riccati equation. In particular, we use this example to demonstrate that our results on Newton’s method do not follow from Vandergraft’s results mentioned in Remark 4.2.13. n , such that (A, Q) is stabilizable and the pair (A, P ) Let A ∈ Kn×n , P, Q ∈ H+ is (β-)detectable. Then one easily checks that the operator R : Hn → Hn defined by R(X) = A∗ X + XA + P − XQX ,

with

D = D+ = H n

satisfies Assumption 4.2.1. Hence, it follows from Theorem 4.2.2 and the Coroln laries 4.2.4 and 4.2.5 that there exists a stabilizing solution X+ ∈ int H+ , satisfying R(X+ ) = 0. If X0 ≥ X+ is any stabilizing matrix for R, i.e. σ(R3X0 ) ⊂ C− , then the Newton sequence starting at X0 converges monotonically to X+ . It is, however, not necessarily true that σ(RX ) ⊂ C− for all X ∈ [X+ , X0 ], as the following example demonstrates. 6 0 −1 6 , P = 0 and Q = I. Since A is stable, the Example 4.4.6 Let A = 0 −1 stabilizability and detectability assumptions are trivially satisfied. The Riccati operator and its derivative are given by R(X) = A∗ X + XA − X 2 , R3X (H) = (A∗ − X)H + H(A − X) . ˆ = 0 is the largest solution of the inequality R(X) ≥ 0, since Clearly X 0 6 2 −1 3 R0 is stable. Also e.g. X0 = 3I is stabilizing, whereas X = −1 1

0

4.4 Illustrative applications

ˆ but A − X = −3 7 satisfies X0 ≥ X ≥ X, 1 −2 stabilizing.

6

121

is not stable, i.e. X is not

If we abandon the detectability assumption, then R3X+ might be singular and the convergence of the Newton iteration may not be quadratic. In this case the use of a double Newton step is very advantageous. In fact, the conditions (4.16) and (4.17) are satisfied with L1 = 0, since the equation is quadratic, such that the remainder term is constant: φX (H) = −HQH for all X ∈ Hn . Hence 2(R(X1 ) − R(X0 )) = R3X0 (X1 − X0 ) + R3X1 (X1 − X0 ) , for all X0 , X1 ∈ Hn . In particular, if X1 = X+ and X+ − X0 ∈ Ker R3X+ , then X+ = X0 − 2(R3X0

−1

(R(X0 )) ,

i.e. the exact solution is obtained at once by a double Newton step (compare [94]). Finally, we mention that some of the properties of the Newton iteration can be illustrated by the simplest scalar examples. Example 4.4.7 (i) Let D = D+ = R and set f (x) = 1 − x2 . The (continuous-time algebraic Riccati-)equation f (x) = 0 has the solutions x− = −1 and x+ = 1. Since f 3 (x) = −2x, a number x ∈ D is stabilizing, if and only if x > 0. In particular, x+ is the unique stabilizing solution. The Newton sequence defined by xk+1 = (x2k + 1)/(2xk ) converges to x+ , if and only if x0 > 0, i.e. x0 is stabilizing. The sequence is monotonic, if x0 ≥ 1. Obviously, xk ≥ 1 for k ≥ 1, if x0 > 0. Hence, the sequence is guaranteed to be monotonic after the first step. But the smaller x0 ≥ 0 is chosen, the further x1 is away from x+ . (ii) Let D = R\{0}, D+ = R+ \{0} and set f (x) = −x+ 25 − x1 . The (discretetime algebraic Riccati-)equation f (x) = 0 has the solutions x− = 12 and x+ = 2. Since f 3 (x) = −1 + x12 , a number x ∈ D is stabilizing, if and only if |x| > 1. Hence x+ ∈ D+ is the unique stabilizing solution. From f 33 (x) = −2 x3 , we see that f is concave over D+ and convex over ] − ∞, 0[. But f is also D+ -concave on D, because for all x O= 0 and y > 0 we have 1 5 1 5 1 − + ( 2 − 1)(y − x) + y − + 2 x x 2 y 1 y > 0. = 2+ x y

f (x) + fx3 (y − x) − f (y) = −x +

Hence, at least for all |x0 | > 1 the Newton sequence xk+1 = converges to x+ = 2.

2xk −(5/2)x2k 1−x2k

5 Solution of the Riccati equation

In Chapter 2 we have discussed various optimal and worst-case stabilization problems for linear stochastic control systems and reformulated them in terms of rational matrix inequalities. Now we analyze these matrix inequalities and the corresponding matrix equations. To this end, we first introduce an abstract form of the Riccati operators met in the Sections 2.1 – 2.3, and the definite and the indefinite constraints mentioned in Remark 2.3.7. Recall that the LQ-stabilization problem and the Bounded Real Lemma lead to Riccati equations with definite constraints, while the disturbance attenuation problem involves an indefinite constraint. We have already noted in Remark 2.3.6 that the LQ-stabilization problem and the worst-case perturbation problem corresponding to the Bounded Real Lemma are special cases of the disturbance attenuation problem. From this point of view, it suffices to discuss the Riccati equation with indefinite constraint. Not surprisingly, however, the equation with definite constraint is easier to handle than the one with indefinite constraint. In fact, it fits exactly into the framework developed in the previous chapter, while the indefinite constraint does not match with the concavity assumptions invented in Section 4.1. Hence we discuss the definite case independently; we derive criteria for the existence of stabilizing solutions and their dependence on parameters. Motivated by results for standard Riccati equations (which we review briefly in Section 5.1.5), we apply a duality transformation to tackle the indefinite case. Thus, we obtain a dual Riccati operator, which also fits into the framework of the previous chapter. The proof that the dual operator possesses the required concavity properties constitutes the main technical obstacle in the present chapter. Using this result, we can derive criteria for the existence of stabilizing solutions and their dependence on parameters, which resemble the criteria for standard Riccati equations. Moreover, since our method is constructive, we can approximate these solutions numerically by Newton’s method or a modification of it. In fact, under fairly general detectability conditions, we even derive means to find stabilizing matrices needed to start the Newton

T. Damm: Rational Matrix Equations in Stochastic Control, LNCIS 297, pp. 123–179, 2004. Springer-Verlag Berlin Heidelberg 2004

124

5 Solution of the Riccati equation

iteration. These results are illustrated in Section 5.5 by numerical examples based on the applications in Section 1.9.

5.1 Preliminaries In the following we introduce a general framework for constrained Riccati equations and inequalities. This framework is aligned with the matrix equations developed in Chapter 2. From our point of view, a Riccati operator is always associated to stochastic linear control system and a certain cost functional – and vice versa. We have to distinguish between control systems with just one input vector and systems with two independent input vectors. If there is just one input the control problem lies in either minimizing or maximizing the cost functional. If there are two independent inputs, then one input is to be chosen to minimize the maximum cost achievable by the other input. As we have mentioned in Remark 2.3.7(ii) this is also called a two player game. The problems with just one input lead to Riccati equations with a definite constraint, while the problems with two inputs lead to Riccati equations with indefinite constraint. In both cases the Riccati operator has basically the same form, and it is the constraint, which tells us, whether we are solving a one-player or a two-player problem. Of course, we can always introduce a trivial inactive second player into a one-player game. In the same way, we can interpret every definite constraint as an indefinite one. To develop a general theory for Riccati equations with indefinite constraint, we will, however, have to rely on certain definiteness assumptions (Assumption 5.1.3; compare also the regularity assumption of Definition 2.3.3). Hence it follows that not every Riccati equation with definite constraint fits into this theory. Therefore we will discuss the two situations independently. 5.1.1 Riccati operators and the definite constraint X ∈ dom+ R Definition 5.1.1 Let n ∈ N. We say that an operator R, mapping some subset of Hn to Hn is a Riccati operator on Hn , if the following hold. For certain numbers s, N ∈ N and all j ∈ {1, 2, . . . , N }, there exist matrices A, Aj0 ∈ Kn×n ,

B, B0j ∈ Kn×s ,

P0 ∈ Hn ,

Q0 ∈ H s ,

S0 ∈ Kn×s . (5.1)

These data define affine linear operators P : Hn → Hn , S : Hn → Kn×s , and Q : Hn → Hs by P (X) = A∗ X + XA +

N O

j Aj∗ 0 XA0 + P0 ,

j=1

S(X) = XB +

N O j=1

j Aj∗ 0 XB0 + S0 ,

5.1 Preliminaries

Q(X) =

N O

125

B0j∗ XB0j + Q0 .

j=1

Then R is well-defined on its domain

R dom R = {X ∈ Hn R det Q(X) O= 0} ,

and for all X ∈ dom R it is given by R(X) = P (X) − S(X)Q(X)−1 S(X)∗ . Moreover, we define the target set dom+ R ⊂ dom R as R dom+ R = {X ∈ dom R R Q(X) > 0} , and call the requirement X ∈ dom+ R a definite constraint on X. Henceforth, whenever we speak of a Riccati operator R, we tacitly assume a set of data (5.1) to be given. In particular, it makes sense to speak of the control system corresponding to R, which has the form dx(t) = Ax(t) dt +

N O j=1

Aj0 x(t) dwj (t) + Bu(t) dt +

N O

B0j u(t) dwj (t) . (5.2)

j=1

Sometimes it is useful to arrange the matrices P (X), S(X), and Q(X) in a Hermitian block matrix. Then we consider the affine linear matrix operator 0 6 P (X) S(X) X L→ = Π(X) + Λ(X) + M ∈ Hn+s , S(X)∗ Q(X) where Π, Λ : Hn → Hn+s , and M ∈ Hn+s , are given by 0 6 P0 S0 M = , S0∗ Q0 0 ∗ 6 A X + XA XB Λ(X) = , B∗X 0 N 0 j∗ j6 O A0 XAj0 Aj∗ 0 XB0 Π(X) = . B0j∗ XAj0 B0j∗ XB0j j=1 We observe that Π is a completely positive operator and recall that = D R(X) = S (Π(X) + Λ(X) + M )/Q(X) for X ∈ dom R. If Λ and Π are fixed (i.e. the control system (5.2) is fixed), we also write R = RM to emphasize the dependence of the operator R on the weight matrix M .

126

5 Solution of the Riccati equation

5.1.2 The indefinite constraint X ∈ dom± R Let R be a Riccati operator on Hn with the given data (5.1). Assume that numbers m, ^ ∈ N are given, such that s = m + ^, and partition the given matrices (for j = 1, . . . , N ) as B = [B2 , B1 ] j j , B10 ] B0j = [B20 S0 = 0 [S20 , S10 ] 6 Q20 S30 Q0 = ∗ S30 Q10

with B2 ∈ Kn×m , B1 ∈ Kn×L , j j with B20 ∈ Kn×m , B10 ∈ Kn×L , n×m with S20 ∈ K , S10 ∈ Kn×L ,

(5.3)

with Q20 ∈ Hm , Q10 ∈ HL , S30 ∈ Km×L .

Analogously, we partition the operators S and Q in the form S(X) = [S2 (X), S1 (X)] with S2 (X) ∈ Kn×m , S1 (X) ∈ Kn×L 0 6 Q2 (X) S3 (X) with Q2 (X) ∈ Hm , Q1 (X) ∈ HL , Q(X) = ∗ S3 (X) Q1 (X) S3 (X) ∈ KL×m , such that PN j∗ j XB20 + Q20 , Q2 (X) = j=1 B20 PN j∗ j Q1 (X) = j=1 B10 XB10 + Q10 .

(5.4)

Definition 5.1.2 For Q1 and Q2 given by (5.4), we set R dom± R = {X ∈ dom R R Q2 (X) < 0 and Q1 (X) > 0} and call the requirement X ∈ dom± R an indefinite constraint on X. Henceforth, whenever we are given a Riccati-operator R on Hn and consider the set dom± R we tacitly think of a partitioning (5.3) of the given data. In particular, this partitioning specifies a stochastic control system with two inputs of the form dx(t) =

Ax(t) dt +

N O

Aj0 x(t) dwj (t)

j=1

+ B1 v(t) dt +

N O

j B10 v(t) dwj (t)

j=1

+ B2 u(t) dt +

N O

j B20 u(t) dwj (t) .

j=1

For the matrix M and the operators Λ and Π we have the partitioning

(5.5)





5.1 Preliminaries

127

P0 S20 S10 ∗ Q20 S30  , M =  S20 ∗ ∗ S10 S30 Q10   ∗ A X + XA XB2 XB1 0 0  , B2∗ X Λ(X) =  0 0 B1∗ X  j∗ j j∗ j j  Aj∗ A0 XA0 A0 XB20 N 0 XB10 O  B j∗ XAj B j∗ XB j B j∗ XB j  . Π(X) = 20 0 20 20 20 10 j=1 B j∗ XAj B j∗ XB j B j∗ XB j 10 0 10 20 10 10 By the quotient formula of Lemma A.2 we can write R as the double Schur complement = D R(X) = S Π(X) + Λ(X) + M/Q(X) = = DN = DD = S S Π(X) + Λ(X) + M/Q1 (X) S Q(X)/Q1 (X) . (5.6) 5.1.3 A definiteness assumption One of our aims is to solve the Riccati equation R(X) = 0 with the indefinite constraint X ∈ dom± R. To achieve this, we will have to impose the following definiteness assumptions. 0 Assumption 5.1.3 Let (i)

6 P0 S20 ≤ 0, (ii) Q20 < 0, and (iii) Q10 > 0. ∗ Q20 S20

5.1.4 Some comments It is easy to see that the abstract Riccati operator and the definite and indefinite constraints cover the special cases from Chapter 2. Remark 5.1.4 (i) Consider the situation of the LQ-control problem in Section 2.1 and the Riccati operator R defined in (2.3) together with the target set dom+ R defined in (2.4). Obviously, this is a Riccati operator with definite constraint in the sense of Section 5.1.1 with s = m and M > 0. Later it will be useful to observe that in this case the equation R(X) = 0 with X ∈ dom+ R can be transformed to an equation with indefinite constraint satisfying Assumption 5.1.3. To this end, we define PN j P˜ (X) = −P (−X) = A∗ X + XA + j=1 Aj∗ 0 XA0 − P0 , PN j ˜ S(X) = −S(−X) = XB + j=1 Aj∗ 0 XB0 − S0 , P N j∗ j ˜ Q(X) = −Q(−X) = j=1 B0 XB0 − Q0 ,

128

5 Solution of the Riccati equation

˜ = −X we have such that with X ˜ X) ˜ = P˜ (X) ˜ − S( ˜ X) ˜ Q( ˜ X) ˜ −1 S( ˜ X) ˜ ∗ = −R(X) . R( ˜ X) ˜ < 0. If we use the formal Obviously Q(X) > 0 is equivalent to Q( partition 0 6 ˜ 2 (X) ˜ S˜3 (X) ˜ Q ˜ ˜ Q(X) = ˜ ˜ ∗ ˜ ˜ S3 (X) Q1 (X) ˜ 2 (X) ˜ = Q( ˜ X) ˜ and the empty matrices S˜3 (X) ˜ and Q ˜ 1 (X) ˜ then with Q X ∈ dom+ R R ˜ = {X ˜ ∈ dom R ˜ R Q2 (X) ˜ ∈ dom± R ˜ < 0 and Q1 (X) ˜ > 0} . ⇐⇒ X Hence we have the following equivalence of the equation with definite constraint and an equation with indefinite constraint: (R(X) = 0 and X ∈ dom+ R) ˜ . ˜ X) ˜ = 0 with X ˜ = −X ∈ dom± R) ⇐⇒ (R( ˜ 20 = −Q0 Note that Assumption 5.1.3 holds for P˜0 = −P0 , S˜20 = −S0 , Q ˜ ˜ and the empty matrices S10 and Q10 . (ii) Similarly, the situation of the Bounded Real Lemma from Section 2.2 fits into both the framework of the definite and the indefinite constraint. Obviously the Riccati operator Rγ from (2.11) and the target set dom+ Rγ from (2.12) are of the type defined in Section 5.1.1 with s = ^. But we can also interpret dom+ Rγ as an indefinite constraint satisfying Assumption 5.1.3, if we set m = 0 and use the formal partition 0 6 Q2 (X) S3 (X) Qγ (X) = S3 (X)∗ Q1 (X) with Q1 (X) = Qγ (X) and the empty matrices Q2 (X) and S3 (X). Note again that Assumption 5.1.3 holds with P0 = −C ∗ C, S10 = −C ∗ D1 , Q10 = γ 2 I − D1∗ D1 , and the empty matrices S20 and Q20 . (iii) The situation of the disturbance attenuation problem in Section 2.3 is our motivating problem for a Riccati operator with indefinite constraint. It is quite easy to guess that in fact our general notation and Assumption 5.1.3 have been inspired by this application. To recover it in the general framework, we just have to set P0 = −C ∗ C , S0 = [S20 , S10 ] = −C ∗ [D2 , D1 ], 0 6 0 6 Q20 S30 −D2∗ D2 −D2∗ D1 Q0 = = . ∗ S30 Q10 −D1∗ D2 γ 2 I − D1∗ D1

0

6

5.1 Preliminaries

129

P0 S20 ≤ 0. By the regularity condition (Definition 2.3.3) ∗ S20 Q20 we have Q20 < 0. The definiteness condition Q10 > 0 is indispensible for an X < 0 with Q1 (X) = Qγ1 (X) > 0 to exist (which is what we are looking for). Hence Assumption 5.1.3 is a natural requirement in this context. (iv) Clearly, not every Riccati equation with definite constraint can be transformed to a Riccati equation with indefinite constraint satisfying Assumption 5.1.3. This situation arises e.g. in LQ-control problems with indefinite state- and input-weight cost like in [26, 218]. (v) The difference between the definite and the indefinite constraint can also be seen if we consider linear matrix inequalities. By Lemma A.2 we have Obviously

In R(X) + In Q(X) = In(Λ(X) + Π(X) + M ) . Hence, for X ∈ dom+ R we have R(X) ≥ 0 ⇐⇒ Π(X) + Λ(X) + M ≥ 0 , while X ∈ dom± R implies R(X) ≥ 0 , rk R(X) = r ⇐⇒ In(Π(X) + Λ(X) + M ) = (m + r, ^, n − r) . In other words, the constrained Riccati inequality X ∈ dom+ R, R(X) = 0 is equivalent to a linear matrix inequality in Hn+s , while X ∈ dom± R, R(X) = 0 is equivalent to a more general inertia condition in Hn+s . (vi) We recall an identity from the proof of Theorem 2.3.4. Let R be a Riccati operator, X ∈ dom R and let the given matrices be partitioned as in Section 5.1.2. Like in (2.25), (2.22), and (2.23) we set Pˆ (X) = P (X) − S1 (X)Q1 (X)−1 S1 (X)∗ , ˆ S(X) = S2 (X) − S1 (X)Q1 (X)−1 S3 (X)∗ , ˆ Q(X) = Q2 (X) − S3 (X)Q1 (X)−1 S3 (X)∗ , −1 ˆ ˆ S(X)∗ , F = F (X) = −Q(X)

PF (X) = (A∗ + F ∗ B2∗ )X + X(A + B2 F ) P i∗ ∗ i∗ i i + N i=1 (A0 + F B20 )X(A0 + B20 F ) ∗ ∗ ∗ + P0 + F S20 + S20 F + F Q20 F P i∗ ∗ i∗ i ∗ SF (X) = XB1 + N i=1 (A0 + F B20 )XB10 + S10 + F S30 . Then, in accordance with (5.6), we have 0 6∗ 0 60 6 ˆ I Pˆ (X) S(X) I R(X) = ∗ ˆ ˆ F F S(X) Q(X) = PF (X) − SF (X)Q1 (X)−1 SF (X)∗ .

(5.7)

130

5 Solution of the Riccati equation

In particular, the inequalities Q1 (X) > 0 and R(X) ≥ 0 together imply PF (X) ≥ 0. We will need this observation in Section 5.3.2, where we discuss solutions to the disturbance attenuation problem. We note a simple consequence of Assumption 5.1.3 concerning the evaluation of R at 0 ∈ Hn . Lemma 5.1.5 Assume the situation of Section 5.1.2 including Assumption 5.1.3. Then 0 ∈ dom± R and R(0) ≤ 0 with dim Ker R(0) = dim Ker M . Proof: Obviously 0 ∈ dom± R. If we set d = dim Ker M , then, by Lemma A.2 we can express the inertia of M in two ways: In M = In Q0 + In S(M/Q0 ) = (^, m, 0) + In R(0) = In Q10 + In S(M/Q10 ) = (^, 0, 0) + (0, n + m − d, d) . Hence In R(0) = (0, n − d, d), i.e. R(0) ≤ 0 and dim Ker R(0) = d.

"

In the following sections we show, how the results from Chapter 4 can be applied to solve equations of the form R(X) = 0. It is easy to see that the derivative of R is resolvent positive. Moreover, R is dom+ R-concave on dom R. Hence, as will be seen, we can apply the results from Chapter 4 directly in the definite case. As for the indefinite constraint, however, R is not dom± R-concave on dom R in general. Therefore, we will have to carry out a certain duality transformation of R before we can apply our results on the Newton iteration. This is the point, where Assumption 5.1.3 comes into play. To motivate the further considerations we take a look at standard Riccati equations at first. 5.1.5 Algebraic Riccati equations from deterministic control We briefly review some results on standard Riccati equations from deterministic control theory. In particular we point both at analogies and differences between the Riccati equations from deterministic and stochastic control. First of all we recall from Remark 2.3.6 that the equation R(X) = 0 reduces to the continuous-time algebraic Riccati equation (CARE) ∗ A∗ X + XA + P0 − XBQ−1 0 B X = 0

(5.8)

if Π = 0 and S0 = 0, and to the discrete-time algebraic Riccati equation (DARE) A0 ∗ XA0 − X + P0 − A0 ∗ XB0 (B0 ∗ XB0 + Q0 )

−1

B0 ∗ XA0 = 0

(5.9)

if A = − 21 I, B = 0, S0 = 0, and N = 1. We concentrate on stabilizing solutions of the CARE (5.8) and cite a result, which is central both for the analysis and the numerical solution of Riccati equations (e.g. [201, 34, 19, 145]).

5.1 Preliminaries

131

Theorem 5.1.6 Let Q0 > 0. Then the following are equivalent: (i)

The CARE (5.8) has a stabilizing solution X+ , i.e. σ(A−BQ0−1 B ∗ X+ ) ⊂ C− . In this event, X+ is the greatest solution of the inequality ∗ A∗ X + XA + P0 − XBQ−1 0 B X ≥ 0 .

If P0 ≥ 0 and (A, P0 ) is observable, then X+ > 0. (ii) The pair (A, B) is stabilizable and there exists a solution to the strict inequality ∗ A∗ X + XA + P0 − XBQ−1 0 B X > 0 .

In this event X+ is the limit of the Newton sequence defined by (P0 + Xk BQ0−1 B ∗ Xk ) , Xk+1 = −L−1 A−BQ−1 B ∗ X 0

k

∗ if σ(A − BQ−1 0 B X 0 ) ⊂ C− . (iii) The pair (A, B) is stabilizable and the Hamiltonian matrix 0 6 ∗ A −BQ−1 0 B H= −P0 −A∗

does not have any purely imaginary eigenvalues. In this event X+ = X1 X2−1 , where [X1∗ , X2∗ ]∗ spans the invariant subspace of H corresponding to the n eigenvalues with negative real part. (iv) The pair (A, B) is stabilizable and the frequency domain condition B ∗ (−iωI − A∗ )−1 P0 (iωI − A)−1 B + Q0 > 0 is valid for all ω ∈ R. Remark 5.1.7 (i) Among the criteria (ii), (iii), and (iv) only (ii) seems to offer an approach to analyze and solve the Riccati equation from stochastic control. While it is natural to apply Newton’s method to the nonlinear matrix equation R(X) = 0, a direct generalization of the Hamiltonian H or the transfer function (iωI − A)−1 B to the stochastic case is not yet available. It would, of course, be rewarding to develop such concepts. (ii) In the theorem we assume Q0 > 0. If Q0 is indefinite, the assertions do not necessarily hold. But if e.g. P0 ≤ 0 one can multiply equation (5.8) from both sides by X −1 and replace X −1 by −Y to obtain the dual equation ∗ −Y A∗ − AY − Y (−P0 )Y − BQ−1 0 B = 0 .

It is straightforward to reformulate the theorem for the dual equation. Note that the stabilizability condition on (A, B) becomes a detectability condition on (−A, P0 ), or – in the terminology of Remark 1.8.6 – an

132

5 Solution of the Riccati equation

anti-detectability condition on (A, P0 ). The term dual Riccati equation in this context has been used e.g. in [103]. We will apply the same duality transformation to solve the indefinite Riccati equation from stochastic control. In this case, however, the dual operator is not a Riccati operator in the sense of Definition 5.1.1. But again we will observe a duality between stabilizability and anti-detectability conditions. 5.1.6 A duality transformation Let R be a Riccati operator in the sense of Definition 5.1.1. Then we define another rational matrix operator R G : dom G = {Y R det Y O= 0, and X = −Y −1 ∈ dom R} → Hn by the relation G(Y ) = Y R(−Y −1 )Y .

(5.10)

In analogy to the deterministic case, we will call G the dual Riccati operator and the equation G(Y ) = 0 the dual Riccati equation. If X is a nonsingular solution of the Riccati equation, then, obviously, X = −Y −1 solves the dual Riccati equation. The explicit form of G is G(Y ) = Y (P (−Y −1 ))Y − Y S(−Y −1 )Q(Y −1 )S(−Y −1 )∗ Y = −Y A∗ − AY −

N O

−1 j Y Aj∗ A0 Y + Y P0 Y 0 Y

(5.11)

j=1 N D = O −1 j − B+ Y Aj∗ Y B − Y S 0 0 0 j=1

=

× Q0 −

N O

B0j∗ Y −1 B0j

N D−1 = D O B+ B0j∗ Y −1 Aj0 Y − S0 Y .

j=1

j=1

We make some simple observations. Lemma 5.1.8 Let Y ∈ dom G and X = −Y −1 . Then the following hold: R(X) = XG(−X −1 )X. @ G (>) (>) (ii) G(Y ) = 0 ⇐⇒ R(X) = 0 . (i)

(iii) If R(X) = 0, then σ(GY 3 ) = σ(RX 3 ).

5.1 Preliminaries

133

Proof: The statements (i) and (ii) are obvious. Assertion (iii) follows from GY 3 (H) = HR(X)Y + Y R(X)H + Y RX 3 (Y −1 HY −1 )Y = Y RX 3 (Y −1 HY −1 )Y ,

(5.12)

because the operator H L→ Y RX 3 (Y −1 HY −1 )Y is similar to H L→ RX 3 (H); in the notation of Kronecker products the similarity transformation is given " by Y ⊗ Y −1 . It is a legitimate question, why the equation G(Y ) = 0 should be easier to solve than R(X) = 0. Unlike in the situation of Remark 5.1.7, the dual Riccati operator G is not a Riccati operator in the sense of Definition 5.1.1. But if Assumption 5.1.3 holds, we will see that G has the concavity properties we need. To anticipate this result, we might consider the special case, where all B0j and S0 vanish and B = I. Like in the deterministic case, the concavity of R depends on the inertia of Q0 , while the concavity of G is independent of Q0 . Hence it suffices to study the Y L→ Y P0 Y , which is concave if P operator j∗ −1 j A0 Y , which we have already P0 ≤ 0, and the operator Y L→ N j=1 Y A0 Y n seen in Lemma 4.4.2 to be H+ -concave. 5.1.7 A regularity transformation In our applications, Q0 is regular. Under this assumption we can transform the Riccati operator by some congruence transformation equivalently to another Riccati operator ˜ X) ˜ ∗, ˜ = P˜ (X) ˜ − S( ˜ X) ˜ Q( ˜ X) ˜ −1 S( ˜ X) R( with ˜ = M

0

˜ P˜ (0) S(0) ˜ ∗ Q(0) ˜ S(0)

6

0 =

D1 0 0 D2

(5.13)

6 ,

(5.14)

∗ where D1 and D2 are the Sylvester normal forms of P0 − S0 Q−1 0 S0 and Q0 , respectively. ∗ ∗ ∗ To this end let U1 (P0 − S0 Q−1 0 S0 )U1 = D1 and U2 Q0 U2 = D2 with nonsingular matrices U1 and U2 . Then 6 60 60 6 0 0 ˜ S( ˜ X) ˜ U1∗ 0 P˜ (X) P (X) S(X) U1 −S0 Q−1 0 = ∗ ∗ ˜ X) ˜ ∗ Q( ˜ X) ˜ S(X)∗ Q(X) −Q−1 0 U2 S( 0 S0 U 2

with ˜ = U1 P (X)U ∗ − S0 Q−1 S(X)∗ U ∗ − U1 S(X)Q−1 S ∗ P˜ (X) 1 1 0 0 0 −1 ∗ + S0 Q−1 0 Q(X)Q0 S0

˜ +X ˜ A˜ + = A˜∗ X

N O i=1

˜ ˜i ˜ A˜i∗ 0 X A0 + P0 ,

134

5 Solution of the Riccati equation

˜ X) ˜ = U1 S(X)U ∗ − S0 Q−1 Q(X)U ∗ = XB ˜ + S( 2 2 0

N O

˜ ˜i A˜i∗ 0 X B0 ,

i=1

˜ X) ˜ = U2 Q(X)U ∗ = Q( 2

N O

˜B ˜i + Q ˜ i∗ X ˜0 , B 0 0

i=1

˜ = X A˜ =

˜ B A˜i0 ˜0i B P˜0

= = = =

˜0 = Q

U1 XU1∗ ∗ ∗ U1−∗ (A − BQ−1 0 S0 )U1 , U1−∗ BU2∗ , ∗ ∗ U1−∗ (Ai0 − B0 Q−1 0 S0 )U1 , −∗ i ∗ U 1 B0 U 2 , ∗ ∗ U1 (P0 − S0 Q−1 0 S0 )U1 = U2 Q0 U2∗ = D2 ,

D1 ,

as one verifies by insertion. Moreover, the transformation formula in Lemma A.2 yields ˜ X) ˜ = U1 R(X)U ∗ , R( 1 such that we have a one-to-one correspondence between the solution sets of both Riccati equations and inequalities, respectively. Hence, without loss of generality, we can assume that M has the normal form (5.14). We will, however, formulate all our results with the original data. Only in some examples we use the normal form.

5.2 Analytical properties of Riccati operators In this section we analyze the Riccati operator R and the dual Riccati operator G in view of the Assumptions 4.2.1 and 4.2.10. In particular, we show that the derivatives of R and G are resolvent positive and we characterize cases where R and G are concave. The following is obvious. Proposition 5.2.1 Let R be a Riccati operator and G its dual operator. Then R and G are analytical and their derivatives are locally bounded on dom R and dom G, respectively. 5.2.1 The Riccati operator R Let R be a Riccati operator in the sense of Definition 5.1.1. To condense the notation we sometimes write PX , SX , and QX instead of P (X), S(X), and Q(X), respectively. Moreover, for a given X in dom R, we set 0 0 6 6∗ I I −1 ∗ Π(·) SX and ΠX (·) = . AX = A − BQX ∗ ∗ −Q−1 −Q−1 X SX X SX

5.2 Analytical properties of Riccati operators

135

Theorem 5.2.2 Let X ∈ dom R, Y ∈ dom+ R. The derivative of R at X is given by

(i)

R3X (H) = LAX (H) + ΠX (H) .

(ii) (iii) (iv) (v) (vi) (vii)

In particular, R3X is the sum of a Lyapunov operator and a completely positive operator and 0 hence resolvent 6∗ 0 positive. 6 I I M R(X) = R3X (X) + −1 ∗ −1 ∗ . −QX SX −QX SX 3 R(Y ) − R(X) − RX (Y − X) −1 −1 ∗ −1 ∗ = −(SY Q−1 Y − SX QX )QY (QY SY − QX SX ). R is dom+ R-concave on dom R. n . dom+ R = dom+ R + H+ If Q0 > 0, then 0 ∈ dom+ R. If, in addition, M ≥ 0 then R(0) ≥ 0. Assume 0 ∈ dom R and R(0) ≤ 0. If (R30 , R(0)) is detectable, then ∃X > 0, ε > 0 : ∀α ∈ ]0, ε] : R(αX) < 0 .

Proof: (i) By the standard rules of calculus, we have −1 3 ∗ ∗ 3 3 R3X (H) = PX (H) − SX (H)Q−1 X SX − SX QX SX (H)

−1 ∗ 3 + SX Q−1 X QX (H)QX SX 0 6∗ = 6 D0 I I Λ(H) + Π(H) = , −1 ∗ ∗ −QX SX −Q−1 X SX

where it is easily seen that

0

LA−BQ−1 S ∗ (H) = X

(ii) This follows from R(X) −

R3X (X)

∗ −Q−1 X SX

X

0 =

0

6∗

I

Λ(H)

6

I

.

∗ −Q−1 X SX

0

6∗

I

I

6

(Λ(X) + Π(X) + M ) ∗ ∗ −Q−1 −Q−1 X SX X SX 0 6∗ 6 0 I I − . (Λ(X) + Π(X)) ∗ ∗ −Q−1 −Q−1 X SX X SX

(iii) We use (ii) to compute R(Y ) − R(X) − R3X (Y − X) = R(Y ) −

(5.15)

R3X (Y

)−

0

I

6∗

0

I

M ∗ ∗ −Q−1 −Q−1 X SX X SX 6∗ 0 60 6 I I PY SY = R(Y ) − −1 ∗ ∗ ∗ S Q −Q−1 −Q S Y Y X X X SX 0 6∗ 0 6 0 6 ∗ I I SY Q−1 Y SY SY =− ∗ ∗ −Q−1 −Q−1 SY∗ QY X SX X SX 0

−1 ∗ −1 ∗ −1 = −(SY Q−1 Y − SX QX )QY (QY SY − QX SX ) .

6

136

5 Solution of the Riccati equation

(iv) This follows from (iii), since the right-hand side is nonpositive definite for Y ∈ dom+ R. PN (v) By definition Q(X) = j=1 B0j∗ XB0j + Q0 . Hence Y ≥ X implies Q(Y ) ≥ Q(X). (vi) By definition 0 ∈ dom+ R ⇐⇒ Q(0) = Q0 > 0. If, in addition, M ≥ 0, ∗ then R(0) = P0 − S0 Q−1 0 S0 ≥ 0 by Lemma A.2. −1 ∗ (vii) Let F0 = −Q0 S0 . By Lemma 1.8.5 there exists an X > 0 such that D = R30 (X) + R(0) = LA+BF0 + Π(Aj +B j F0 ) (X) + R(0) < 0 . 0

0

For 1 > α > 0 we have R(αX) ≤ α(R(0) + R30 (X) + O(α)) which is negative, for sufficiently small α. " As in Chapter 4 (cf. Assumption 4.2.1), a matrix X ∈ dom R is called stabilizing (for R) if σ(R3X ) ⊂ C− and almost stabilizing, if σ(R3X ) ⊂ C− ∪iR. We call R (almost) stabilizable, if there exists an (almost) stabilizing matrix X for R. ∗ Corollary 5.2.3 If X is stabilizing for R, then F = −Q−1 X SX is a stabilizing feedback gain matrix for the stochastic control system

dx(t) = Ax(t) dt +

N O

Aj0 x(t) dwj (t) + Bu(t) dt +

j=1

N O

B0j u(t) dwj (t) .

j=1

Proof: By (5.15) R3X (H) = (A + BF )∗ X + X(A + BF ) 6 6∗ 0 j∗ N 0 j 60 O I I A0 XAj0 Ai∗ 0 XB0 + . F F B0∗ XAj0 B0j∗ XB0j i=1 Hence the stability of the closed-loop system follows from Theorem 1.5.3 and " the stability of R3X .

Remark 5.2.4 (i)

In the special case of the CARE (5.8) we have R3X = LA−BQ−1 B ∗ X 0

and X is stabilizing, if and only if σ(A − BQ0−1 B ∗ X) ⊂ C− . (ii) In the special case of the DARE (5.9) we have R3X (H) = −H + V ∗ HV where 0 6 I . V = [A0 B0 ] −1 (Q0 + B0∗ XB0 ) B0∗ XA0

5.2 Analytical properties of Riccati operators

137

So R3X is stabilizing, if and only if ρ(V ) < 1. We see that our concept of stabilization generalizes and unifies the concepts from continuous and discrete-time deterministic systems. O ∅. Then R is stabilizable, if and Proposition 5.2.5 Assume that dom+ R = only if (A, (Aj0 ), B, (B0j )) is stabilizable in the sense of Definition 1.7.1. Proof: If R is stabilizable, then so is (A, (Aj0 ), B, (B0j )) by Corollary 5.2.3. ˆ ∈ dom+ R. For arbitrary X ∈ dom R, set Z = X − X ˆ and Conversely let X ˆ ˆ 3 = (R)3 for all X ∈ dom R. By construction R(Z) = R(X). Obviously (R) Z X ˆ ˆ R(Z) = RM (Z) with 0 6 6 0 ˆ S(X) ˆ Pˆ0 Sˆ0 P (X) ˆ M = ˆ∗ ˆ . (5.16) = ˆ ˆ ∗ Q(X) S0 Q 0 S(X) ˆ 0 > 0 and (R) ˆ 3 does not depend on Pˆ0 , we can assume without loss Since Q Z ˆ of generality that M > 0. If (A, (Aj0 ), B, (B0j )) is stabilizable, then by Lemma 1.7.7 there exists an X0 > 0, such that ˆ 0) 0 ≥ R(X ˆ 3X (X0 ) + =R 0

0

I

ˆ 0 )∗ ˆ 0 )−1 S(X −Q(X

6∗

ˆ M

0

I

6

ˆ 0 )∗ ˆ 0 )−1 S(X −Q(X

ˆ 3X (X0 ) , >R 0 where the equation follows from Theorem 5.2.2(ii). ˆ 3 ) ⊂ C− by Theorem 3.6.1. Hence σ(R3X +Xˆ ) = σ(R X0 0

"

Finally, we draw a conclusion from Corollary A.4. ˜ let Proposition 5.2.6 For fixed Λ and Π and M ≥ M R(X) = RM (X) = P (X) − S(X)Q(X)−1 S(X)∗ ˜ −1 ˜ ˜ ˜ ˜ S(X)∗ . R(X) = RM (X) = P˜ (X) − S(X) Q(X) ˜ satisfies In Q(X) = In Q(X), ˜ ˜ If X ∈ dom R ∩ dom R then R(X) ≥ R(X) and D = D = −1 ˜ ˜ ˜ S(X)∗ . Ker R(X) − R(X) ⊂ Ker Q(X)−1 S(X)∗ − Q(X) ˜ ˜ 3 , R(X)) ˜ If, in addition, R(X) ≥ 0 and (R is β-detectable then (R3X , R(X)) X is β-detectable. Proof: The first part is a reformulation of Corollary A.4. It only remains n is an to prove the detectability assertion. To this end, assume that V ∈ H+ 3 ∗ eigenvector of (RX ) with the eigenvalue λ, and R(X)V = 0. Since R(X) ≥ ˜ ˜ R(X) ≥ 0, it follows that also R(X)V = 0. In particular

138

5 Solution of the Riccati equation

im V

1 2

= D ˜ = im V ⊂ Ker R(X) − R(X) D = −1 ˜ ˜ S(X)∗ . ⊂ Ker Q(X)−1 S(X)∗ − Q(X)

(5.17)

˜ 3 )∗ (V ), as the following lines show. With F = This implies (R3X )∗ (V ) = (R X −1 ∗ ˜ ∗ ˜ ˜ we have −Q(X) S(X) , F = −Q(X)−1 S(X) @0 6 0 6∗ 0 6 0 6∗ G I I I I ˜ 3X )∗ (V ) = (Λ + Π)∗ V . (5.18) − ˜ V ˜ (R3X )∗ (V ) − (R F F F F 1 1 The inclusion (5.17) implies F V = F˜ V and F V 2 = F˜ V 2 such that – indeed – the right-hand side of (5.18) vanishes. ˜ 3 )∗ with the same eigenvalue λ, and R(X)V = Thus, V is an eigenvector of (R X 3 ˜ ˜ 0. Since (RX , R(X)) is assumed to be β-detectable, it follows that λ ∈ C− . " Hence (R3X , R(X)) is β-detectable.

˜ Note that, by definition, the condition In Q(X) = In Q(X) holds, if X ∈ ˜ ˜ dom+ R ∩ dom+ R or X ∈ dom± R ∩ dom± R. 5.2.2 The dual operator G Assume the situation of Section 5.1.2. In Theorem 5.2.7 below we establish an analogue to Theorem 5.2.2 for the dual operator G. This theorem paves the way to a solution of the Riccati equation with indefinite constraint and the disturbance attenuation problem. All assertions of the theorem are valid under Assumption 5.1.3. But not all assertions of this section require all parts (i), (ii), and (iii) of Assumption 5.1.3. Hence, we do not assume any of these requirements, unless it is explicitly stated. Recall that the dual operator G to R is defined by G(Y ) = Y R(−Y −1 )Y . We define the target set

R n R X = −Y −1 ∈ dom± R} . dom+ G = {Y ∈ int H+

The operator G is more complicated than R, and the discussion below involves rather lengthy computations. A good choice of notation is essential. In addition to the conventions from Section 5.1.2 we use the partition 6 N 0 j∗ j6 O A0 XAj0 Aj∗ Π0 (X) Σ(X) 0 XB0 = Π(X) = Σ(X)∗ Π21 (X) B0j∗ XAj0 B0j∗ XB0j j=1  j∗   j j∗ j  A0 XAj0 Aj∗ N Π0 (X) Σ2 (X) Σ1 (X) 0 XB20 A0 XB10 O  B j∗ XAj B j∗ XB j B j∗ XB j  =  Σ2 (X)∗ Π2 (X) Σ3 (X)  = 20 20 0 20 20 10 ∗ ∗ j=1 B j∗ XAj B j∗ XB j B j∗ XB j Σ1 (X) Σ3 (X) Π1 (X) 10 20 10 10 10 0 0

5.2 Analytical properties of Riccati operators

139

with an obvious correspondence of the blocks. Furthermore, we set PY = −Y A∗ − AY − Y Π0 (Y −1 )Y + Y P0 Y , SY = B + Y Σ(Y −1 ) − Y S0 , QY = Q0 − Π21 (Y −1 ) . With these substitutions in (5.11) we can write G more compactly as ∗ G(Y ) = PY − SY Q−1 Y SY .

Finally, we define the abbreviations −1 ∗ AY = −A + SY Q−1 )) + Y (P0 − Π0 (Y −1 )) , Y (S0 − Σ(Y 0 0 6∗ 6 Y Y −1 −1 Π(Y HY ) ΠY (H) = , ∗ ∗ Q−1 Q−1 Y SY Y SY

which will allow us to write the derivative of G in a compact form as well. Theorem 5.2.7 Let Y ∈ dom G, Z ∈ dom+ G (in particular dom+ G O= ∅). (i)

(ii) (iii)

(iv) (v)

The derivative of G at Y is given by GY3 (H) = LA∗ (H) + ΠY (H) . In particular, GY3 is the sum of a Lyapunov operator and a completely positive operator and positive. 0 hence6∗resolvent 0 6 Y Y 3 G(Y ) = GY (Y ) − M . ∗ Q−1 Q−1 S∗ Y SY 0Y Y 6 P0 S20 Let Assumption 5.1.3(i) hold, i.e. ≤ 0. ∗ Q20 S20 Then G(Z) − G(Y ) − GY3 (Z − Y ) ≤ 0. In words, G is dom+ G-concave on dom G. Let Assumption 5.1.3(ii) hold, i.e. Q20 < 0. n n ⊂ int H+ . Then dom+ G = dom+ G + H+ Let Assumption 5.1.3 hold (in particular R(0) ≤ 0 by Lemma 5.1.5), and let (−R30 , R(0)) be detectable. Then G is stabilizable, and a stabilizing matrix Y > 0 can be found in the form −νX −1 where ν > 0, and X < 0 satisfies R30 (X) + R(0) < 0. In particular, if R(0) < 0, then for all Y0 > 0 there exists an ν0 > 0, 3 such that σ(GνY ) ⊂ C− for all ν ≥ ν0 . 0

Before we proceed with the technical proof, let us make some remarks. Remark 5.2.8 (i) As a consequence of Theorem 5.2.7 and Proposition 5.2.1 3 we obtain the following. Let f = G, D = dom G, D+ = dom+ G, TX = GX and let Assumption 5.1.3(i) and (ii) hold. Then the hypotheses (H1)–(H4) and (H7) of Assumption 4.2.1 are satisfied. If, in addition, Assumption 5.1.3(iii) holds and (−R30 , R(0)) is detectable, then also (H5) is satisfied.

140

5 Solution of the Riccati equation

(ii) Assume the regularity condition S0 = 0. Then detectability of the pair (−R30 , R(0)) is the same as anti-detectability of (A, (Aj0 ), −P0 ) in the terminology of Remark 1.8.6. Theorem 5.2.7(v) establishes an interesting relation between the stabilizability of the dual operator G and the antidetectability of the original system. . Proof: In products of the form V ∗ W V we sometimes write [ .. ] for the right factor, if it is the conjugate transpose of the left factor. (i) By the standard rules of calculus, we have PY3 (H) = −HA∗ − AH + H(P0 − Π0 (Y −1 ))Y + Y (P0 − Π0 (Y −1 ))H + Y Π0 (Y −1 HY −1 )Y SY3 (H) = H(Σ(Y −1 ) − S0 ) − Y Σ(Y −1 HY −1 ) Q3Y (H) = Π21 (Y −1 HY −1 ) . Thus −1 3 −1 3 −1 ∗ ∗ ∗ GY3 (H) = PY3 (H) − SY3 (H)Q−1 Y SY − SY QY SY (H) + SY QY QY (H)QY SY

= −AH − HA∗ + H(P0 − Π0 (Y −1 ))Y + Y (P0 − Π0 (Y −1 ))H ∗ −1 ∗ + H(S0 − Σ(Y −1 ))Q−1 HY −1 )Q−1 Y SY + Y Σ(Y Y SY −1 ∗ −1 + SY Q−1 )) H + SY Q−1 HY −1 )∗ Y Y (S0 − Σ(Y Y Σ(Y

∗ −1 + Y Π0 (Y −1 HY −1 )Y + SY Q−1 HY −1 )Q−1 Y SY Y Π21 (Y D = −1 ∗ )) + Y (P0 − Π0 (Y −1 )) H (5.19) = − A + SY Q−1 Y (S0 − Σ(Y = D ∗ −1 ))Y + H − A∗ + (S0 − Σ(Y −1 ))Q−1 Y SY + (P0 − Π0 (Y 6 0 6∗ 0 6 0 Y Y Π0 (Y −1 HY −1 ) Σ(Y −1 HY −1 ) . + ∗ ∗ Σ(Y −1 HY −1 )∗ Π21 (Y −1 HY −1 ) Q−1 Q−1 Y SY Y SY

(ii) We write (5.19) as 

∗ Y GY3 (H) = − AH − HA∗ +  H  (5.20) ∗ Q−1 S Y Y    Y Π0 (Y −1 HY −1 ) P0 − Π0 (Y −1 ) Σ(Y −1 HY −1 ) 0 S0 − Σ(Y −1 )   H  . ×  P0 − Π0 (Y −1 ) −1 −1 ∗ ∗ −1 ∗ ∗ Σ(Y HY ) S0 − Σ(Y ) Π21 (Y −1 HY −1 ) Q−1 Y SY If we insert H = Y and Π21 (Y −1 ) = Q0 − QY we obtain the formula (ii). (iii) We have to show that G is dom+ G-concave on dom G, i.e. G(Y ) − G(Z) + GY3 (Z − Y ) ≥ 0 . Letting H = Z − Y in (5.20) we obtain by a simple reordering of terms

5.2 Analytical properties of Riccati operators

141

∗ Y GY3 (Z − Y ) = A(Y − Z) + (Y − Z)A∗ +  Z − Y  ∗ Q−1 Y SY   Π0 (Y −1 (Z − Y )Y −1 ) P0 − Π0 (Y −1 ) Σ(Y −1 (Z − Y )Y −1 )  0 S0 − Σ(Y −1 ) P0 − Π0 (Y −1 ) × −1 −1 ∗ ∗ −1 ∗ −1 −1 Σ(Y (Z − Y )Y ) S0 − Σ(Y ) Π21 (Y (Z − Y )Y )   Y × Z  ∗ Q−1 Y SY ∗  Y = A(Y − Z) + (Y − Z)A∗ − 2Y P0 Y +  Z  QY−1 SY∗   Σ(Y −1 ZY −1 ) − S0 Π0 (Y −1 + Y −1 ZY −1 ) P0 − Π0 (Y −1 )  P0 − Π0 (Y −1 ) 0 S0 − Σ(Y −1 ) × −1 −1 ∗ −1 ∗ −1 −1 −1 −Y ) (Σ(Y ZY ) − S0 ) (S0 − Σ(Y )) Π21 (Y ZY   Y ×  Z  . ∗ Q−1 Y SY 

Hence G(Y ) − G(Z) + GY3 (Z − Y ) −1 ∗ = − AY − Y A∗ + Y (P0 − Π0 (Y −1 ))Y − SY Q−1 ))Q−1 Y (Q0 − Π21 (Y Y SY

∗ 3 + AZ + ZA∗ + Z(Π0 (Z −1 ) − P0 )Z + SZ Q−1 Z SZ + GY (Z − Y )   ∗   Y Y Π0 (Y −1 ZY −1 ) −Π0 (Y −1 ) Σ(Y −1 ZY −1 ) Π0 (Z −1 ) −Σ(Y −1 )   Z  =  Z   −Π0 (Y −1 ) −1 ∗ −1 −1 ∗ −1 ∗ ∗ Σ(Y ZY ) −Σ(Y ) Π21 (Y −1 ZY −1 ) Q Y SY Q−1 Y SY   ∗   Y Y −P0 P0 −S0 ∗ (5.21) +  Z   P0 −P0 S0   Z  + SZ Q−1 Z SZ . −1 ∗ −1 ∗ ∗ ∗ −S0 S0 −Q0 Q Y SY Q Y SY

∗ Now we factorize SZ Q−1 Z SZ in a similar fashion like the other summands. We define

˜ = Σ(·) − S0 , Σ(·) such that −1 ˜ SY = B + Y Σ(Y ) ˜ −1 ) − Y Σ(Y ˜ −1 ) in In the following computation, we replace SZ by SY + Z Σ(Z the first step and QY by QZ + Π21 (Z −1 )− Π21 (Y −1 ) = QZ + Π21 (Z −1 − Y −1 ) in the last step: −1 ∗ −1 ˜ ∗ −1 ∗ ˜ −1 )∗ Y ) Z − SY Q−1 SZ Q−1 Z Σ(Y Z SZ = SY QZ SY + SY QZ Σ(Z

142

5 Solution of the Riccati equation

˜ −1 )∗ Z ˜ −1 )Q−1 Σ(Z ˜ −1 )Q−1 S ∗ + Z Σ(Z + Z Σ(Z Y Z Z ˜ −1 )∗ Y − Y Σ(Y ˜ −1 )Q−1 Σ(Y ˜ −1 )Q−1 S ∗ − Z Σ(Z

(5.22) Y Z Z −1 ˜ −1 ˜ −1 −1 ∗ −1 −1 ∗ ˜ ˜ − Y Σ(Y )QZ Σ(Z ) Z + Y Σ(Y )QZ Σ(Y ) Y ∗  Y =  Z  ∗ Q−1 Y SY   ˜ −1 )Q−1 Σ(Y ˜ −1 )∗ Σ(Y p p Z ˜ −1 )∗ ˜ −1 )∗ Σ(Z ˜ −1 )Q−1 Σ(Z ˜ −1 )Q−1 Σ(Y  ×  −Σ(Z p Z Z −1 ˜ −1 ˜ −1 −1 ∗ −1 ∗ QY QZ Σ(Z ) QY QZ QY −QY QZ Σ(Y )   Y × Z  (5.23) ∗ S Q−1 Y Y ∗     ∗ ˜ −1 ) ˜ −1 ) Y −Σ(Y −Σ(Y  Q−1   ˜ −1 ) ˜ −1 ) =  Z    Σ(Z Σ(Z Z −1 ∗ −1 −1 −1 −1 Q Y SY Π21 (Z − Y ) Π21 (Z − Y )     0 0 S0 − Σ(Y −1 ) .   0 0 Σ(Z −1 ) − S0 +   ..  . ˜ −1 )∗ − S0∗ Q0 + Π21 (Z −1 − 2Y −1 ) S0∗ − Σ(Y −1 )∗ Σ(Z 

∗ Substituting this expression for SZ Q−1 Z SZ in formula (5.21) we obtain   ∗  Y Y G(Y ) − G(Z) + GY3 (Z − Y ) =  Z  Θ  Z  . ∗ ∗ Q−1 Q−1 Y SY Y SY

Here Θ = Θ1 + Θ2 + Θ3 with   Σ(Y −1 ZY −1 − Y −1 ) Π0 (Y −1 ZY −1 ) −Π0 (Y −1 )  , p Π0 (Z −1 ) Σ(Z −1 − Y −1 ) Θ1 =  −1 p p Π21 (Y ZY −1 − 2Y −1 + Z −1 )   ∗  ˜ −1 ) ˜ −1 ) Σ(Y Σ(Y   Q−1  , ˜ −1 ) ˜ −1 ) Θ2 =  −Σ(Z −Σ(Z Z −1 −1 −1 −1 Π21 (Z − Y ) Π21 (Z − Y )   −P0 P0 0 Θ3 =  P0 −P0 0  . 0 0 0 The remaining part of the proof amounts to verifying that Θ ≥ 0 for Z ∈ dom+ G. 0 6 Q2 (Z −1 ) S3 (Z −1 ) To estimate Θ2 we remember that QZ = with −1 −1 −1 S3 (Z

)

Q1 (Z

)

Q2 (Z −1 ) < 0 and Q1 (Z −1 ) > 0 for Z ∈ dom+ G. Therefore also the Schur

5.2 Analytical properties of Riccati operators

143

complement

D = ˆ := S QZ /Q2 (Z −1 ) = Q1 (Z −1 ) − S3 (Z −1 )∗ Q2 (Z −1 )−1 S3 (Z −1 ) Q

of QZ with respect to the left-upper block is positive definite. By the matrix inversion formula of Lemma A.2 we have 0 6 Q2 (Z −1 )−1 0 = Q−1 Z 0 0 0 0 6 6∗ −1 −1 Q2 (Z ) S3 (Z −1 ) ˆ −1 Q2 (Z −1 )−1 S3 (Z −1 ) Q + , I I which is the sum of a negative and a positive semidefinite matrix. Resubsti˜ and Π21 by their defining expressions we get tuting Σ    ∗ ˜ −1 ) ˜ −1 ) Σ(Y Σ(Y  Q−1   ˜ −1 ) ˜ −1 ) Θ2 =  −Σ(Z −Σ(Z Z −1 −1 −1 −1 Π21 (Z − Y ) Π21 (Z − Y )  ∗   Σ2 (Y −1 ) − S20 Σ1 (Y −1 ) − S10 0 6   −Σ2 (Z −1 ) + S20 −Σ1 (Z −1 ) + S10  Q2 (Z −1 )−1 0   ..    ≥   Π2 (Z −1 − Y −1 ) Σ3 (Z −1 − Y −1 )  0 0 . −1 −1 ∗ −1 −1 Σ3 (Z − Y ) Π1 (Z − Y )  ∗   Σ2 (Y −1 ) − S20 Σ2 (Y −1 ) − S20 −1  −Σ2 (Z −1 ) + S20   S20    Q2 (Z −1 )−1  −Σ2 (Z−1 ) + −1 ˜ = −1 −1  Π2 (Z − Y )   Π2 (Z − Y )  =: Θ2 . Σ3 (Z −1 − Y −1 )∗ Σ3 (Z −1 − Y −1 )∗ Hence ˜2 + Θ3 = S Θ ≥ Θ1 + Θ where



@0

G 6N ˜ + S˜ Θ1 + Θ3 Σ −1 (Z ) , − Q 2 ˜ ∗ + S˜∗ −Q2 (Z −1 ) Σ

 Σ2 (Y −1 )   −Σ2 (Z −1 )  ˜ =  Σ  Π2 (Z −1 − Y −1 )  , Σ3 (Z −1 − Y −1 )∗



 −S20  S20   S˜ =   0 , 0

and −Q2 (Z −1 ) = Π2 (Z −1 ) − Q20 > 0. We finally show that 6 0 ˜ + S˜ Σ Θ1 + Θ3 ˜ + S˜ Π2 (Z −1 ) − Q20 ≥ 0 , Σ which implies Θ ≥ 0 for Z ∈ dom+ G and therefore proves (iii). For the constant term we have

144

5 Solution of the Riccati equation



−P0 P0  P0 −P0 0 6  Θ3 S˜ 0 =  ∗  0 ˜ S −Q20  0 0 ∗ ∗ S20 −S20 since

0

P0 S02 S20 Q20

 0 0 −S20 0 0 S20   00 0   ≥ 0, 00 0  0 0 −Q20

6 ≤0.

i i i ) the remaining term can be written , B20 , B10 Setting Di := diag(Ai0 , Ai0 , B20 as

0

6 O N ˜ Σ Θ1 Di∗ Υ Di = ˜ ∗ Π2 (Z −1 ) Σ i=1

with

     Υ =   



  ∗ Y −1 Y −1  −Z −1   −Z −1   −1     Z − Y −1  Z  Z −1 − Y −1  Z −1 − Y −1 Z −1 − Y −1 /

Y −1 −Z −1 Z −1 − Y −1 Z −1 − Y −1



 Y −1  −Z −1   −1   Z − Y −1  Z −1 − Y −1 5

      .   

Z −1

For Z ∈ dom+ G we have Z −1 > 0, and obviously S(Υ/Z −1 ) = 0. Hence Υ ≥ 0, which completes our proof of (iii). (iv) By definition, Y ∈ dom+ G if and only if Y > 0, Q2 (−Y −1 ) < 0, and Q1 (−Y −1 ) > 0. Since Q2 (−Y −1 ) < Q20 < 0 for all Y > 0, and the mapping n Y L→ Q1 (−Y −1 ) > 0 is monotonically increasing on H+ , the assertion follows. (v) The assumption Q20 < 0, Q10 > 0 is equivalent to 0 ∈ dom± R. Since dom R is open, there exists an ε > 0, such that the closure of the ball B(0, ε) is contained in dom R. On this ball, R is analytic, and, by compactness, the second derivative is bounded. Hence there exists a number L > 0, such that for all X1 , X2 ∈ B(0, ε) 0 large enough, such that −ν −1 X ∈ B(0, ε). With Y = −X −1 formula (5.12) yields = D 3 GνY (ν −1 Y ) = Y 2R(ν −1 X) + (R3ν −1 X (−ν −1 X) Y

5.3 Existence and properties of stabilizing solutions

= = Y R(ν −1 X) + R(0) + R30 (ν −1 X) + O(ν −2 ) D + R3ν −1 X (−ν −1 X) Y = D = Y R(ν −1 X) + R(0) + O(ν −2 ) Y = D = Y (2 − ν −1 )R(0) + ν −1 (R(0) + R30 (X) + O(ν −1 )) Y .

145

The last term is strictly negative for sufficiently large ν > 0, since R(0) ≤ 0 (by Lemma 5.1.5) and R(0) + R30 (X) < 0. By Theorem 3.2.10, the inequality 3 3 (ν −1 Y ) < 0 implies σ(GνY ) ⊂ C− . GνY In particular, if R(0) < 0, then for arbitrary Y0 > 0 there exists a sufficiently small α > 0, such that R(0) + R30 (−αY0−1 ) < 0. Hence we can repeat the previous considerations with X = −αY0−1 . We find that −νX −1 = αν Y0 stabilizes G for all sufficiently large ν. "

5.3 Existence and properties of stabilizing solutions Now we are ready to apply the results from Chapter 4 to the different types of Riccati equations derived in Chapter 2. For both the definite and the indefinite case, we first clarify the relation between Riccati equations and inequalities. Then we discuss properties of the stabilizing solutions and their dependence on parameters. In particular, we consider the cases of LQ-control and the Bounded Real Lemma. Iterative methods to compute stabilizing solutions are examined in Section 5.4. The results of the present section have appeared partly in [46, 47, 48, 40]. 5.3.1 The Riccati equation with definite constraint Let R be a Riccati operator in the sense Definition 5.1.1. For brevity we sometimes write FX = −Q(X)−1 S(X)∗ , if X ∈ dom R. Our first result is an immediate consequence of Theorem 4.2.2 and Theorem 5.2.2. Theorem 5.3.1 Let R be stabilizable. Then the following implications hold. (i)

ˆ ≥0 ˆ ∈ dom+ R : R(X) ∃X =⇒

∃X+ ∈ dom+ R : R(X+ ) = 0, σ(RX+ ) ⊂ C− ∪ iR. and ∀X ∈ dom+ R with R(X) ≥ 0 : X+ ≥ X.

ˆ ∈ dom+ R : R(X) ˆ >0 (ii) ∃X ˆ ∈ dom+ R : R(X) ˆ ≥ 0, ⇐⇒ ∃X

146

5 Solution of the Riccati equation

ˆ is β-detectable (A + BFXˆ , (Aj0 + B0j FXˆ ), R(X)) ˆ σ(RX+ ) ⊂ C− . ⇐⇒ ∃X+ ∈ dom+ R : R(X+ ) = 0, X+ > X, Moreover, if X+ ∈ dom+ R satisfies R(X+ ) = 0, and σ(RX+ ) ⊂ C− , ˆ for all X ˆ ∈ dom+ R with R(X) ˆ > 0. then X+ > X Proof: By Proposition 5.2.1 and Theorem 5.2.2, Assumption 4.2.1 (H1)–(H5) and (H7) is satisfied with f = R, D = dom R, D+ = dom+ R and Tx = R3X . Hence, (i) follows from Theorem 4.2.2 and the continuous dependence of σ(R3X ) on X ∈ dom R, and (ii) follows from the Corollaries 4.2.5 and 4.2.7. " As an immediate consequence of Corollary 4.2.6, we have another necessary and sufficient criterion for the existence of stabilizing solutions. Corollary 5.3.2 The equation R(X) = 0 has a stabilizing solution X+ ∈ ˆ ∈ dom+ R such that X1 ≥ dom+ R, if and only if there exist matrices X1 , X 3 ˆ ˆ ˆ X, R(X1 ) ≤ 0, R(X) ≥ 0, and (RXˆ , R(X)) is β-detectable. Based on the notion of observability instead of β-detectability, we derive the following sufficient criterion for a matrix X1 to be stabilizing. ˆ ∈ dom+ R, such that X1 ≥ X, ˆ R(X) ˆ ≥ 0, Corollary 5.3.3 If X1 , X 3 ˆ R(X1 ) ≤ 0, and (A + BFX1 , R(X)) is observable, then σ(RX1 ) ⊂ C− and ˆ X1 > X. ˆ ≤ R3 (X ˆ − X1 ). Hence ˆ − X1 ) ≤ LA+BFX (X Proof: By concavity R(X) X1 1 ˆ we are in the situation of Theorem 3.6.1(k) with X = −(X − X1 ) and Y = ˆ It follows now from the equivalent statements (c) and (g) of Theorem −R(X). ˆ " 3.6.1 that σ(R3X1 ) ⊂ C− and X > 0, i.e. X1 > X. Below, we will analyze the dependence of the stabilizing solution X+ on the weight matrix 6 0 P0 S0 . (5.24) M= S0∗ Q0 First, let us note two results for the special cases corresponding to LQstabilization and the Bounded Real Lemma. In the situation of LQ-stabilization we have Q0 > 0 and M ≥ 0, and the following holds (compare also [212, 69] for the case, where all B0j vanish). Corollary 5.3.4 Assume that Q0 > 0, M ≥ 0, and R is stabilizable. Let −1 ∗ ∗ F0 = −Q−1 0 S0 and M0 = P0 − S0 Q0 S0 . Then the following hold: (i)

The equation R(X) = 0 has a greatest solution X+ ≥ 0, which is almost stabilizing, i.e. σ(R3X+ ) ⊂ C− ∪ iR.

5.3 Existence and properties of stabilizing solutions

147

(ii) If (A + BF0 , (Aj0 + B0j F0 ), M0 ) is β-detectable, then σ(R3X+ ) ⊂ C− . (iii) If (A + BF0 , M0 ) is observable then σ(R3X+ ) ⊂ C− and X+ > 0. Furthermore, in the cases (ii) and (iii), the matrix X+ is the unique nonnegative definite solution of the equation R(X) = 0, and the control u = FX+ x stabilizes system (5.2) and minimizes the cost-functional 6∗ 0 6 < ∞0 x(t; x0 , u) x(t; x0 , u) M dt . J(x0 , u) = E u(t) u(t) 0 ˆ = 0 ∈ dom+ R and R(0) ≥ 0. Proof: The assumptions guarantee that X Therefore, (i) and (ii) follow immediately from Theorem 5.3.1. Moreover, under the detectability assumption in (ii) it follows from Corollary 4.2.5, that a nonnegative definite solution of the equation R(X) = 0 is necessarily stabilizing and therefore coincides with X+ . By Theorem 5.2.2(ii) we have 0 6∗ 0 6 I I 3 M RX+ (X+ ) = − =: M+ ≤ 0 . FX+ FX+ By Lemma 1.8.7, the observability of (A + BF0 , M0 ) implies that also (A + BFX+ , M+ ) is observable. Hence (iii) follows from Theorem 3.6.1. Since we can repeat this argument with any nonnegative definite solutions of R(X) = 0 we find again that X+ is unique with this property. Moreover, if σ(R3X+ ) ⊂ C− then FX+ stabilizes the system, by Corollary 5.2.3 and minimizes the cost-functional by Corollary 2.1.2. " In the situation of the Bounded Real Lemma, we have Q0 > 0 and P0 ≤ 0, and the following holds. Corollary 5.3.5 Assume that P0 ≤ 0, Q0 > 0 and there exists a matrix ˆ ∈ dom+ R such that R(X) ˆ ≥ 0. Consider the statements X (a) (b) (c) (d) (e) (f )

LA + ΠA0 is stable, R30 is stable, ∃X ∈ dom+ R : R(X) = 0 and X ≤ 0, ∃X ∈ dom+ R : R(X) ≥ 0 and X ≤ 0, X ∈ dom+ R and R(X) ≥ 0 ⇒ X ≤ 0, X ∈ dom+ R and R(X) ≥ 0 ⇒ X < 0.

The following implications hold. (i) (a)⇒(e), (b)⇒(e), (b)⇒(c), and – trivially – (f )⇒(e)⇒(d), (c)⇒(d). (ii) If (A, (Aj0 ), P0 ) is β-detectable, then the assertions (a)–(e) are equivalent. (iii) If (A, P0 ) is observable, then the assertions (a)–(f ) are equivalent.

148

5 Solution of the Riccati equation

Proof: ‘(a)⇒(e)’: On dom+ R the following inequality holds by definition R(X) = P (X) − S(X)Q(X)−1 S(X)∗ ≤ P (X) = LA (X) + ΠA0 (X) + P0 (5.25) Thus R(X) ≥ 0 implies LA (X) + ΠA0 (X) ≥ −P0 whence X ≤ 0 if LA + ΠA0 is stable (by Theorem 3.2.10). ‘(b)⇒(e)’: By concavity ∗ 3 R(X) ≤ R(0) + R30 (X) = P0 − S0 Q−1 0 S0 + R0 (X),

X ∈ dom R. (5.26)

Thus, Q(0) = Q0 > 0 and R(X) ≥ 0 imply 0 ∈ dom+ R and R30 (X) ≥ −P0 whence X ≤ 0 if R30 is stable. ‘(b)⇒(c)’: This follows immediately from Theorem 5.3.1 and the implication ‘(b)⇒(e)’ just proven. Now let (A, (Aj0 ), P0 ) be β-detectable. To establish the equivalence of the assertions (a)–(e), it suffices to prove (d)⇒(a) and (d)⇒(b). ‘(d)⇒(a)’: If R(X) ≥ 0 for some X ∈ dom+ R, X ≤ 0, then in particular LA (X) + ΠA0 (X) ≥ −P0 ; hence (a) by Theorem 3.6.1. ‘(d)⇒(b)’: In view of (5.26), we only have to show that (A + BF0 , (Aj0 + B0j F0 ), M0 ) is β-detectable (with F0 , M0 from Corollary 5.3.3). This, however, follows easily from the fact that M0 X = 0 implies F0 X = 0. Now let (A, P0 ) be observable. To establish the equivalence of the assertions (a)–(f), it suffices to prove (d)⇒(a), (d)⇒(f), and (d)⇒(b). The first two implications follow (like above) from LA (X) + ΠA0 (X) ≥ −P0 and Theorem 3.6.1. The last follows (like above) from the fact that (A + BF0 , M0 ) is observable. The proof is complete. "

Remark 5.3.6 Roughly speaking, to apply Theorem 4.2.2 one needs a staˆ to the inequality bilizing matrix X0 and the existence of a solution X R(X) ≥ 0. The latter condition is trivially satisfied in the situation of ˆ = 0. In the situation of the Bounded the LQ-control problem with X Real Lemma, on the other hand, X0 = 0 must be a stabilizing matrix (at least under the detectability assumption). Hence, in either problem, one of these conditions is usually satisfied. Let us name further conditions that ensure the stabilizability of R. ˆ ∈ dom+ R ∩ Hn : Corollary 5.3.7 Assume that P0 ≤ 0, Q0 > 0 and ∃X − ˆ ≥ 0. If one of the following conditions holds, then R is stabilizable. R(X) ˆ > 0. (i) R(X) (ii) ∃X ≥ 0 : R(X) < 0. (iii) (A + BF0 , (Aj0 + B0j F0 ), M0 ) is detectable.

5.3 Existence and properties of stabilizing solutions

149

Proof: Since R(0) ≤ 0, it follows from Corollary 4.2.5 that both (i) and (ii) imply the stabilizability of R. By Theorem 5.2.2(vii), the detectability condition (iii) implies (ii). "

Parameter dependence of the stabilizing solution To analyze the dependence of the stabilizing solution X+ on the matrix M , it is convenient to introduce the following subsets of Hn+m (depending on Λ and Π but these are fixed). M+ = {M : dom+ RM O= ∅} , M0 = {M : RM is stabilizable} , M1 = {M : Λ(X) + Π(X) + M ≥ 0 is solvable in cl dom+ RM } , M2 = {M : RM (X) = 0 has a stabilizing solution in dom+ RM } . Proposition 5.3.8 (i) M0 O= ∅ ⇒ M0 ⊃ M+ (ii) Let M ∈ Hn+s , X ∈ dom RM . −M 3 3 Then −X ∈ dom R−M and (RM X ) = (R−X0 ) . In particular M0 = −M0 . n+m = int M1 . (iii) M0 O= ∅ ⇒ M2 = M2 + H+ Proof: (i): This is a reformulation of Proposition 5.2.5. (ii): If we replace M and X by −M and −X, then Q(X) and S(X) change their sign. Therefore, the product Q(X)−1 S(X)∗ is invariant, and so is the derivative of R. (iii): By the inertia formula of Lemma A.2, it is easily seen that int M1 = {M : ∃X ∈ dom+ RM such that RM (X) > 0} . Hence M2 = M+ ∩M0 ∩int M1 by Theorem 5.3.1. Obviously M+ ∩int M1 = int M1 = int M1 + Hn+s . Moreover, if M0 O= ∅ then M0 ⊃ M+ by (i). Thus M+ ∩ M0 = M+ = M+ + Hn+s , which proves M2 = int M1 = int M1 + Hn+s . "

Theorem 5.3.9 There exists a (real) analytic order-preserving function X+ : M2 → Hn such that X+ (M ) is the stabilizing solution of RM (X) = 0 for all M ∈ M2 . R Proof: Clearly D = {(M, X) ∈ Hn+m × Hn R QM (X) > 0} is non-empty and open in the real vector space Hn+m × Hn and the map G : D → Hn defined by

150

5 Solution of the Riccati equation

G : (M, X) L→ G(M, X) := RM (X) is (real) analytic. As a consequence, also the derivative ? F3 ∂G ∂G : (M, X) L→ (M, X) = RM X ∂X ∂X is an analytic map from D to L(Hn ). Now assume that M0 ∈ M2 and let X+ = X+ (M0 ) ∈ dom+ RM0 be the ∂G stabilizing solution of RM0 (X) = 0. Then (M0 , X0 ) ∈ D and ∂X (M0 , X0 ) = M0 3 (R )X0 is stable, in particular invertible. As a consequence of the implicit function theorem for analytic functions [55], there is an open ball B(M0 , ε0 ) in Hn+m such that for all M ∈ B(M0 , ε0 ) there exists a unique solution X(M ) ∈ Hn of RM (X) = 0 which depends analytically on M ∈ B(M0 , ε0 ). But then ? F3 ∂G (M, X(M )) = RM X(M) ∂X F3 ? is continuous (even analytic) on B(M0 , ε0 ) and since σ( RM0 X(M ) ) ⊂ C− 0 F3 ? there exists ε ∈]0, ε0 [ such that σ( RM X(M) ) ⊂ C− for all M ∈ B(M0 , ε). M L→

Hence X(M ) is a stabilizing solution of RM (X) = 0 for all M ∈ B(M0 , ε) and so B(M0 , ε) ⊂ M2 . The restriction of X+ (·) to B(M0 , ε) coincides with X(·) on B(M0 , ε). Therefore X+ (·) : M2 → Hn is analytic and by Theorem 5.3.11 below order-preserving. "

The continuity of X+ also leads to the following result for linear matrix inequalities. Corollary 5.3.10 If M ∈ M2 , then X+ (M ) is the largest solution of the linear matrix inequality Λ(X) + Π(X) + M ≥ 0 in cl dom+ R. Proof: By Remark 5.1.4(v) and Theorem 5.3.1 it is clear that X+ (M ) is the largest solution of the linear matrix inequality Λ(X) + Π(X) + M ≥ 0 ˆ ∈ cl dom+ RM be another solution of this inequality. in dom+ RM . Let X ˆ ≥ 0. Hence ˆ ∈ dom+ RM+εI and RM+εI (X) For arbitrary ε > 0 we have X ˆ By continuity also X+ (M ) ≥ X. ˆ X+ (M + εI) ≥ X. " Now we prove that X+ depends monotonically on M (cf. [207, 208, 166] for the deterministic case). ˆ ∈ dom+ RM0 Theorem 5.3.11 Let M1 ≥ M0 . If there exists a solution X to the inequality RM0 (X) ≥ 0 and RM1 is stabilizable, then there exists = D a M1 M 3 M 0 0 ˆ If (R ) , R (X) ˆ is greatest solution X+ to R (X) = 0 and X+ ≥ X. ˆ X

β-detectable, then X+ is stabilizing for RM1 . In particular, if M0 ∈ M2 , then M1 ∈ M2 and X+ (M0 ) ≤ X+ (M1 ).

5.3 Existence and properties of stabilizing solutions

151

Proof: By M1 ≥ M0 we have dom+ RM0 ⊂ dom+ RM1 and RM1 (X0 ) ≥ 0 by Remark 5.1.4(v). Thus, by Theorem 5.3.1, there exists a greatest solution X1 to RM1 (X) = 0 and X1 ≥ X0 . If the detectability assumption holds, then by Theorem 5.3.1 there exists an ˜ > 0. Again by M1 ≥ M0 and Remark ˜ ∈ dom RM0 such that RM0 (X) X M1 ˜ 5.1.4(v) we have R (X) > 0. Thus again by Theorem 5.3.1 the matrix X1 is stabilizing. " An analogous argument shows that X+ depends on M in a concave fashion. Theorem 5.3.12 Let M0 , M1 ∈ Hn+s be arbitrary, and set Mτ := (1 − τ )M0 + τ M1 for τ ∈ [0, 1]. Assume that for i = 0, 1 there exist solutions Xi ∈ dom+ RMi to RMi (X) = 0 and that RMτ0 is stabilizable for some τ0 ∈ ]0, 1[. Then there exists a greatest solution Xτ0 to RMτ0 (X) = 0 and Xτ0 ≥ (1 − τ0 )X0 + τ0 X1 . If X0 or X1 is stabilizing then so is Xτ0 . ˆ τ0 := (1 − τ0 )X0 + τ0 X1 . Obviously X ˆ τ0 ∈ dom RMτ0 and by Proof: Set X Remark 5.1.4(v) 0 ≤ (1 − τ0 ) (Π(X0 ) + Λ(X0 ) + M0 ) + τ0 (Π(X1 ) + Λ(X1 ) + M1 ) ˆ τ0 ) + Λ(X ˆ τ0 ) + Mτ0 , = Π(X (5.27) ˆ τ0 ) ≥ 0. Thus by Theorem 5.3.1 there exists a greatest soluwhence RMτ0 (X Mτ0 ˆ τ0 . tion Xτ0 to R (X) = 0 and Xτ0 ≥ X If without loss of generality X0 is stabilizing, then by Theorem 5.3.1 there ˜ τ0 := ˜ 0 ) > 0. Now we set X ˜ 0 ∈ dom RM0 such that RM0 (X exists an X Mτ0 ˜ τ0 ) + ˜ (1 − τ0 )X0 + τ0 X1 ∈ dom R and conclude as in (5.27), that Π(X ˜ Λ(Xτ0 ) + Mτ0 > 0. Thus again by Theorem 5.3.1 the greatest solution of the equation RMτ0 (X) = 0 is stabilizing. "

Behaviour on the boundary Finally we take a look at the boundary of M2 . The question arises, whether X+ (M ) explodes, as M approaches ∂M2 . The next Proposition gives a sufficient criterion to exclude such a behaviour. Proposition 5.3.13 Let (Mk )k∈N be a bounded decreasing sequence in M2 . If the pair (A, B) is controllable, then the X+ (Mk ) are bounded and converge to the greatest solution of Λ(X) + Π(X) + M ≥ 0 where M = limk→∞ Mk . This result is of particular interest in view of a non-strict version of the Bounded Real Lemma 2.2.4. Corollary 5.3.14 Assume that system (2.8) is internally stable and
152

5 Solution of the Riccati equation

Proof: If for k ≥ 1, we set 0 6 −C ∗ C −C ∗ D1 Mk = , −D1∗ C (γ + k1 )2 I − D1∗ D1 then by Lemma 2.2.4 there exists an X < 0, satisfying X ∈ dom+ RMk and RMk (X) > 0. Hence, R is stabilizable by Theorem 5.3.7, such that Mk ∈ M2 . Now, the assertion follows from Proposition 5.3.13. " To prove Proposition 5.3.13, we need two simple lemmata. Lemma 5.3.15 Let (Mk )k∈N be an unbounded increasing sequence of Hermitian matrices in Hn . Then there exists a nonzero vector e ∈ Kn such that limk→∞ 1x, Mk x2 = ∞ for all x ∈ Kn with 1x, e2 = O 0. Proof: Replacing Mk by Mk − M0 we may suppose without restriction of generality that Mk ≥ 0. By a compactness argument we find a subsequence (kj )j∈N such that the limit H = limj→∞ Mkj /<Mkj < exists. Since
j→∞

Mkj x2 = 1x, Hx2 = |α|2 + 1z, Hz2 ≥ |α|2 , <Mkj <

whence 1x, Mkj x2 → ∞ as j → ∞, and so limk→∞ 1x, Mk x2 = ∞ by monotonicity. " 0

6 Pk Sk form an Sk∗ Qk unbounded increasing sequence in Hn+L and assume that 0. Then, for arbitrary x ∈ Kn , u ∈ KL , 0 6 0 6 x x , Mk 2=∞ ⇐⇒ lim 1x, Pk x2 = ∞ . lim 1 u u k→∞ k→∞

Lemma 5.3.16 Suppose the Hermitian matrices Mk =

Proof: Again we may assume that Mk ≥ 0, k ∈ N. In order to prove “⇒” suppose that 0 6 0 6 x x 2 = 1x, Pk x2 + 2 Re1x, Sk u2 + 1u, Qk u2 → ∞ as k → ∞ . , Mk 1 u u The Cauchy-Schwarz inequality with respect to the semidefinite scalar product induced by Mk yields 1x, Pk x2 1u, Qk u2 ≥ |1x, Sk u2|2 .

5.3 Existence and properties of stabilizing solutions

153

Hence 1x, Pk x2 → ∞ and ‘⇒’ is proved. To show the converse implication let us assume αk := 1x, Pk x2 → ∞. Then 0 6 0 6 √ x x , Mk 1 2 ≥ α2k − |1x, Sk u2| ≥ α2k − αk γ
"

By combining the preceding two lemmata we obtain the following corollary, which will be needed in the proof of Proposition 5.3.13. 0

6 Pk Sk ∈ Hn+L , k ∈ N is an unbounded inSk∗ Qk creasing sequence of Hermitian matrices with bounded lower right block Qk , then there exists a nonzero vector e ∈ Kn , such that for all x ∈ Kn with 1e, x2 O= 0 and all u ∈ KL : 0 6 0 6 x x , Mk lim 1 2=∞. u u k→∞ Corollary 5.3.17 If Mk =

Proof of Proposition 5.3.13: To simplify the notation, we write Xk = X+ (Mk ). For each k ∈ N the matrix Xk is the largest solution of the linear matrix inequality Λ(X) + Π(X) + Mk ≥ 0 in cl dom+ RMk . By Theorem 5.3.11 the sequence (Xk ) is monotonically decreasing. It remains to show that ˆ ≤ Xk for all k ∈ N. Then, by continuity, the there exists a lower bound X limit limk→∞ Xk = X∞ is the largest solution of Λ(X) + Π(X) + M ≥ 0 in cl dom RM (compare Corollary 5.3.10 and its proof). By the Kalman-Yakubovich-Popov criterion we have for all k: 6∗ = = D 6 0 D0 (iωI − A)−1 B (iωI − A)−1 B ≥ 0 (5.28) Π Xk + M k ∀ω ∈ R : I I (compare [216, 145], or Theorem 5.1.6(iv)). We assume now that the sequence (Xk )k∈N is not bounded. Then there are two possibilities: Either Π(Xk ) is bounded below for k → ∞ or not. In the first ˆ := limk→∞ Π(Xk ) and (5.28) remains case there exists (by monotonicity) Π ˆ + M . Thus, by the valid, if we pass to the limit and replace Π(Xk ) + Mk by Π sufficiency of the Kalman-Yakubovich-Popov criterion, there exists a solution ˆ of the inequality Λ(X) + Π ˆ + M ≥ 0. But since Π ˆ + M ≤ Π(Xk ) + Mk X ˆ for all k, the matrix X also satisfies Λ(X) + Π(X) + Mk ≥ 0 and hence is a lower bound for all Xk . This contradicts our assumption. So assume that (Π(Xk ))k∈N is not bounded. Since both (Π(Xk ))k∈N and (Mk )k∈N are decreasing the sequence (Π(Xk ) + Mk )k∈N is decreasing and unbounded. We want to show that this is incompatible with (5.28). By (5.28) the lower right ^ × ^ block of Π(Xk ) + Mk is nonnegative definite for all k because limω→∞ <(iωI − A)−1 B< = 0. Thus the assumption of Corollary

154

5 Solution of the Riccati equation

5.3.17 is satisfied with −Π(Xk ) − Mk in the role of Mk . Hence there exists a nonzero vector e ∈ Kn such that for all x ∈ Kn with 1e, x2 O= 0 and u ∈ KL : 0 6 0 6 x x 2 = −∞ . , (Π(Xk ) + Mk ) lim 1 u u k→∞ By the controllability of (A, B) there exist an ω0 ∈ R and a vector u ∈ KL such that 1e, (iω0 I − A)−1 Bu2 O= 0. For if e∗ (iωI − A)−1 B vanished identically in ω ∈ R, then all coefficients in the Laurent expansion e∗ (iωI − A)−1 B = P∞ e∗ Ak B n−1 B) = Kn the k=0 (iω)k+1 would vanish. Since e O= 0 and Im(B, AB, . . . , A latter is impossible. For the constructed u we have 6∗ 0 D 0 (iω I − A)−1 B 6 −1 B = 0 ∗ (iω0 I − A) Π(Xk ) + Mk lim u u = −∞ I I k→∞ in contradiction to (5.28).

"

Some comments on coupled Riccati equations The discussion of the coupled Riccati inequalities in Section 2.3.3 is beyond our scope. We only wish to reflect shortly on their specific difficulties in the stochastic case. Let us recall the following definitions: P1 (X) = XA + A∗ X +

N O

i ∗ 2 ∗ Ai∗ 0 XA0 − C1 C1 + γ C2 C2 ,

i=1

S1 (X) = XB1 +

N O

i Ai∗ 0 XB0 ,

i=1

Q1 (X) = γ 2 I +

N O

B0i∗ XB0i ,

i=1

P2 (X, Y ) = Y A + A∗ Y +

N O

i ∗ ∗ Ai∗ 0 XA0 − C1 C1 + Y B2 B2 Y ,

i=1

S2 (X, Y ) = Y B1 +

N O

i Ai∗ 0 XB0 .

i=1

With these operators we define R1 (X) = P1 (X) − S1 (X)Q1 (X)−1 S1 (X)∗ , R2 (X, Y ) = P2 (X, Y ) − S2 (X, Y )Q1 (X)−1 S2 (X, Y )∗

(5.29) (5.30)

5.3 Existence and properties of stabilizing solutions

155

R and set dom+ R1 = {X ∈ Hn R Q1 (X) > 0}. The task is to find matrices 0 > Y > X ∈ dom+ R1 , such that R1 (X) > 0 and R2 (X, Y ) > 0. Clearly, the operator R1 is of the form considered in Section 5.3.1. The same is true for the operator Y L→ R2 (X, Y ), if X is fixed. Hence one might first ˆ ˆ to the inequality R1 (X) > 0 and then, with this X, look for a solution X ˆ Y ) > 0. In both cases we can rely on the results discuss the inequality R2 (X, from Section 5.3.1. If considered separately, each inequality can be replaced by an equation with a stabilizing solution. But if we take the coupling into account, then it becomes unclear, which solutions of each inequality we are to choose. The condition Y > X suggests to look for a smallest solution X− of the inequality R1 (X) ≥ 0 rather than the largest. With this X− fixed, one can then compute the largest solution of the inequality R2 (X− , Y ) > 0. In fact, this is the method of choice in the deterministic case (cf. [75]). In the stochastic case, however, there arise two difficulties. On the one hand, unlike in the deterministic case, there are no useful conditions for a smallest solution to exist. In the deterministic case, the smallest solution to the inequality R1 (X) ≥ 0 is obtained as the negative of the largest solution of R1 (−X) = 0. Here one exploits the fact that X L→ R1 (−X) is also a Riccati operator (of the same form) if Π = 0. In the stochastic case this is not true, because, simply speaking, X L→ Π(−X) is not positive if Π O= 0. On the other hand, the coupling in the Riccati equations also implies that the ˆ of the inequality R2 (X, ˆ Y ) ≥ 0 becomes smaller, if X ˆ largest solution Y+ (X) ˆ becomes smaller. Even worse, the inequality itself might not be solvable, if X ˆ is too small. Hence, we see competing principles at work. We need X to be large and small at once. This conflict has also been observed in [106]. 5.3.2 The Riccati equation with indefinite constraint Let R be a Riccati operator on Hn and assume the situation of Section 5.1.2 including Assumption 5.1.3. By this assumption, there exist matrices C and D2 of appropriate sizes, such that 6 0 ∗6 0 5 C / P0 S20 C D2 , and Q20 = D2∗ D2 > 0 . =− ∗ Q20 S20 D2∗ We introduce these matrices to emphasize the connection of our results to the disturbance attenuation problem in Section 2.3. The dual operator G is defined according to (5.10). In analogy to Theorem 5.3.1, we study the relation between the Riccati-type inequality R(X) ≥ 0 and the equation R(X) = 0. As in the deterministic case (e.g. [103, 145]), our results involve detectability and observability conditions for the original data and stabilizability conditions for the dual operator G. First, we formulate a consequence of Theorem 4.2.2.

156

5 Solution of the Riccati equation

Theorem 5.3.18 Let G be stabilizable. Then the following are equivalent: ˆ < 0, such that R(X) ˆ ≥ 0. ˆ ∈ dom± R, X (i) ∃X F ? (ii) ∃X+ ∈ dom± R, X+ < 0, such that R(X+ ) = 0, σ RX+ 3 ⊂ C− ∩ iR and X+ is the largest solution of the inequality R(X) ≥ 0 in dom± R ∩ n int H− . Proof: By Lemma 5.1.8, assertion (i) is equivalent to the existence of a ˆ −1 ∈ dom+ G, satisfying G(Yˆ ) ≥ 0. Makpositive definite matrix Yˆ = −X ing use of Theorem 5.2.7, we can apply Theorem 4.2.2 to show the existence of a solution Y+ ∈ dom+ G to the equation G(Y+ ) = 0. Moreover σ(GY3 + ) ⊂ C− ∪ iR by continuity, and Y+ is the largest solution of the inequality G(Y ) ≥ 0 in dom+ G. Hence, again by Lemma 5.1.8, assertion (ii) n . The converse is trivial. " holds with X+ = −Y+−1 ∈ dom± R ∩ int H− By Theorem 5.2.7(v), the operator G is stabilizable, if (−R30 , R(0)) is de−1 ∗ ∗ tectable. The latter holds (cf. Remark 1.8.6), if (A + BQ−1 0 S0 , P0 − S0 Q0 S0 ) is observable. These conditions involve all system parameters. In the following result on stabilizing solutions we have an alternative observability assumption, which only involves the parameters of the system without the perturbation input v. We introduce the regularized data A = A − B2 (D2∗ D2 )−1 D2∗ C ,

Aj0 = Aj0 − B20 (D2∗ D2 )−1 D2∗ C , ∗

P0 = −C C + C

∗

D2 (D2∗ D2 )−1 D2∗ C

(5.31) .

Note that A = A, Aj0 = Aj0 and P0 = −C ∗ C under the regularity assumption D2∗ C = 0. Theorem 5.3.19 Assume that (a) G is stabilizable, or (b) the pair (A, P0 ) is observable. Then the following are equivalent: (i) ∃X ∈ dom± R, X < 0, such that R(X) > 0. F ? (ii) ∃X+ ∈ dom± R, X+ < 0, such that R(X+ ) = 0 and σ RX+ 3 ⊂ C− . Proof: The equivalence of (i) and (ii) under the assumption that G is stabilizable follows immediately from Corollary 4.2.7 and Lemma 5.1.8. By Lemma 5.3.23 below, the observability of (A, P0 ) together with (i) implies the stabilizability of G. Hence (b) implies the equivalence of (i) and (ii). "

Remark 5.3.20 If X+ is a stabilizing solution of Rγ (X) = 0, where Rγ is given in (2.21), then F defined in (2.22) solves the γ-suboptimal stochastic H ∞ problem for system (2.13). This follows from the fact that RγF (X) = 0 and (RγF )3X is stable by Corollary 2.3.5. Hence
5.3 Existence and properties of stabilizing solutions

157

If G is not stabilizable, we can still apply Theorem 4.2.2 to approximate a solution of the inequality R(X) > 0. The idea is to perturb R slightly, such that G becomes stabilizable and the strict inequality remains solvable. For ε > 0 we define Rε : X L→ R(X) − εI and its dual G ε : Y L→ G(Y ) − εY 2 . Proposition 5.3.21 Assume that for some ε > 0 there exists an X ∈ dom± R, X < 0, such that R(X) > εI. Consider the sequence (Yk ) produced by Newton’s method applied to the equation G ε (Y ) = 0 starting at Y0 = νI. If ν > 0 is chosen sufficiently large, then the Yk converge quadratically to a stabilizing solution Y+ε ∈ dom+ G of this equation. Its negative inε verse X+ = −(Y+ε )−1 ∈ dom± R is a stabilizing solution of the equation R(X) = εI; moreover, it is the largest solution of the inequality R(X) ≥ εI n . in dom± R ∩ int H− Proof: The operator G ε is well-defined on dom G and the assertions of Theorem 5.2.7 carry over to G ε . Since R(0) − εI < 0, Theorem 5.2.7(v) yields that ε 3 the operator GνI is stable for ν G 1. By assumption, the inequality G ε (Y ) > 0 is solvable in dom+ G. Thus, the result follows from Theorem 4.2.2. " ε converge to a solution of the Riccati equation. As ε → 0, the X+

Corollary 5.3.22 Assume that there exists an X ∈ dom± R, such that R(X) > D 0. Then there exists an X+ ≤ 0 such that R(X+ ) = 0 and = σ R3X+ ⊂ C− ∪ iR. Proof: For sufficiently small ε the assumptions of Proposition 5.3.21 are ε < 0 are the largest solutions of R(X) ≥ εI in satisfied. Since the X+ ε −1 n ) ∈ dom+ G, dom± R∩int H− , they increase as ε decreases. Moreover, −(X+ ε ε such that X+ ≤ 0 for all ε > 0. Hence the X+ converge to some X+ ≤ 0 as ε → 0, and the assertions hold by continuity. " If we consider the matrix X+ from Corollary 5.3.22 we can fill the gap in the proof of Theorem D by showing that the observability of (A, P0 ) implies = 5.3.19 3 X+ < 0 and σ RX+ ⊂ C− . ˆ ∈ dom± R, such that R(X) ˆ > Lemma 5.3.23 Assume that there exists an X 0. If (A, P0 ) is observable, then there exists an X ∈ dom R, X < 0, such + ± + D = that R(X+ ) = 0 and σ R3X+ ⊂ C− .

Proof: It remains to show that the matrix X+ ≤ 0 from Corollary 5.3.22 is negative definite and stabilizing. By Remark 5.1.4(vi) and the definition of PF in (2.23) it follows that (A + B2 F )∗ X+ + X+ (A + B2 F ) ≥ (C + D2 F )∗ (C + D2 F ) ,

158

5 Solution of the Riccati equation

whence by Lemma 1.8.7 (i) and Corollary 3.6.1 we have X+ < 0, i.e. X+ ∈ −1 dom± R. Thus also Y+ = −X+ ∈ dom+ G and G(Y+ ) = 0. Moreover Y+ is the ˆ −1 ∈ largest solution of the inequality G(Y ) ≥ 0. By our assumption Yˆ = −X dom+ G and G(Yˆ ) > 0. We conclude, that Yˆ ≤ Y+ . Since G is concave on dom+ G, we have Since GY+ 3

GY+ 3 (Yˆ − Y+ ) ≥ G(Yˆ ) − G(Y+ ) = G(Yˆ ) > 0 . F ? is resolvent positive, Theorem 3.2.10 yields σ GY+ 3 ⊂ C− .

"

Thus we have also established the second part of Theorem 5.3.19. In accordance with Theorem 5.3.1 it is natural to ask, whether we can replace the observability condition by β-detectability. At this point, it is helpful to recall the underlying disturbance attenuation problem from Section 2.3. Our aim was to construct a feedback-gain matrix F , such that the closed-loop system dx = (A + B2 F )x dt +

N O

j (Aj0 + B20 F )x dwj + B1 v dt +

j=1

N O

j B10 v dwj

j=1

z = (C + D2 F )x + D1 u is internally stable and the perturbation operator LF has norm less than γ. This is achieved by F = −(Q2 (X) − S3 (X)Q1 (X)−1 S3 (X)∗ )−1 × (S2 (X)∗ − S3 (X)Q1 (X)−1 S1 (X)∗ ) , if X = X+ is a stabilizing solution of the Riccati equation R(X) = 0. But it is slightly too much to require σ(R3X+ ) ⊂ C− , because this implies also the stability of the closed-loop system dx = (A + BF )x dt +

N O

(Aj0 + B0j F )x dwj

j=1

= (A + B2 F + B1 F )x dt +

N O

j j (Aj0 + B20 F + B10 F )x dwj .

j=1

If we merely look for an internally stabilizing feedback F , then we just need j σ(LAˆ + ΠAˆ0 ) ⊂ C− with Aˆ = A + B2 F and Aˆj0 = Aj0 + B20 F as in (2.15). This is exactly what we can guarantee under the weaker assumptions. Proposition 5.3.24 Assume that (A, (Aj0 ), P0 ) is β-detectable and there exists a solution X+ ∈ dom± R to the equation R(X) = 0. Then X+ is internally stabilizing, i.e. D = σ LAˆ + ΠAˆ0 ⊂ C− , j where Aˆ = A + B2 F , Aˆj0 = Aj0 + B20 F , and F is defined in (5.7).

5.3 Existence and properties of stabilizing solutions

Proof: By Remark 5.1.4(vi) and the definition of Aˆ and D = LAˆ + ΠAˆ0 ≥ (C + D2 F )∗ (C + D2 F ) =

159

Aˆj0 it follows that Cˆ ∗ Cˆ

ˆ (Aˆj ), C) ˆ is β-detectable by Lemma 1.8.7 with Cˆ = C + D2 F . The triple (A, 0 " (ii), such that the result follows once more from Theorem 3.6.1.

Remark 5.3.25 (i) If X+ is an internally stabilizing solution of Rγ (X) = 0, where Rγ is given in (2.21), and F is defined according to (2.22) then the perturbation operator LF corresponding to system (2.14) satisfies 0. Assume that there exists a stabilizing solution X+ ∈ int Hn ⊂ dom+ R. ˜ = By Remark 5.1.4(i) we can consider the equivalent equation −R(−X) ˜ ˜ ˜ = R(X) = 0. Obviously X > 0 satisfies R(X) = 0 if and only if X n ˜ ˜ ˜ ˜ −X ∈ int H− ⊂ dom± R satisfies R(X) = 0. By assumption, Y+ = ˜ −1 = X −1 > 0 stabilizes the dual operator G˜ and G(Y ˜ + ) = 0. Hence, −X + + by Theorem 5.3.19, Y+ is the largest positive definite solution of the ˜ + < 0 is the largest negative definite solution ˜ ) = 0 and X equation G(Y ˜ ˜ + is the smallest positive of R(X) = 0. We conclude that X+ = −X definite solution of R(X) = 0. This again proves that there can be at most one positive definite solution. Parameter dependence of the largest solution By G M we denote the operator dual to RM , where M may vary, and Λ and Π are fixed. In the indefinite case it seems more difficult to classify those M that allow a largest solution X+ ∈ dom± RM of the equation RM (X) = 0. We will restrict ourselves to some relevant cases. As before, let M have the form   P0 S20 S10 ∗ Q20 S30  . M =  S20 ∗ ∗ S10 S30 Q10 In analogy to the previous section, we introduce the following general sets: N± = {M : P0 ≤ 0, Q20 < 0, Q10 > 0} , N0 = {M ∈ N± : G M is stabilizable} , n }, N1 = {M ∈ N± : RM (X) > 0 is solvable in dom± RM ∩ int H− M M n ¯1 = {M ∈ N± : R (X) ≥ 0 is solvable in dom± R ∩ int H− } , N

160

5 Solution of the Riccati equation

N2 = {M ∈ N± : RM (X) = 0 has a stabilizing solution n in dom± RM ∩ int H− },

¯2 = {M ∈ N± : RM (X) = 0 has an internally stabilizing solution N n in dom± RM ∩ H− }.

These definitions are just meant as abbreviations that allow us to state the following results more concisely. Recall also the definition of A, Aj0 , and P0 in (5.31). Corollary 5.3.26 (i) N0 ⊃ int N± . ¯1 . (ii) N0 ∩ N1 = N2 ⊂ N0 ∩ N (iii) int N1 ⊂ N2 ⊂ N1 . ¯2 . (iv) If (A, (Aj0 ), P0 ) is β-detectable, then N2 ⊂ N Proof: (i), (ii), (iii), and (iv) follow from Theorem 5.2.7(v), Theorem 5.3.18, Corollary 5.3.22, and Proposition 5.3.24, respectively. Alternatively, we can derive (iv) from the Corollaries 2.3.5 and 5.3.5. " The smooth dependence of X+ on M ∈ N2 follows exactly as in the proof of Theorem 5.3.9. n Theorem 5.3.27 There exists a (real) analytic function X+ : N2 → int H− M such that X+ (M ) is the stabilizing solution of R (X) = 0 for all M ∈ N2 . It can be extended continuously to N0 ∩ N¯1 .

Finally, we analyze the monotonic dependence of X+ on M and derive some ˜ , M] ⊂ sufficient criteria for X+ to be defined on a given order interval [M n+s H . Again, the informal principle of Remark 5.3.6 applies. Roughly speakˆ ≥ 0 in ing, the existence of X+ (M ) follows from the solvability of RM (X) M M ˜ dom± R and the stabilizability of G . Existence intervals [M , M ] will now ˆ ˆ ˆ ≥ M, be specified by giving criteria that RM (X) ≥ 0 is solvable for all M ˆ ˆ ≤ M . Here we make use of the connection and G M is stabilizable for all M between the stabilizability of the dual operator G and detectability criteria for the original system. The following proposition transforms this idea into some precise statements. ˜ ∈ cl N± , such that M ≥ M ˜ , and define A, ˜ Proposition 5.3.28 Let M, M ˜ j M M ˜ ˜ ˜ A0 , and P0 in analogy to (5.31). We set R = R and R = R , with the ˜ respectively. Then the following hold: dual operators G and G, (i) (ii) (iii) (iv) (v)

˜ dom+ G ⊃ dom+ G. ˜ ˜ ∀X ∈ dom± R, X < 0: R(X) ≥ R(X). ˜ P˜0 ). If (A, P0 ) is observable, then so is (A, j ˜ (A˜j ), P˜0 ). If (A, (A0 ), P0 ) is β-detectable, then so is (A, 0 ˜ 3 , R(0)), ˜ If (±R30 , R(0)) is detectable, then so is (±R respectively. 0

5.3 Existence and properties of stabilizing solutions

161

˜ ∈ N1 , then If (A, P0 ) is observable or (−R30 , R(0)) is detectable and M ˜ , M ] ⊂ N2 . [M ˜ ∈ N1 , then [M ˜ , M ] ⊂ N¯2 . (vii) If (A, (Aj0 ), P0 ) is β-detectable and M ˜ ¯ ˜ ˜ ). (viii) If M, M ∈ N0 ∩ N1 and M ≥ M , then X+ (M ) ≥ X+ (M (ix) If (Mk ) is a monotonically increasing sequence in N2 then there exists the limit matrix X∞ = limk→∞ X+ (Mk ) ≤ 0. If, moreover, the Mk converge to M∞ ∈ cl N± , and X∞ ∈ dom± RM∞ , then RM∞ (X∞ ) = 0.

(vi)

Proof: (i): By definition, we have X ∈ dom+ G M if and only if QM 1 (X) > 0. ˜ M n ˜ , which proves the Obviously QM (X) ≥ Q (X) for all X ∈ H , if M ≥ M 1 1 assertion. ˜ as Schur-complements R(X) = S(Λ(X) + Π(X) + (ii): If we write R and R ˜

n M/Q1 (X)), we can directly apply Corollary A.4, since X ∈ dom± RM ∩int H− ˜ implies In Q(X) = In Q(X). n (iii) and (iv): We note that 0 ≥ P ≥ P˜ . Hence, if P˜ V = 0 for some V ∈ H+ , −1 −1 ∗ ∗ ˜ S˜ V . If, then also P V = 0; moreover, by Corollary A.4, also Q20 S20 V = Q 20 20 in addition, V is an eigenvector of L∗A˜ or L∗A˜ + ΠA˜0 , then it follows that V is also an eigenvector of L∗A or L∗A + ΠA0 , respectively, with the same eigenvalue β. If we assume (A, P0 ) to be observable, or (A, (Aj0 ), P0 ) to be β-detectable, then necessarily β < 0, which proves the assertions. (v) Let X < 0 or X > 0 be given such that R30 (X) + R(0) < 0. Then ˜ ≤ R(0) ≤ 0, we have x∗ R30 (X)x < 0 for all x ∈ Ker R(0). Since R(0) ˜ ˜ Ker R(0) ⊂ Ker R(0), and x ∈ Ker R(0) implies F0 x = F˜0 x by Corollary ˜ 30 (X)x < 0 for all x ∈ Ker R(0), ˜ A.4. Hence also x∗ R and a similar argument ˜ ˜ 3 (αX) + R(0) < 0 for some α > 0. as in the proof of Lemma 1.7.3 yields R 0 (vi) and (vii): From (iii) and (iv) it follows that the observability or de˜ , M ]. From (ii) we know tectability assumption holds for all elements of [M ˜ ∈N ¯1 implies [M ˜ , M ] ⊂ N1 or [M ˜ , M ] ⊂ N¯1 , respectively. ˜ ∈ N1 or M that M Hence the assertions (vi) and (vii) follow from Corollary 5.3.26. ˜ ) are the largest solu˜ ∈ N0 ∩ N¯1 then X+ (M ) and X+ (M (viii) If M, M ˜ M M ˜ implies tions of R (X) ≥ 0 and R (X) ≥ 0, respectively. Since M ≥ M ˜ M M ˜ )) ≥ R (X+ (M ˜ )) = 0 we have X+ (M ) ≥ X+ (M ˜ ). R (X+ (M (ix) By (ii) the X+ (Mk ) ≤ 0 are monotonically increasing, and hence converge. The assertion holds by continuity. "

Remark 5.3.29 For M ∈ N1 , Corollary 5.3.22 guarantees the existence of an almost stabilizing solution X+ ∈ dom± RM ∩ −Hn of the inequality RM (X) = 0. But we have not shown that X+ is the largest solution of the inequality RM (X) ≥ 0. Since we have neither found a counter-example, we leave this question open. But since X+ is obtained as the monotonic limit of stabilizing solutions of RM−εI (X) = 0, it depends monotonically on M .

162

5 Solution of the Riccati equation

5.4 Approximation of stabilizing solutions Since our approach in the previous section is constructive, it lends itself immediately for computational purposes. As we have shown, we can apply all the results in Chapter 4 on Newton’s method and modified Newton iterations to solve the equations R(X) = 0 and G(X) = 0. Each Newton-step consists in solving a linear equation with a resolvent positive operator, which was the topic of Section 3.5. In the following, we summarize the main facts; in particular we show, how a stabilizing matrix can be found to start the Newton iteration. Moreover, in Section 5.4.3, we present an alternative nonlinear fixed point iteration. 5.4.1 Newton’s method First, we recall the explicit form of Newton’s method and its modification to solve the equations R(X) = 0 and G(X) = 0. The equation R(X) = 0 For X ∈ dom R, let us set F (X) = −Q(X)−1 S(X)∗ . Then, by Theorem 5.2.2, the iteration R(X) = 0 takes the form Xk+1 = Xk − (R3Xk )−1 (R(Xk )) 6∗ 0 6G @0 I I = −(R3Xk )−1 M , F (Xk ) F (Xk ) where R3Xk (H) = (LAXk + ΠXk )(H) 0 = LA−BF (Xk ) (H) +

I F (Xk )

0

6∗ Π(H)

I F (Xk )

6 .

In each step, we have to evaluate F (Xk ) and compute either Xk+1 directly as the solution of 0 6∗ 0 6 I I 3 RXk (Xk+1 ) = − M (5.32) F (Xk ) F (Xk ) or the increment Xk+1 − Xk as the solution of R3Xk (Xk+1 − Xk ) = −R(Xk ) .

(5.33)

Both equations are of the type discussed in Section 3.5; by our results in Chapter 4 we know that they are solvable for all k, if X0 is stabilizing and the inequality R(X) ≥ 0 is solvable in dom+ R. The first equation corresponds to the algorithm proposed for standard Riccati

5.4 Approximation of stabilizing solutions

163

equations e.g. in [131, 174]. But it has been mentioned e.g. in [12] that the second exhibits better robustness properties in the presence of rounding errors. In Section 3.5 we have developed different strategies to solve such equations. If the dimension is small then a direct solution is possible. Otherwise, the particular cases, when (i) rk Π is small, or (ii) <Π< is small, are important. In the first case R3X is a low-rank perturbation of a Lyapunov operator, and we can apply the results from Section 3.5.3. If rk Π = r and we have a factorization Π = Φ ◦ Ψ , where Φ : Kr → Hn and Ψ : Hn → Kr , r n then rk Π0X ≤ r 6and ΠX 0 = ΦX6◦ Ψ with ΦX : K → H defined by ∗ I I ΦX (·) = Φ(·) . In other words, a factorization of Π F (Xk ) F (Xk ) has to be computed only once. As we have seen in Section 3.5.3, the cost for the solution of equation (5.33) remains moderate, if e.g. r ≤ n. (ii) In the second case, an iterative method as described in Section 3.5.4 is promising. From our theoretical results, we know that LAXk + ΠXk is a convergent regular splitting of R3Xk . Hence the solution of the equation R3Xk (H) = −R(Xk ) is given as the limit of the sequence

(i)

HL+1 = −L−1 AX (R(Xk ) + ΠXk (HL )) k

with arbitrary H0 . We have seen in Section sec:Ljusternik that the speed of convergence can be improved significantly by a Ljusternik-acceleration. The question arises, as to which precision we need to compute H = Xk+1 − Xk . During the first steps of the Newton iteration one might not require the highest precision in the computation of H. Clearly, Xk+1 must be stabilizing to guarantee that the Newton sequence is well-defined; moreover we should have R(Xk+1 ) ≤ 0. If H = HL is to be be accepted, then Xk+1 = Xk + HL should at least satisfy these requirements. The same remarks apply, if we choose a different regular splitting of R3Xk . If for all X ∈ dom R the decomposition R3X = LX + PX is a regular splitting, then we can replace the Newton iteration (5.33) by the modified Newton iteration LXk (Xk+1 − Xk ) = −R(Xk ) . If X0 is stabilizing, then, by Theorem 4.2.11, the Xk converge to X+ . To obtain a reasonable regular splitting we can again apply the strategies developed in Section 3.5.4. An obvious choice, of course, is given by LX = LAX and PX = ΠX . The equation G(Y ) = 0 In the notation of Section 5.2.2, we can write the Newton iteration to solve the equation G(Y ) = 0 in the equivalent forms

164

5 Solution of the Riccati equation

GY3 k+1 (XY +1 ) =

0

Y FYk

0

6∗ M

Y FYk

6 (5.34)

∗ where FY = Q−1 Y SY , or

GY3 k+1 (Yk+1 − Yk ) = −G(Yk ) .

(5.35)

Since GY3 = LY + ΠY all the ideas presented for the equation R(X) = 0 apply also to the equation G(Y ) = 0; we do not repeat them in detail. One difference between the two equations lies in the fact that we need the inverse of Y to evaluate G, which might cause numerical difficulties. On the other hand, compared to the solution of the n2 -dimensional equation (5.33), the computation of Y −1 can be regarded as relatively easy. Moreover, the −1 and B0j∗ Y −1 , such that we do inverse Y −1 occurs only in products Aj∗ 0 Y not really need Y −1 explicitly. It suffices to solve the corresponding linear equations. 5.4.2 Computation of stabilizing matrices The applicability of our methods hinges on the possibility to find a stabilizing initial matrix X0 or Y0 . We show how such a matrix can be computed. Let us first consider the dual operator G. ∗ Lemma 5.4.1 Let M ∈ N± and assume that (A − BQ−1 0 S0 , R(0)) is observable. Then the standard Riccati equation −1 ∗ ∗ ∗ 2 (A − BQ−1 0 S0 ) X + X(A − BQ0 S0 ) − R(0) − X = 0

(5.36)

has a solution X+ > 0. −1 For sufficiently large ν > 0, the matrix Y0 = νX+ is contained in dom+ G and stabilizes G. 2 >0 Proof: The existence of X+ > 0 follows from Theorem 5.1.6. Since X+ 3 and R0 ≥ LA−BQ−1 S ∗ , the equality (5.36) for X = X+ implies 0

0

0 > LA−BQ−1 S ∗ (−X+ ) + R(0) ≥ R30 (−X+ ) . 0

0

Now we can apply Theorem 5.2.7(v) to finish the proof.

"

If the strict inequality R(0) < 0 holds, then a stabilizing matrix Y0 for G can e.g. be found in the form Y0 = νI. We make use of this observation to compute a stabilizing matrix for RM with M ∈ M+ . Computation of a stabilizing matrix for RM with M ∈ M+ ˆ ∈ dom+ RM . As we have seen in the proof of ProposiLet M ∈ M+ and X ˆ +X ˆ0 , tion 5.2.5, we can find a stabilizing matrix X0 for RM in the form X

5.4 Approximation of stabilizing solutions

165

ˆ 0 is a stabilizing matrix for RM with M ˆ defined in (5.16). Since the where X ˆ M stabilizability of R is independent of Pˆ0 we can replace Pˆ0 by an arbitrary ˆ > 0. Then, by matrix in Hn and assume without loss of generality that M ˆ M Lemma 1.7.7 and Proposition 5.2.5 the operator R is stabilizable if and only ˆ 0 ) < 0. This inequality is equivalent ˆ 0 > 0, such that RMˆ (X if there exists an X to ˆ

ˆ

ˆ 0 ) > 0, R−M (−X

ˆ 0 < 0, ˆ <0. −X where −M (5.37) R ˆ ˆ 0 +ΠB0 (X) < 0}, then we recognize this If we set dom± RM = {X ∈ Hn R − Q ˆ 1 (X) as the special case of an inequality with indefinite constraint (where Q ˆ ˆ is empty). Since M < 0, Assumption 5.1.3 holds, and the dual operator G −M ˆ ˆ is stabilizable by some Y0 = νI. If now R is stabilizable, i.e. −M ∈ N1 , ˆ ∈ N2 , and the Newton sequence starting at Yˆ0 converges to a then also −M ˆ ˆ + Yˆ+−1 stabilizing solution Yˆ+ of G −M (Y ) = 0. By our construction X0 = X is a stabilizing matrix. ˆ ∈ dom+ RM we can thus proceed as follows (see also our paper [39]). Given X ˆ 0 = Q(X), ˆ Sˆ0 = S(X) ˆ and choose a matrix Pˆ0 , such that 1. Set Q 0 6 ˆ ˆ ˆ = P0 S0 > 0. M ˆ0 Sˆ0∗ Q ˆ 2. Find ν > 0 large enough, such that σ((G −M )3νI ) ⊂ C− . 3. Set Y0 = νI and compute Y1 , Y2 , . . . according to (5.35) . ˆ 4. Watch in each step, whether the iterate Yk is stabilizing for G −M . If one M Yk is not stabilizing, then R is not stabilizable. n ˆ + Yˆ −1 is stabilizing 5. If the Yk converge to some Y∞ ∈ int H+ then X0 = X + M for R . n n or converge to some Y∞ ∈ ∂H+ then RM If otherwise the Yk leave int H+ is not stabilizable. In extreme cases it is, of course, numerically hard to decide whether X∞ ∈ n ∂H+ or not; one can use a stopping criterion, and regard the system as not stabilizable if e.g. det Yk < ε for some k. 5.4.3 A nonlinear fixed point iteration Finally, we present a different approach to solve the equation R(X) = 0 with indefinite constraint X ∈ dom± R, which has its origin in the fact that there exist powerful methods for deterministic algebraic Riccati equations. ˜ of Π into positive operators Π 0 and Let there be given a splitting Π = Π 0 + Π ˜ ˜ = 0. For Π. In particular, we may think of the trivial splitting Π 0 = Π, Π M ˜ fixed Λ and variable M let R denote the Riccati operator corresponding to ˜ and M . We assume that it is easier to compute solutions of the equation Λ, Π, M ˜ = 0, because ˜ R (X) = 0 than of RM (X) = 0. In particular, this is true if Π M ˜ then R is a standard Riccati operator. For arbitrary M let X+ (M ) and

166

5 Solution of the Riccati equation

˜ + (M ) denote the stabilizing solutions of R(X) = 0 and R(X) ˜ X = 0, which for the moment, we assume to exist. ˜ + is Now let M be fixed and consider the equation RM (X) = 0. Since X relatively easy to compute, it makes sense to consider the nonlinear fixed point iteration ˜ + (M + Π 0 (Xk )), Xk+1 = X

X0 = 0 .

(5.38)

˜ + (M + Π 0 (X)), then If X+ is a fixed point of the mapping X L→ X ˜ M+Π 0 (X+ ) (X+ ) = R(X+ ) , 0=R

(5.39)

˜ + ) = M + Π(X+ ). Hence X+ = X+ (M ). Now let since (M + Π 0 (X+ )) + Π(X us provide sufficient criteria for the sequence defined by (5.38) to converge. In accordance with our previous notation, we write G˜M for the dual operator of ˜ M , and Y = −X −1 . R Theorem 5.4.2 Let M ∈ N± and assume the following: (i) (A, P0 ) is observable (see (5.31)) or (−R30 , R(0)) is detectable. ˆ ∈ dom± R ∩ int Hn : RM (X) ˆ > 0. (ii) ∃X − Then for all k ∈ N the Xk are well-defined by (5.38) and converge monotonically to X+ (M ). ˆ 0] the following hold. Proof: It is crucial to observe that for all X ∈ [X, (a) (b) (c)

ˆ −1 ∈ dom+ G˜M+Π 0 (X) , Yˆ = −X 0 G˜M+Π (X) (Yˆ ) > 0, 0 G˜M+Π (X) is stabilizable.

Let us verify these assertions. 0 ˆ < 0, Π ˜ 1 (X) ˆ + Q10 + (a) By definition Yˆ ∈ dom+ G˜M+Π (X) if and only if X 0 0 ˜ ˆ Π1 (X) < 0, and Π2 (X) + Q20 + Π2 (X) > 0. The first two inequalities hold by ˆ and X ˆ ∈ dom± RM . assumption, while the last follows from Π20 (X) > Π20 (X) 0 ˆ ˆ and RM (X) ˆ =R ˜ M+Π (X) (X) ˆ > 0, it (b) Since M + Π 0 (X) ≥ M + Π 0 (X) M+Π 0 (X) ˆ ˜ follows from Proposition 5.2.6 that R (X) > 0. (c) This follows from Proposition 5.3.28(vi). Proceeding with the proof of convergence, we show that the Xk are monotonˆ ically decreasing and bounded below by X. ˆ As an induction hypotheses, let us assume that By assumption, X0 = 0 > X. ˆ It suffices to for some k ≥ 0 we have constructed X0 ≥ X1 ≥ . . . ≥ Xk ≥ X. ˜ + (M + Π 0 (Xk )) is well-defined as the largest solution prove that Xk+1 = X ˜ M+Π 0 (Xk ) and satisfies Xk ≥ Xk+1 ≥ X. ˆ of R Using the observations (a), (b), and (c) with X = Xk and applying The˜ + (M + Π 0 (Xk )) < 0 in ˆ < Xk+1 = X orem 5.3.19, we find a solution X M+Π 0 (Xk ) ˜ dom± R . It remains to show that Xk+1 ≤ Xk . For k = 0 this is clear

5.5 Numerical examples

167

from Xk+1 < 0. Hence, we consider the case k ≥ 1 and take advantage of our induction hypotheses Xk−1 ≥ Xk . This implies M +Π 0 (Xk−1 ) ≥ M +Π 0(Xk ), ˜ + , we have Xk ≥ Xk+1 . whence by the monotonicity of X "

5.5 Numerical examples Let us now return to some of the examples presented in Section 1.9. In order to illustrate our results, we analyze stabilizabilty and disturbance attenuation problems for these systems and compare controllers obtained for different parameter values. We deliberately take some time to write out the corresponding Riccati equations and their dual versions explicitly and to describe how to find initial matrices for the Newton iteration. Moreover, we illustrate the benefit of double Newton steps and discuss some problems of regularization. 5.5.1 The two-cart system In Section 1.9.3 we have introduced the state-space equations dx = (Ax + B1 v + B2 u)dt + A0 x dw , z = Cx + Du as a stochastic model for a two-cart system with a random parameter. The Riccati equation of the stochastic γ-suboptimal H ∞ problem is given by Rγ (X) = A∗ X + XA + A∗0 XA0 − C ∗ C ? F + X B2 (D∗ D)−1 B2∗ − γ −2 B1 B1∗ X = 0 . For the dual operator G γ (Y ) = −Y A∗ − AY − Y A∗0 Y −1 A0 Y − Y C ∗ CY + B2 (D∗ D)−1 B2∗ − γ −2 B1 B1∗ R n with dom G γ = {Y ∈ Hn R det Y O= 0}, and dom+ G γ = int H+ we have 3

GYγ0 (H) = (−A∗ − C ∗ CY0 − A∗0 Y0−1 A0 Y0 )∗ H + H(−A∗ − C ∗ CY0 − A∗0 Y0−1 A0 Y0 ) + (Y0−1 A0 Y0 )∗ H(Y0−1 A0 Y0 ) . If (A, C) is observable, then (A, A0 , C) is anti-detectable (compare Remark 3 1.8.6), i.e. (−Rγ0 , R(0)) is detectable and G γ is stabilizable by Theorem 5.2.7. A stabilizing Y0 can e.g. be found in the form αX0−1 where X0 > 0 solves the standard Riccati equation

168

5 Solution of the Riccati equation

−A∗ X − XA + C ∗ C − X 2 = 0 .

(5.40)

Indeed, if X0 solves this equation, then A∗ (−X0 ) + (−X0 )A − C ∗ C < 0, and 3 we can apply Theorem 5.2.7(viii). Moreover, since GYγ0 does not depend on γ, we can choose Y0 independently of γ. A numerical example was presented in [194], and we use this example to test our theoretical results. Let     0 0 00 0 0 10  0  0 0 0 0 0 0 1    A=  −5/4 5/4 0 0  , A0 =  −1/4 1/4 0 0  , 1/4 −1/4 0 0 5/4 −5/4 0 0     0 T6 0 6 0 1000 B1 0010    0 . 0 1 0 0 = , D = , C = 2 B2T 0001 1 0000 The matrix D2 plays the role of a regularization. We will discuss this issue in later examples. With these data we found the matrix   2 0 0 −1  0 20 0   Y0 =  (5.41)  0 02 0  −1 0 0 2 to be stabilizing. For γ = 2 we could reproduce (by Newton’s method in 10 steps) the solution obtained by Ugrinovskii; by a bisection search we found the optimal attenuation value to be γ∗ ≈ 1.8293. For γ = 1.8293 the solution   4.6440 −6.4837 −4.7299 −3.3828  −6.4837 19.1359 17.4201 10.4627   X+ = Y+−1 ≈   −4.299 17.4201 18.9092 9.8687  > 0 −3.3828 10.4627 9.8687 7.0731 of Rγ (X) = 0 is obtained in 12 steps, whereas for γ = 1.8292 the 11th iterate Y11 is not stabilizing (i.e. by Theorem 5.3.18 in this case Gγ (Y ) = 0 is not n solvable in int H+ ). The question remains, whether it is adequate to model the parameter uncertainty as white noise. For comparison, we study the system x˙ = (A + δ(t)A0 )x + B1 v + B2 u , z = Cx + Du

(5.42)

with bounded parameter uncertainty δ(t). In view of Theorem 2.3.8 we wish to determine the largest value α = αmax , such that G γ (Y ) − αY = 0

5.5 Numerical examples

169

has a stabilizing solution in dom+ G γ . √ For γ = 2 we found αmax = 0.0756, whence for all |δ(t)| ≤ d =: αmax = 0.275 system (5.42) is stabilizable with attenuation level γ = 2. This bound corresponds to |∆(t)| = |k(t) − k0 | < 0.0687 =: ∆, which is rather far away from 0.75. Considering different attenuation levels γ we found (by a bisection search) the following maximal admissible uncertainty bounds ∆: 1.893 2 4 10 100 1000 ∞ γ ∆ 0 0.0687 0.1994 0.2826 0.4146 0.5590 0.8803 . ∆/k0 0 0.05 0.15 0.23 0.33 0.45 0.70 Apparently we can guarantee stability for an arbitrary time-varying parameter k : t L→ k(t) ∈ [0.5, 2] only if we abandon disturbance attenuation. But if we aim at an attenuation value of γ = 2 or γ = 4, we can still allow deviations of k(t) from k0 of about 5 or 15 percent, respectively. Double Newton steps If γ is close to the optimal attenuation value γ ≈ 1.893, then the Newton iteration is rather slow and almost linear. As we have pointed out in Section 4.3, we can benefit from a double Newton step in this situation. To see this effect, we consider the cases γ = 2 and γ ≈ 1.893 and apply Newton’s method to the equation G γ (Y ) = 0 with initial matrix Y0 from (5.41). In each step of the iteration we compute, additionally, the matrix Zk+1 = Yk − (GY3 k )−1 (G(Yk )) and the residual errors
single Newton step double Newton step

0

residual error (logarithmic scale)

residual error (logarithmic scale)

5

−5 −10 −15 −20

0

2

4 k

6

8

single Newton steps double Newton steps

0 −5 −10 −15 −20

0

5

10

15 k

20

25

30

We see that for γ = 2 the Newton iteration converges quadratically and the double Newton steps do not bring any advantage. But if γ is close to the optimum, the situation changes. The convergence of the Newton iteration is practically linear and it takes about 25 steps to compute the solution with acceptable accuracy. The same accuracy, however, is obtained by a double Newton step already in the 18th iteration.

170

5 Solution of the Riccati equation

5.5.2 The automobile suspension system As another system with an uncertain stiffness parameter, we have introduced an automobile suspension system in Section 1.9.4. We recall the linearized state space-model dx = Ax dt + B1 v dt + B2 u dt + A0 x dw + B10 v dw , z = Cx , where



   0 0 1 0 0 0 00   0 0 0 1  0 0 0  ,  , A0 =  0 A=  −k1 /m1 0 0 0 0 k1 /m1 −c/m1 c/m1  0 −σ/m2 0 0 k1 /m2 −(k1 + k)/m2 c/m2 −c/m2 T        0 0 0 10  0   0   0  0 0        B1 =   0  , B10 =  0  , B2 =  b/m1  , C =  0 1  . k/m2 σ/m2 00 0

The following parameter values were given in [89] (with the corresponding units in square brackets): m2 = 28.57[kg] k = 120[kN/m] k1 = 20[kN/m] c = 1.6[kN s/m] . Moreover, we assume m1 = 1000[kg]. Remember that k2 = k + σ w˙ models the random stiffness of the tire. Stabilization Let us first try to determine the maximal noise intensity σmax , up to which the system is internally stabilizable. To this end, we set M = −I5 and consider the Riccati inequality Rσ (X) = A∗ X + XA + σ 2 A∗0 XA0 − I + XB2 B2∗ X > 0 . n We know that this inequality is solvable in int H− , if and only if the system is internally stabilizable. This is equivalent to the dual equation

G σ (Y ) = −Y A∗ − Y A − σ 2 Y A∗0 Y −1 A0 Y − Y 2 + B2 B2∗ = 0 , (5.43) n having a stabilizing solution in int H+ (compare Section 5.4.2). For different values of σ we apply Newton’s method with initial matrix Y0 = νI for some

5.5 Numerical examples

171

ν > 0 to solve (5.43). It turns out that even for very large noise intensities (e.g. σ = 105 ), the algorithm produces a solution Y+ > 0 and thus a stabilizing feedback-gain matrix F . Of course, this is not reasonable. Indeed, we see that Y+ is very close to singularity for large σ, and the norm of F becomes very large. Setting e.g. σ = 1000, we compute F = 107 [−0.0337, −5.4188, −0.0039, −0.7284] , which is unacceptable. Hence, we introduce the additional constraint λ1 (Y+ ) > 10−7 , where λ1 is the minimal eigenvalue of Y+ > 0, and the tolerance level 10−7 is about the square root of the machine precision. With this constraint, we find σmax ≈ 24.9370 with a corresponding stabilizing feedback-gain matrix Fmax = [−203.9290, 314.8963, −614.7729, 249.5053] .

(5.44)

To compute a guaranteed margin for robust stabilizability with respect to bounded uncertainties we consider the Riccati inequality Rσ (X) + αI > 0 (compare the Sections 1.6 and 2.3.2). With α = 4/100, we found the inequality to be solvable for σ = 52/3 = 17, ¯ 3. Hence the corresponding feedback-gain matrix F = [−154.6190, 272.6836, −547.9517, 260.2774] √ stabilizes the system robustly for |k2 − k| ≤ ασ = 52/15 = 3.4¯6. Disturbance attenuation Now let us address the disturbance attenuation problem described in Section 1.9.4. We try to choose F in such a way that the unevenness of the road v has small effect on the output y. First of all, we note that this control system is not regular in the sense of Definition 2.3.3, since D2 = 0. To apply our results, we have to regularize the system by incorporating the control u into the output z. To this end we might consider the extended output process 0 6 0 6 0 6 C 0 z ˜ ˜ ˜ ˜ , D2 = . (5.45) = Cx + D2 u with C = z˜ = 0 I u For a given feedback-matrix F ∈ R1×4 we define the perturbation operator LF : v L→ z˜ as in Section 2.3.1. We have 0, if and only ∗ if (γ 2 I + B01 XB01 )−1 > 0 and Rσ,γ (X) = A∗ X + XA + σ 2 A∗0 XA0 − C˜ ∗ C˜ − XB2 B2∗ X ∗ XB01 )−1 B1∗ X + XB1 (γ 2 I + σ 2 B01 > 0. As usually, we consider the equivalent dual inequality

172

5 Solution of the Riccati equation

˜ − B2 B ∗ G σ,γ (Y ) = −Y A∗ − AY − σ 2 Y A∗0 Y −1 A0 Y − Y C˜ ∗ CY 2 ∗ + B1 (γ 2 I + σ 2 B01 Y −1 B01 )−1 B1∗ > 0 .

It is easily seen that (A, C) is observable. Hence, by Lemma 5.4.1, the operator G σ,γ is stabilizable, and, again, a stabilizing matrix Y0 > 0 can e.g. be found in the form Y0 = νX0−1 , where X0 > 0 solves the standard Riccati equation (5.40). For different values of γ we try to solve the equation G σ,γ (Y ) = 0 by Newton’s method starting from Y0 . By a bisection search we determine the smallest γ, for which the sequence converges. Choosing σ = 0, we find γ > 17. With respect to our initial design problem, this does not make sense. We wanted to attenuate the effect of the unevenness of the road, not to increase it by the factor 17. What has happened? Of course, the problem lies with the regularization. Incorporating the control input u into the output z, we have necessarily increased the norm of the perturbation operator L : v L→ z˜. This effect can be diminished, if instead of (5.45), we set 0 6 0 6 0 6 z ˜ 2 = ε 0 u (5.46) ˜ +D ˜ 2 u with C˜ = C , D z˜ = = Cx 0 I u with some ε > 0. The choice of an adequate regularization, in fact, is not trivial, even in the deterministic case (compare e.g. [76]). On the one hand, we wish to choose ε as small as possible, such that L approximately describes the original input-output behaviour. On the other hand, choosing ε too small is likely to cause numerical problems. Moreover, in the present example, we find that small values of ε and γ result in high-gain feedback, i.e. an unbounded increase of the norm of F . To choose some bound, let us accept only feedback gain matrices F with
(5.47)

Now let us activate the noise term. Setting σ = 10 we get γ ≈ 0.6047 with F = [−87.4351, 0.2910, −447.5771, −1.8587] . For σ = 20 we have γ ≈ 6.2285 and F20 = [−84.3813, 0.2154, −421.5963, −1.7306] .

(5.48)

To get an idea of how our controllers work, we consider (for different F ) the closed-loop system dx = (A + B2 F )x dt + B1 v dt + A0 x dw + B10 v dw , z = Cx . We denote the solutions by x(·) = x(·; x0 , v) and the output process by z(·) = z(·; x0 , v) = Cx(·). If the system is stable, then the norm of z(·) is given by


<

∞ 0

F ? E
< 0

5.5 Numerical examples ∞

173

D = F ? trace CE x(t)x(t)T C T dt .

Applying the formula given in Remark 1.4.4(ii), we can dif? solve Tordinary F ferential equations for m(t) = E (x(t)) and P (t) = E x(t)x(t) , and thus D =

plot trace CP (t)C T on some interval. To this end we use the Runge-Kuttasolver ode45 implemented in MATLAB. As the disturbance input, we choose the simple step function ) 1 for 0 ≤ t ≤ 1 , v(t) = (5.49) 0 for 1 < t . First let us compare the controllers F0 and F20 from (5.47) and (5.48). We apply them for the cases σ = 0 and σ = 20. Noise level 0

−4

10

x 10

F0 F20

8

F0 F20

E||z(t)||2

6

2

E||z(t)||

x 10

8

6 4

4

2

2

0

0

−2

Noise level 20

−4

10

0

5

10 Time t in sec

15

−2

20

0

10

20 Time t in sec

30

40

In this special example, both controllers behave almost equally well. In fact, the deterministic control F0 behaves slightly better, even in the presence of noise. None of them, however stabilizes the system for e.g. σ = 21. Observe how slowly convergence takes place for σ = 20. LQ control

0.4

noise level 24 noise level 0

0.35 0.3 0.25

E||z(t)||2

The controller Fσmax defined in (5.44) guarantees stability up to the noise level σmax > 24.9. But it has poor disturbance attenuation behaviour (as was to be expected, since it is an LQcontroller). We plot the second moment of z(t) for the noise levels σ = 0 and σ = 24.

0.2

0.15 0.1 0.05 0

0

5

10 Time t in sec

15

20

174

5 Solution of the Riccati equation

5.5.3 The car-steering problem Now let us consider the car-steering problem presented in Section 1.9.5. We recall the linear state-space model for the four-wheel steering problem in the case of a stochastically perturbed adhesion coefficient µ, which were, basically, taken from [1]: D = (1) (2) dx = Ax dt + σ A0 x dw1 + A0 x dw2 D = (1) (2) + Bu dt + σ B0 u dw1 + B0 u dw2 , where

. A =

cr +cf cr Lr −cf Lf − 2mv 2mv22 cr Lr +cf L2f cr Lr +cf Lf − 2Jv 2J

−

and

. (1) A0

=

(1)

=

B0

0

c

c L

f f f − mv − mv 2

c f Lf J cf mv c f Lf J

c f L2

−6 Jvf 0 , 0

1

4

0 ,

B = .

4 ,

(2) A0

=

(2)

=

B0

cr − mv

c r Lr 0 Jcr 0 mv 0 crJLr

cf 2mv c f Lf 2J

cr 2mv r Lr − c2J

c r Lr mv 2 2 c r Lr − Jv 6

6 ,

4 ,

.

The meaning of the variables was explained in Section 1.9.5. To be more concrete, let us assume that the car in question in fact is a specific bus, for which [1] gives the following parameter values [with the corresponding units in square brackets]: ^f = 3.67 [m] , ^r = 1.93 [m] , ^0 = 6.12 [m] , cf = 198000 [N/rad] , cr = 470000 [N/rad] , v ∈ [1, 20] [ms−1 ] , m ∈ [9950, 16000] [kg] , J = i2 µm , i2 = 10.85[m2 ] . It is clear that the mass of the bus can vary significantly, depending on whether the bus is full or empty. Similarly, the velocity takes value in an interval. But both parameters can be assumed to be constant in a specific driving situation. If not stated otherwise, we will set v = 10 and m = 10000. Stabilization Proceeding as in the previous example we analyze the stabilizability of this system for different parameter values. We choose the weight matrix M = −I ∈ H4 and consider the Riccati inequality

5.5 Numerical examples

Rσ (X) = A∗ X + XA + σ 2

2 O

j Aj∗ 0 XA0 − I

j=1

 − XB σ 2

2 O

175

−1

B0j∗ XB0j − I 

B∗X > 0 .

j=1

The operator dual to Rσ is G σ (Y ) = −Y A∗ − AY − σ 2

2 O

−1 j Y Aj∗ A0 Y − I 0 Y

j=1

 − Y B −σ 2

2 O

−1

B0j∗ Y −1 B0j − I 

B∗Y .

j=1

If the equation G σ (Y ) has a stabilizing solution Y+ > 0, then the system is stabilizable, and  F = − σ 2

2 O

−1 B0j∗ Y −1 B0j + I 

B∗

j=1

defines a stabilizing feedback. Again, we apply the Newton iteration to the equation G σ (Y ) with initial matrix Y0 = 10I. The largest value of σ for convergence to occur is approximately σmax = 1.3367. The corresponding stabilizing feedback gain matrix is 0 6 0.8893 0.3745 F = 0.9414 0.1926 The eigenvalues of the closed-loop system have all real part less or equal than β = −0.0026. For the double speed v = 20 we have σmax = 0.8723. It is also of particular interest to determine a margin for robust stabilization with respect to bounded parameter uncertainties. That means, we would like to find an upper bound ∆, such that the system (1)

(2)

(1)

(2)

x˙ = Ax + δf (t)A0 x + δr (t)A0 x + Bu + δf (t)B0 u + δr (t)B0 u , is stable for arbitrary measurable functions δj : R+ → R satisfying max |δf /r (t)| < ∆ .

t∈R+

Remember that here δf /r is the deviation of the adhesion coefficient µf /r at the front or the rear wheel from its nominal value µf /r = 1/2. To find such a bound, we consider the inequality

176

5 Solution of the Riccati equation

Rσ (X) + αX > 0 . If this inequality has a solution X < 0, then by Proposition 1.6.2, the bound M α ∆=σ 2 complies with our requirements. It is an interesting question, how the parameter α should be chosen here. For simplicity, we try α = 2; for different velocities v we find the margins v 10 20 30 40 50 ∆ 0.4377 0.4131 0.4025 0.4002 0.3943 and the corresponding controllers. We conclude that one should drive slowly on ice. Disturbance attenuation Now let us consider the extended state space system 1 2 dx = (Ax + B1 v + B2 u) dt + (A10 x + B20 u)σ dw1 + (A20 x + B20 u)σ dw2 z = Cx ,

with

   A =  

cr +cf cr Lr −cf Lf − 2mv 2mv22 cr Lr +cf L2f cr Lr +cf Lf − 2Jv 2J

−

0 v   0 0  B1 =  v, 0  c − f  cfmv  Lf (1) A0 =  J  0 0  cf (1)

B20

1 ^0 

cf 2mv c f Lf 2J

 B2 =   0 0  c f Lf − mv2 0 0  cf L2f − Jv 0 0 , 0 0 0 0 00  mv 0  c f Lf 0  (2)  J =   0 0  , B20 = 0 0

100



 0 0 ,  0 0

v0

cr 2mv r Lr − c2J

0 0

   ,  

 00  c r Lr (2) 0 0 , J A0 =   0 0 0 0 0 0 00  T  cr  0 mv 0 0  0 c r Lr  J ,   C=  0 0  0 1 0 0 cr − mv

c r Lr mv 2 2 c r Lr − Jv

and the extended state-vector x = [β, r, ∆, µ]T (cf. Section 1.9.5). As before, we can determine the maximal noise intensity for the system to be stabilizable. We found this to be σmax = 1.0972 with the corresponding controller

0 Fmax =

5.5 Numerical examples

0.5486 0.2762 −0.0208 −0.0001 0.9064 0.1840 −0.0077 −0.0000

6

177

.

Requiring decay rate α = 1 the maximal σ is σ = 0.3522, such that the corresponding controller 0 6 −1.0124 −3.9174 −2.6474 −0.0355 F0 = 0.6139 2.8578 1.6995 −0.0606 guarantees stability for arbitrary µ ∈ [0.15, 0.85]. To address the disturbance attenuation problem, we have to regularize the system, as we did in (5.46), i.e. we set 0 6 0 6 0 6 z C 0 ˜ ˜ ˜ ˜ z˜ = = Cx + D2 u with C = , D2 = ε u. u 0 I As we have already mentioned, the choice of ε is a problem in itself. For different noise intensities σ, we found that we should choose different ε to obtain good attenuation bounds. In the following table we list up stabilizers we have found for different heuristic choices of σ and ε. σ

ε

γ

0

0 0.01 0.17 F1 = 0

0 0.1 1.62 F2 0.1 0.06 1.21 F3 0.1 0.1 1.94 F4 0.2 0.1 4.58 F5

−14.6115 −12.9933 −17.4868 −50.0112 −25.0023 −14.8357 −25.5760 −89.9460

6

−64.8261 −509.0833 −365.0595 12.2410 = 7.4033 118.1471 78.9999 −13.2715 6 0 −7.0193 −8.1538 −9.6202 −7.8461 = −6.6617 −3.8288 −7.4196 −8.8143 0 6 −5.3309 −8.4378 −8.7158 −4.6053 = −5.4354 −2.7635 −6.0313 −6.1384 6 0 −4.1149 −3.4930 −5.0236 −2.0495 = −1.9438 −1.7667 −3.0303 −1.3690

6

Note that we have chosen the value ε = 0.1 for all noise intensities σ = 0, σ = 0.1, and σ = 0.2, such that the results are comparable. The other values were chosen, to obtain better margins γ. Let us have a look at the behaviour of the different closed-loop systems, when the perturbation input v is the stepfunction defined in (5.49). As previously, we plot E|z(t)|2 over the time t. First we consider the controllers F1 –F4 in the situation, they are designed for, i.e. F1 and F2 in the absence of noise and F3 and F4 in the case of noise intensity σ = 0.1. As was to be expected, we can achieve much better disturbance attenuation properties in the absence of noise. But the value of γ, depending on the regularization parameter ε may be misleading. In the example, F2 outperforms F1 in spite of the much worse attenuation value.

178

5 Solution of the Riccati equation Noise intensity 0

−4

15

x 10

E|z(t)|2

E|z(t)|

F3 F4

0.15

2

10

5

0

−5

Noise intensity 0.1

0.2

F1 F2

0.1

0.05

0

0.5

1 Time t in sec

1.5

0

2

0

0.5

1 Time t in sec

1.5

2

Now we consider the controller F2 and F5 for different noise intensities. We see that the performance of F2 deteriorates rapidly, as σ increases. In fact, F2 fails to be stabilizing for σ = 0.0485, while F5 , designed for noise intensity σ = 0.2, still stabilizes the system. The controller F2

3.5 3

The controller F5

5

noise intensity 0.0484 noise intensity 0.0483

noise intensity 0.2 noise intensity 0.1 noise intensity 0

4

2

E|z(t)|2

E|z(t)|2

2.5

1.5

3 2

1 1

0.5 0

0

5

10 Time t in sec

15

20

0

0

0.5

1 Time t in sec

1.5

2

Let us compare these controllers designed for disturbance attenuation with the LQ-type controllers Fmax and F0 that guarantee maximal robustness margins. The controller Fmax

5

8

x 10

7

E|z(t)|2

6

The controller F0

10000

noise intensity 1 noise intensity 0.5 noise intensity 0

noise intensity 0.4 noise intensity 0.2 noise intensity 0

8000

5

6000

4 4000

3 2

2000

1 0

0

20

40 60 Time t in sec

80

100

0

0

2

4 6 Time t in sec

8

10

The difference is significant. Again, we observe the trade-off between good attenuation bounds and large robustness margins with respect to parameter uncertainty.

5.5 Numerical examples

179

Finally, let us have a look how the controllers F2 and F5 behave in the absence of noise, but for different fixed adhesion coefficients µ. Instead of the covariance E(|z(t)|2 ), we plot the deterministic output z(t) now. The controller F2

0.4 0.3

adhesion 0.85 adhesion 0.5 adhesion 0.15

6 4 z(t)

z(t)

0.2 0.1

2 0

0 −0.1

The controller F5

8

adhesion 0.85 adhesion 0.5 adhesion 0.15

−2 0

1

2 3 Time t in sec

4

5

−4

0

1

2 3 Time t in sec

4

5

We observe that F2 copes much better with the deterministic situation than F5 , which was designed for stochastic parameter uncertainty. Note, in particular, that in the example F2 behaves better for µ = 0.5 than for µ = 0.85 (it was designed for µ = 0.5). Some of these result have been published in [43].

A Hermitian matrices and Schur complements

Throughout the text, K denotes either the field of real or the field of complex numbers. For simplicity we write X ∗ rather than X T for the transpose of a real matrix and call a real symmetric matrix Hermitian. At some occasions we still need the notation X T for the transpose of a real or complex matrix – without conjugation. Let Hn ⊂ Kn×n denote the Hilbert space of real or complex n × n Hermitian matrices, endowed with the Frobenius inner product 1X, Y 2 = trace (XY ∗ ) = trace (XY ) and the corresponding (Frobenius) norm < · <. It should be noted that Hn is a real vector space, even in the case K = C. Its dimension is n(n + 1)/2 for K = R and n2 for K = C. A matrix X ∈ Hn is said to be nonnegative definite, if x∗ Xx ≥ 0 for all x ∈ Kn . In this case we write X ≥ 0. If for all x O= 0 the strict inequality x∗ Xx > 0 holds, then X is called positive n definite, and we write X > 0. This canonical ordering of Hn induced by H+ n n is also called Loewner ordering. For brevity, we sometimes write H− = −H+ . It is well-known that a matrix X ∈ Hn is positive definite if and only if all its eigenvalues are positive. More generally one defines the inertia of X ∈ Hn to be the triple In X = (π, ν, δ) ∈ N3 , where π, ν, δ denotes the sum of multiplicities of positive, negative and zero eigenvalues of X, respectively. If det X O= 0, then In X = In X −1 . In the following we discuss Schur complements as an important and elegant means to analyze definiteness and inertia of a matrix. We choose the names for variables such that the formulae can be applied easily in our context of Riccati equations (compare e.g. Section 2.3). 6 P S we define S∗ Q the Schur complement S(M/Q) = P − SQ−1 S ∗ with respect to Q, if Q is invertible, and the Schur complement S(M/P ) = Q − S ∗ P −1 S with respect to P , if P is invertible. 0

Definition A.1 For a Hermitian 2 × 2-block matrix M =

T. Damm: Rational Matrix Equations in Stochastic Control, LNCIS 297, pp. 181–184, 2004. Springer-Verlag Berlin Heidelberg 2004

182

A Hermitian matrices and Schur complements

The following lemma collects some important results ([160, Thms 2.7, 3.1, 3.3] and [32, Lemma 2.2]). Whenever the inverse of a matrix occurs in this Lemma, it is tacitly assumed, that the matrix is nonsingular. Lemma A.2 (i)

Inertia formula:

In M = In Q + In S(M/Q) = In P + In S(M/P ) . In particular M > 0 ⇐⇒ Q > 0 and P − SQ−1 S ∗ > 0. and if Q > 0, then M ≥ 0 ⇐⇒ P − SQ−1 S ∗ ≥ 0. (ii) Inversion formula: 6 6∗ 0 −1 6 0 0 −1 0 −P −1 S S P −1 −P −1 M = + S(M/P ) . 0 0 I I 0 6 Q2 S21 (iii) Quotient formula: If Q = then S12 Q1 N D = S(M/Q) = S S(M/Q1 ) S(Q/Q1 ) . ˜ be partitioned like M and set F = Q−1 S ∗ , (iv) Difference formula: Let M ˜ −1 S˜∗ , and ∆F = F − F˜ . Then F˜ = Q ˜ /Q − Q) ˜ = S(M/Q) − S(M ˜ /Q) ˜ + ∆F ∗ (Q−1 − Q ˜ −1 )−1 ∆F . S(M − M 6 0 U V (v) Transformation formula: Let T = have the same partitioning as 0 W M . Then S(T M T ∗ /W QW ∗ ) = U S(M/Q)U ∗ . Schur complements are used to establish the equivalence of Riccati-type matrix inequalities and linear matrix inequalities. For instance, the constrained inequality R(X) = P (X) − S(X)Q(X)−1 S(X)∗ > 0

with Q(X) > 0

is equivalent to the linear matrix inequality 6 0 P (X) S(X) >0. S(X)∗ Q(X) As a further illustrative application we note a simple consequence of Lemma A.2: Corollary A.3 If X, Y ∈ Hn are invertible and In X = In Y , then In(X − Y ) = In(Y −1 − X −1 ) . D = In particular, if X, Y > 0, then X > Y ⇐⇒ Y −1 > X −1 .

A Hermitian matrices and Schur complements

0

Proof: Consider the matrix M = By Lemma A.2 we have

6

183

X I . I Y −1

In M = In X + In(Y −1 − X −1 ) = In Y −1 + In(X − Y ) . Since In X = In Y = In Y −1 , the result follows.

"

˜ be partitioned as M with nonsingular Q ˜ and Q. AsCorollary A.4 Let M ˜ ˜ sume M ≤ M and In Q = In Q. ˜ /Q) ˜ and Then S(M/Q) ≥ S(M D = D = ˜ −1 S˜∗ . ˜ /Q) ˜ ⊂ Ker Q−1 S ∗ − Q Ker S(M/Q) − S(M ˜ then S(M/Q) > S(M ˜ /Q). ˜ If even M > M ˜ . Then, in particular, Q > Q ˜ and since Proof: We first assume M > M −1 −1 ˜ ˜ by assumption In Q = In Q, Corollary A.3 yields Q > Q . Hence, by ˜ /Q) since S(M/Q) > S(M the difference formula of Lemma A.2, we have D = ˜ 0 > 0. ˜ /Q0 − Q S M −M ˜ , then for small ε > 0, we can replace M by M + εI without Now, if M ≥ M affecting the assumptions. Hence, the first assertion holds by continuity. To prove the kernel inclusion, we have to take some more care with the perturbation argument. By the difference formula, the assertion is obvious, if ˜ In the general case, we consider the transformation Q > Q. 6 0 6 0 6 0 I 0 P − SQ−1 S ∗ 0 I −F ∗ M = . 0 I −F I 0 Q Applying e.g. a Sylvester transformations with unitary W1 , we have 0 6 Q+ 0 ∗ W1 QW1 = ∈ Hπ+ν 0 Q− with Q+ > 0 and Q− < 0. For θ > 1 we set Wθ = diag(θIπ , 1/θIν )W1 , such that 0 2 6 θ Q+ 0 Wθ QWθ∗ = > W1 QW1∗ . 0 θ−2 Q− With the inverse transformation matrix 0 0 6−1 0 6−1 6 I −F ∗ I F ∗ W1∗ I 0 T = = 0 I 0 W1∗ 0 W1 we define

184

A Hermitian matrices and Schur complements

0

0 6 6 P − SQ−1 S ∗ 0 Pθ Sθ ∗ Mθ = T , T =: 0 Wθ QWθ∗ Sθ∗ Qθ

˜ and Qθ > Q ≥ Q. ˜ Moreover, by the transformation such that Mθ ≥ M ≥ M formula, S(Mθ /Qθ ) = S(M/Q). It follows that for all θ > 1 we have the inclusion D = D = ˜ −1 S˜∗ . ˜ /Q) ˜ ⊂ Ker Q−1 S ∗ − Q Ker S(M/Q) − S(M θ θ ∗ −1 ∗ By construction limθ→1 Q−1 S , which proves the assertion. θ Sθ = Q

"

References

1. J. Ackermann. Robust Control. Springer-Verlag, 1993. 2. B. D. O. Anderson and J. B. Moore. Linear Optimal Control. Prentice Hall, Englewood Cliffs, New Jersey, 1971. 3. A. A. Andronov, A. A. Vitt, and L. S. Pontryagin. Statistische Auffassung dynamischer Systeme. Phys. Z. Sowjetunion, 6:1–24, 1934. 4. P. Ara and M. Mathieu. Local Multipliers of C ∗ -Algebras. Springer-Verlag, 2003. 5. W. Arendt. Resolvent positive operators. Proc. London Math. Soc., 54(3):321– 349, 1987. 6. L. Arnold. Stochastische Differentialgleichungen. R. Oldenburg Verlag, 1973. 7. L. Arnold, H. Crauel, and V. Wihstutz. Stabilization of linear systems by noise. SIAM J. Control Optim., 21:451–461, 1983. 8. V. I. Arnold. Gew¨ ohnliche Differentialgleichungen. Springer, 2001. 9. T. Ba¸sar and P. Bernhard. H ∞ -Optimal Control and Related Minimax Design auser, Boston, 2nd edition, 1995. Problems. A Dynamic Game Approach. Birkh¨ 10. G. P. Barker, R. D. Hill, and R. D. Haertel. On the completely positive and positive-semidefinite-preserving cones. Linear Algebra Appl., 56:221–229, 1984. 11. R. H. Bartels and G. W. Stewart. Solution of the matrix equation AX + XB = C: Algorithm 432. Comm. ACM, 15:820–826, 1972. 12. P. Benner and R. Byers. An exact line search method for solving generalized continuous-time algebraic Riccati equations. IEEE Trans. Autom. Control, 43(1):101–107, 1998. 13. A. Berman, M. Neumann, and R. J. Stern. Nonnegative Matrices in Dynamic Systems. John Wiley & Sons, New York, 1989. 14. A. Berman and R. J. Plemmons. Nonnegative Matrices in the Mathematical Sciences. Classics in Applied Mathematics. SIAM, 1994. 15. D. Bernstein and D. C. Hyland. Optimal projection equations for reducedorder modeling, estimation and control of linear systems with multiplicative noise. J. Optim. Th. & Appl., 58:387–409, 1988. 16. D. S. Bernstein. Robust static and dynamic output-feedback stabilization: Deterministic and stochastic perspectives. IEEE Trans. Autom. Control, AC32(12):1076–1084, 1987. 17. J.-M. Bismut. Linear-quadratic optimal stochastic control with random coefficients. SIAM J. Control, 14:419–444, 1976.

186

References

18. S. K. Biswas. Robust stabilization of linear systems in the presence of Gaussian perturbation of parameters. Optim. Contr. Appl. Meth., 19:271–286, 1998. 19. S. Bittanti, A. J. Laub, and J. C. Willems. The Riccati Equation. Communication and Control Engineering Series. Springer-Verlag, Berlin, Heidelberg, New York, 1991. 20. J. L. Bogdanoff and S. J. Citron. Experiments with an inverted pendulum subject to random parametric excitation. J. Acoustic Soc., 38(9):447–452, 1965. 21. S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix Inequalities in Systems and Control Theory, volume 15 of Studies in Applied Mathematics. SIAM, 1994. 22. R. W. Brockett. Finite Dimensional Linear Systems. John Wiley & Sons, New York, 1970. 23. L. Burlando. Monotonicity of spectral radius for positive operators on ordered Banach spaces. Arch. Math., 56:49–57, 1991. 24. D. Carlson and R. Hill. Generalized controllability and inertia theory. Linear Algebra Appl., 15:177–187, 1976. 25. B. M. Chen. Robust and H∞ control. Communications and Control Engineering Series. Springer-Verlag, 2000. 26. S. Chen, X. Li, and X. Y. Zhou. Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim., 36(5):1685–1702, 1998. 27. S. J. Cho, S.-H. Kye, and S. G. Lee. Generalized Choi maps in threedimensional matrix algebras. Linear Algebra Appl., 171:212–224, 1992. 28. M.-D. Choi. Positive linear maps on C ∗ -algebras. Canadian J. Math., 4:520– 529, 1972. 29. M.-D. Choi. Completely positive linear maps on complex matrices. Linear Algebra Appl., 10:285–290, 1975. 30. M.-D. Choi. Positive semidefinite biquadratic forms. Linear Algebra Appl., 4:95–100, 1975. 31. A. N. Churilov. On the solution of a quadratic matrix equation. In E. L. Tonkov, editor, Nonlinear oscillations and control theory. No.2., pages 24–33. Udmurt State University, Izhevsk, 1978. 32. D. J. Clements and H. K. Wimmer. Monotonicity of the optimal cost in the discrete-time regulator problem and Schur complements. Automatica, 37:1779– 1786, 2001. 33. E. G. Collins, Jr. and A. S. Hodel. Efficient solutions of linearly coupled Lyapunov equations. SIAM J. Matrix. Anal. Appl., 18(2):291–304, 1997. 34. W. A. Coppel. Matrix quadratic equations. Bull. Austr. Math. Soc., 10:377– 401, 1974. 35. R. F. Curtain, editor. Stability of Stochastic Dynamical Systems, volume 294 of Lecture Notes in Mathematics. Springer-Verlag, 1972. 36. R. F. Curtain and A. J. Pritchard. Functional analysis in modern applied mathematics, volume 132 of Mathematics in Science and Engineering. Academic Press, London, New York, 1977. 37. G. Da Prato. Direct solution of a Riccati equation arising in stochastic control theory. Applied Mathematics and Optimization, 11:191–208, 1984. 38. G. Da Prato and J. Zabczyk. Stochastic Equations in Infinite Dimensions, volume 45 of Encyclopedia of Mathematics and Its Applications. Cambridge University Press, 1992.

References

187

39. T. Damm. Generalized Riccati equations occuring in robust stochastic control. In V. Zakharov, editor, Proceedings of the 11th IFAC Workshop, Saint Petersburg, Russia, 3-6 July 2000. Pergamon Press, 2000. 40. T. Damm. State-feedback H ∞ -control of stochastic linear systems. In Third NICONET Workshop on Numerical Software in Control Engineering, pages 45–50, Louvain-la-Neuve, Belgium, 2001. 41. T. Damm. Minimal representations of inverted Sylvester and Lyapunov operators. Linear Algebra Appl., 363:35–41, 2002. 42. T. Damm. State-feedback H ∞ -type control of linear systems with time-varying parameter uncertainty. Linear Algebra Appl., 351–352:185–210, 2002. 43. T. Damm. A car-steering problem with random adhesion coefficient. PAMM Proc. Appl. Math. Mech., 2:83–84, 2003. 44. T. Damm. Stability of linear systems and positive semigroups of symmetric matrices. In L. Benvenuti, A. D. Santis, and L. Farina, editors, Positive Systems, volume 294 of Lecture Notes in Control and Information Sciences, pages 207–214. Springer-Verlag, 2003. 45. T. Damm and D. Hinrichsen. Matrix (in)equalities for linear stochastic systems. In Proceedings of MTNS-98, Padova, Italy, 1998. Il Poligrafio. 46. T. Damm and D. Hinrichsen. On a rational matrix equation occuring in stochastic control. In Proceedings of the 5th European Control Conference 1999, Karlsruhe, pages DM–2, Karlsruhe, 1999. 47. T. Damm and D. Hinrichsen. On the parameter dependence of a class of rational matrix equations occuring in stochastic optimal control. In Proceedings of MTNS-2000, page IS19A, Perpignan, France, 2000. 48. T. Damm and D. Hinrichsen. Newton’s method for a rational matrix equation occuring in stochastic control. Linear Algebra Appl., 332–334:81–109, 2001. 49. T. Damm and D. Hinrichsen. Newton’s method for concave operators with resolvent positive derivatives in ordered Banach spaces. Linear Algebra Appl., 363:43–64, 2003. 50. B. N. Datta. Stability and inertia. Linear Algebra Appl., 302–303:563–600, 1999. 51. J. de Pillis. Linear transformations which preserve Hermitian and positive semidefinite operators. Pacific J. Math., 23(1):129–137, 1967. 52. C. E. de Souza and M. D. Fragoso. On the existence of maximal solutions for generalized algebraic Riccati equations arising in stochastic control. Systems & Control Letters, 14:233–239, 1990. 53. E. de Souza and S. Bhattacharyya. Controllability, observability and the solution of AX − XB = C. Linear Algebra Appl., 39:167–188, 1981. 54. J. W. Demmel. Applied Numerical Linear Algebra. SIAM, Philadelphia, 1997. 55. J. Dieudonn´e. Foundations of Modern Analysis, volume 10-I of Pure and Applied Mathematics. Academic Press, New York, 1969. 56. D. Z. Djokovi´c. Characterization of Hermitian and skew-Hermitian maps between matrix algebras. Linear Algebra Appl., 12:165–170, 1975. 57. V. Dr˘ agan, A. Halanay, and A. Stoica. An LMI solution to a disturbance attenuation problem with state feedback for stochastic systems. Rev. Roum. Sci. Tech. Elektrotech. Energ., 41(4):513–519, 1996. 58. V. Dr˘ agan, A. Halanay, and A. Stoica. A small gain theorem for linear stochastic systems. Systems & Control Letters, 30:243–251, 1997.

188

References

59. V. Dr˘ agan, T. Morozan, and A. Halanay. Optimal stabilizing compensator for linear systems with state dependent noise. Stochastic Analysis and Application, 10:557–572, 1992. 60. F. Dullerud, G. E.; Paganini. A course in robust control theory, volume 36 of Texts in Applied Mathematics. Springer-Verlag, New York, 2000. 61. A. El Bouhtouri, D. Hinrichsen, and A. J. Pritchard. H ∞ type control for discrete-time stochastic systems. Int. J. Robust & Nonlinear Control, 9(13):923–948, 1999. 62. A. El Bouhtouri, D. Hinrichsen, and A. J. Pritchard. On the disturbance attenuation problem for a wide class of time invariant linear stochastic systems. Stochastics and Stochastics Reports, 65:255–297, 1999. 63. A. El Bouhtouri and A. J. Pritchard. A Riccati equation approach to maximizing the stability radius of a linear system by state feedback under structured stochastic Lipschitzian perturbations. Systems & Control Letters, 21:475–484, 1993. 64. L. El Ghaoui. State-feedback control of systems with multiplicative noise via linear matrix inequalities. Systems & Control Letters, 24(3):223–228, 1995. 65. L. Elsner. Monotonie und Randspektrum bei vollstetigen Operatoren. Arch. Ration. Mech. Anal., 36:356–365, 1970. 66. L. Elsner. Quasimonotonie und Ungleichungen in halbgeordneten R¨ aumen. Linear Algebra Appl., 8:249–261, 1974. 67. M. Fiedler and V. Pt` ak. On matrices with nonpositive off-diagonal elements and positive principal minors. Czech. Math., 12:382–400, 1962. 68. A. Fischer, D. Hinrichsen, and N. K. Son. Stability radii of Metzler operators. Vietnam J. Math, 26:147–163, 1998. 69. M. D. Fragoso, O. L. V. Costa, and C. E. de Souza. A new approach to linearly perturbed Riccati equations in stochastic control. Applied Mathematics and Optimization, 37:99–126, 1998. 70. B. A. Francis. A course in H∞ control theory. Number 88 in Lecture Notes in Control and Information Sciences. Springer-Verlag, Berlin-Heidelberg-New York, 1987. 71. G. Freiling and V. Ionescu. Monotonicity and convexity properties of matrix Riccati equations. IMA J. Math. Control Inf., 18(1):61–72, 2001. 72. G. Freiling, G. Jank, and H. Abou-Kandil. Generalized Riccati difference and differential equations. Linear Algebra Appl., 241-243:291–303, 1996. 73. A. Friedman. Stochastic Differential Equations and Applications. Number 28 in Probability and mathematical statistics. Academic Press, New York, 1975. ¨ 74. G. Frobenius. Uber Matrizen aus positiven Elementen. Sitzungsberichte der K¨ onigl. Preuss. Akad. Wiss, pages 471–476, 1908. 75. P. Gahinet and P. Apkarian. A linear matrix inequality approach to H∞ control. Int. J. Robust & Nonlinear Control, 4:421–448, 1994. 76. P. Gahinet and A. J. Laub. Numerically reliable computation of optimal performance in singular H∞ control. SIAM J. Control Optim., 35(5):1690–1710, 1997. 77. Z. Gaji´c and M. T. J. Qureshi. Lyapunov Matrix Equation in System Stability and Control, volume 195 of Mathematics in Science and Engineering. Academic Press, San Diego, California, 1995. 78. F. R. Gantmacher. Matrizentheorie. Springer, 1986. 79. Z. Y. Gao and N. U. Ahmed. Stabilizability of certain stochastic systems. Int. J. Systems Sci., 17(8):1175–1185, 1986.

References

189

80. T. C. Gard. Introduction to Stochastic Differential Equations, volume 114 of Monographs and Textbooks in Pure and Applied Mathematics. Marcel Dekker, 1988. 81. I. I. Gikhman. On the theory of differential equations of stochastic processes. I, II. Am. Math. Soc., Transl., II. Ser. 1:111–137, 139–161, 1955. 82. I. I. Gikhman and A. V. Skorokhod. Stochastic Differential Equations. Springer-Verlag, 1972. 83. S. K. Godunov. Modern Aspects of Linear Algebra, volume 175 of Translations of Mathematical Monographs. American Mathematical Society, Providence, Rhode Island, 1998. 84. I. Gohberg, P. Lancaster, and L. Rodman. On the Hermitian solutions of the symmetric algebraic Riccati equation. SIAM J. Control Optim., 24:1323–1334, 1986. 85. G. H. Golub, S. Nash, and C. F. Van Loan. A Hessenberg-Schur method for the problem AX + XB = C. IEEE Trans. Autom. Control, AC-24:909–913, 1979. 86. G. H. Golub and C. F. van Loan. Matrix Computations, volume 3 of Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins University Press, Baltimore, Maryland, 1990. 87. M. Green and D. J. N. Limebeer. Linear Robust Control. Prentice-Hall, Englewood Cliffs, New Jersey, 1995. 88. A. Greenbaum. Iterative Methods for Solving Linear Systems, volume 17 of Frontiers in Applied Mathematics. SIAM, Philadelphia, 1997. 89. R. Griesbaum. Zur Stabilit¨ at dynamischer Systeme mit stochastischer Parametererregung. PhD thesis, Fakult¨ at f¨ ur Maschinenbau, Universit¨ at Karlsruhe, 1999. 90. P. Gritzmann, V. Klee, and B.-S. Tam. Cross-positive matrices revisited. Linear Algebra Appl., 223/224:285–305, 1995. 91. C.-H. Guo. Newton’s method for discrete algebraic Riccati equations when the closed-loop matrix has eigenvalues on the unit circle. SIAM J. Matrix. Anal. Appl., 20(2):279–294, 1998. 92. C.-H. Guo. Iterative solution of a matrix Riccati equation arising in stochastic control. Preprint, Department of Mathematics and Statistics, University of Regina, 2000. 93. C.-H. Guo. Nonsymmetric algebraic Riccati equations and Wiener-Hopf factorization of M -matrices. SIAM J. Matrix. Anal. Appl., 23(1):225–242, 2001. 94. C.-H. Guo and P. Lancaster. Analysis and modification of Newton’s method for algebraic Riccati equations. Math. Comp., 67(223):1089–1105, 1998. 95. S. J. Hammarling. Numerical solution of the stable, non-negative definite Lyapunov equation. IMA J. Numer. Anal., 2:303–323, 1982. 96. B. Hassibi, A. H. Sayed, and T. Kailath. Indefinite-quadratic estimation and control: A unified approach to H 2 and H ∞ theories, volume 15 of Studies in Applied Mathematics. SIAM, 1999. 97. U. G. Haussmann. Optimal stationary control with state and control dependent noise. SIAM J. Control, 9(2):713–739, 1971. 98. U. G. Haussmann. Stabilization of linear systems with multiplicative noise. In R. F. Curtain, editor, Stability of Stochastic Dynamical Systems, volume 294 of Lecture Notes in Mathematics, pages 125–130. Springer-Verlag, July 1972. 99. M. L. J. Hautus. Controllability and observability conditions of linear autonomous systems. Indag. Math., 31:443–448, 1969.

190

References

100. M. L. J. Hautus. Operator substitution. Linear Algebra Appl., 205-206:713– 739, 1994. 101. U. Helmke and J. B. Moore. L2 sensitivity minimization of linear system representations via gradient flows. J. Math. Syst. Estim. Control, 5(1):79–98, 1995. 102. G. A. Hewer. An iterative technique for the computation of the steady state gains for the discrete optimal regulator. IEEE Trans. Autom. Control, AC16:382–384, 1971. 103. G. A. Hewer. Existence theorems for positive semidefinite and sign indefinite stabilizing solutions of H∞ Riccati equations. SIAM J. Control Optim., 31:16– 29, 1993. 104. R. D. Hill. Linear transformations which preserve Hermitian matrices. Linear Algebra Appl., 6:257–262, 1973. 105. R. D. Hill and S. R. Waters. On the cone of positive definite matrices. Linear Algebra Appl., 90:81–88, 1987. 106. D. Hinrichsen and A. J. Pritchard. Stability radii with stochastic uncertainty and their optimization by output feedback. SIAM J. Control Optim., 34:1972– 1998, 1996. 107. D. Hinrichsen and A. J. Pritchard. Stochastic H∞ . SIAM J. Control Optim., 36(5):1504–1538, 1998. 108. D. Hinrichsen and A. J. Pritchard. Mathematical Systems Theory, chapter 1: Mathematical Models. Manuscript, Bremen, 2001. 109. D. Hinrichsen and N. K. Son. µ-analysis and robust stability of positive linear systems. Appl. Math. Comput., 8(2):253–268, 1998. 110. M. Hochbruck and G. Starke. Preconditioned Krylov subspace methods for Lyapunov matrix equations. SIAM J. Matrix. Anal. Appl., 16(1):156 – 171, 1994. see also: IPS Research Report 92–17, ETH Z¨ urich, Switzerland (1992). 111. A. Hochhaus. Existenz-, Konvergenz- und Vergleichss¨ atze f¨ ur verallgemeinerte Riccatische Matrix-Gleichungen. PhD thesis, Fakult¨ at f¨ ur Naturwissenschaften, Universit¨ at Duisburg, 2002. 112. R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, Cambridge, Massachusetts, 1985. 113. R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, Cambridge, 1991. 114. M. S. Howe. The mean square stability of an inverted pendulum subject to random parametric excitation. J. Sound Vibration, 32(3):407–421, 1974. 115. N. Ikeda and S. Watanabe. Stochastic Differential Equations and Diffusions Processes. North Holland, 1981. 116. V. Ionescu, C. Oara, and M. Weiss. Generalized Riccati Theory and Robust Control: A Popov Function Approach. John Wiley & Sons, Chichester, 1999. 117. K. Ito. On a stochastic integral equation. Proc. Japan Acad., 22(1–4):32–35, 1946. 118. K. Ito. On stochastic differential equations. Mem. Am. Math. Soc., 4, 1951. 119. R. E. Kalman. Contributions to the theory of optimal control. Bol. Soc. Matem. Mexico, 5:102–119, 1960. 120. R. E. Kalman. Mathematical description of linear dynamical systems. SIAM J. Control, 1:152–192, 1963. 121. R. E. Kalman, P. L. Falb, and M. A. Arbib. Topics in mathematical system theory. McGraw-Hill, New York, 1969.

References

191

122. L. V. Kantorovich. Functional analysis and applied mathematics. Usp. Mat. Nauk, 3(6):89–185, 1948. 123. L. V. Kantorovich and G. P. Akilov. Functional Analysis in Normed Spaces. Pergamon Press, New York, 1964. Translated by D. E. Brown, edited by A. P. Robertson. 124. J. Kao and V. Wihstutz. Stabilization of companion form systems by mean square noise. Stochastics Stochastics Rep., 49:1–25, 1994. 125. I. Karatzas and S. E. Shreve. Brownian Motion and Stochastic Calculus. 2nd ed., volume 113 of Graduate Texts in Mathematics. Springer-Verlag, New York, 1991. 126. I. Y. Kats and N. N. Krasovskij. On the stability of systems with random parameters. J. Appl. Math. Mech., 24:1225–1246, 1961. 127. V. L. Kharitonov. Asymptotic stability of an equilibrium position of a family of systems of linear differential equations. J. Diff. Eqns., 11:2086–2088, 1978. 128. V. L. Kharitonov. Distribution of the roots of the characteristic polynomial of an autonomous system. Autom. Remote Control, 42(5):589–593, 1981. 129. V. L. Kharitonov. Robust stability analysis of time delay systems: A survey. In Commande et Structure des Syst`emes, pages 1–12, Nantes, 1998. Conference IFAC. 130. R. Z. Khasminskij. Stochastic Stability of Differential Equations. Sijthoff & Noordhoff, Alphen aan den Rijn, NL, 1980. 131. D. L. Kleinman. On an iterative technique for Riccati equation computations. IEEE Trans. Autom. Control, AC-13:114–115, 1968. 132. D. L. Kleinman. On the stability of linear stochastic systems. IEEE Trans. Autom. Control, AC-14:429–430, 1969. 133. D. L. Kleinman. Numerical solution of the state dependent noise problem. IEEE Trans. Autom. Control, AC-21:419–420, 1976. 134. P. E. Kloeden and E. Platen. Numerical Solution of Stochastic Differential Equations. Springer-Verlag, Berlin, 1995. 135. H.-W. Knobloch and H. Kwakernaak. Lineare Kontrolltheorie. Springer-Verlag, Berlin, 1985. ¨ die analytischen Methoden in der Wahrscheinlichkeit136. A. N. Kolmogorov. Uber srechnung. Mathematische Annalen, 104:415–458, 1931. 137. M. A. Krasnosel’skii, G. M. Vainikko, P. P. Zabreiko, Y. B. Rutitskii, and V. Y. Stetsenko. Approximate Solution of Operator Equations. Wolters-Noordhoff, Groningen, 1972. 138. M. A. Krasnosel’skij. Positive Solutions of Operator Equations. P. Noordhoff Ltd., Groningen, The Netherlands, 1964. 139. M. A. Krasnosel’skij, J. A. Lifshits, and A. V. Sobolev. Positive Linear Systems - The Method of Positive Operators, volume 5 of Sigma Series in Applied Mathematics. Heldermann Verlag, Berlin, 1989. 140. M. G. Krein and M. A. Rutman. Linear operators leaving invariant a cone in a Banach space. Amer. Math. Soc. Transl., 26:199–325, 1950. 141. N. V. Krylov. Controlled Diffusion Processes, volume 14 of Applications of Mathematics. Springer-Verlag, New York, 1980. 142. N. V. Krylov. Introduction to the Theory of Diffusion Processes, volume 142 of Translations of Mathematical Monographs. American Mathematical Society, 1995. 143. H. J. Kushner. Stochastic stability and control. Academic Press, New York, 1967.

192

References

144. H. Kwakernaak and R. Sivan. Linear Optimal Control Systems. John Wiley & Sons, New York, 1972. 145. P. Lancaster and L. Rodman. Algebraic Riccati Equations. Oxford, 1995. 146. C.-K. Li and H. J. Woerdeman. Special classes of positive and completely positive maps. Linear Algebra Appl., 255:247–258, 1997. 147. G. Lindblad. On the generators of quantum dynamical semigroups. Commun. Math. Phys., 1:219–224, 1976. 148. D. G. Luenberger. Optimization by Vector Space Methods. John Wiley & Sons, New York, 1969. 149. I. Marek. Frobenius theory of positive operators: Comparison theorems and applications. SIAM J. Appl. Math., 19:607–628, 1970. 150. P. J. McLane. Optimal stochastic control of linear systems with state- and control-dependent disturbances. IEEE Trans. Autom. Control, AC-16(6):793– 798, 1971. 151. V. Mehrmann. The Autonomous Linear Quadratic Control Problem, Theory and Numerical Solution. Number 163 in Lecture Notes in Control and Information Sciences. Springer-Verlag, Heidelberg, July 1991. 152. B. P. Molinari. The time invariant linear quadratic optimal control problem. Automatica, 13:347–357, 1977. 153. T. Morozan. Stabilization of some stochastic discrete-time control systems. Stochastic Analysis and Application, 1:89–116, 1983. 154. T. Morozan. Parametrized Riccati equations associated to input-output operators for time-varying stochastic differential equations with state-dependent noise. Technical Report 37, Institute of Mathematics of the Romanian Academy, 1995. 155. R. Nabben. A characterization of M-matrices. Preprint 95-002, SFB 343, Universit¨ at Bielefeld, 1995. 156. B. Oeksendal. Stochastic Differential Equations. Springer-Verlag, 5th edition, 1998. 157. A. Ostrowski and H. Schneider. Some theorems on the inertia of general matrices. J. Math. Anal. Appl., 4:72–84, 1962. ¨ ¨berwiegender Hauptdiagonale. die Determinanten mit u 158. A. M. Ostrowski. Uber Comm. Math. Helv., 10:69–96, 1937. 159. D. Ottl and C. R¨ ommich. Schwingungen von Autor¨ adern infolge mechanisch unrunder Reifen. Automobiltechnische Zeitschrift, 98(6):351–357, 1996. 160. D. V. Oulette. Schur complements and statistics. Linear Algebra Appl., 36:187– 295, 1981. 161. T. Penzl. Numerical solution of generalized Lyapunov equations. Adv. Comput. Math., 8(1–2):33–48, 1998. ¨ 162. O. Perron. Zur Theorie der Ubermatrizen. Math. Ann., 64:248–263, 1907. 163. I. R. Petersen, V. A. Ugrinovskii, and A. V. Savkin. Robust Control Design Using H ∞ -Methods. Springer-Verlag, London, 2000. 164. Y. A. Phillis. Optimal stabilization of stochastic systems. J. Math. Anal. Appl., 94:489–500, 1983. 165. H. Radjavi and P. Rosenthal. Simultaneous triangularization. Springer-Verlag, New York, 2000. 166. A. C. M. Ran and R. Vreugdenhil. Existence and comparison theorems for algebraic Riccati equations for continuous- and discrete-time systems. Linear Algebra Appl., 99:63–83, 1988.

References

193

167. W. T. Reid. Riccati Differential Equations. Academic Press, New York, 1972. 168. M. Reurings. Symmetric Matrix Equations. PhD thesis, Vrije Universiteit Amsterdam, 2003. 169. S. Richter, L. D. Davis, and E. G. Collins, Jr. Efficient computation of the solutions to modified Lyapunov equations. SIAM J. Matrix. Anal. Appl., 14(2):420–431, April 1993. 170. H. H. Rosenbrock. State Space and Multivariable Theory. Nelson-Wiley, 1970. 171. P. Sagirow. Stochastic Methods in the Dynamics of Satellites. Number 57 in Courses and lectures: International Centre for Mechanical Sciences. SpringerVerlag, Wien, New York, 1970. 172. A. A. Sagle and R. E. Walde. Introduction to Lie Groups and Lie Algebras, volume 51 of Pure and Applied Mathematics. Academic Press, New York– London, 1973. 173. J. C. Samuels and A. C. Eringen. On stochastic linear systems. J. Math. Phys., 38:83–103, 1959. 174. N. R. Sandell. On Newton’s method for Riccati equation solution. IEEE Trans. Autom. Control, AC-19:254–255, 1974. 175. T. Sasagawa. LP -stabilization problem for linear stochastic control systems with multiplicative noise. J. Optim. Th. & Appl., 61(3):451–471, 1989. 176. T. Sasagawa and J. L. Willems. Parametrization method for calculating exact stability bounds of stochastic linear systems with multiplicative noise. Automatica, 32(12):1741–1747, 1996. 177. H. H. Schaefer. Topological Vector Spaces. Springer-Verlag, Berlin, Heidelberg, New York, 1971. 178. C. W. Scherer. The Riccati Inequality and State-Space H∞ -Optimal Control. PhD thesis, Mathematical Institute, University of W¨ urzburg, 1990. 179. C. W. Scherer. The state feedback H∞ -problem at optimality. Automatica, 30:293–305, 1994. 180. H. Schneider. Positive operators and an inertia theorem. Numer. Math., 7:11– 17, 1965. 181. H. Schneider and M. Vidyasagar. Cross-positive matrices. SIAM J. Numer. Anal., 7(4):508–519, 1970. 182. M. A. Shayman. Geometry of the algebraic Riccati equation, Part I. SIAM J. Control Optim., 21:375–394, 1983. 183. M. A. Shayman. Geometry of the algebraic Riccati equation, Part II. SIAM J. Control Optim., 21:395–409, 1983. 184. E. D. Sontag. Mathematical Control Theory, Deterministic Finite Dimensional Systems. Springer-Verlag, New York, 2nd edition, 1998. 185. R. J. Stern and H. Wolkowitz. Exponential nonnegativity on the ice-cream cone. SIAM J. Matrix. Anal. Appl., 12:755–778, 1994. 186. G. W. Stewart and J. Sun. Matrix Perturbation Theory. Computer Science and Scientific Computing. Academic Press, Boston, 1990. 187. W. F. Stinespring. Positive functions on C ∗ -algebras. Proc. Amer. Math. Soc., 6:211–216, 1955. 188. R. L. Stratonoviˇc. A new form of representing stochastic integrals and equations. Vestnik Moskov. Univ., Ser. I 19(1):3–12, 1964. 189. W.-S. Tang. On positive linear maps between matrix algebras. Linear Algebra Appl., 79:45–51, 1986.

194

References

190. G. Tessitore. Some remarks on the Riccati equation arising in an optimal control problem with state- and control- dependent noise. SIAM J. Control Optim., 30(3):717–744, May 1992. 191. G. Tessitore. On the mean-square stabilizability of a linear stochastic differential equation. In J.-P. Zolesio, editor, Boundary Control and Variation, volume 163 of Lect. Notes Pure Appl. Math., pages 383–400, New York, June 1994. 5th Working Conference held in Sophia Antipolis, France, Marcel Dekker. 192. H. L. Trentelman, A. A. Stoorvogel, and M. L. J. Hautus. Control Theory for Linear Systems. Springer-Verlag, London, 2001. 193. M. Turelli. Random environments and stochastic calculus. Theor. Popul. Biol., 12:140–178, 1977. 194. V. A. Ugrinovskii. Robust H ∞ -control in the presence of stochastic uncertainty. Internat. J. Control, 71(2):219–237, 1998. 195. V. A. Ugrinovskii and I. R. Petersen. Absolute stabilization and minimax optimal control of uncertain systems with stochastic uncertainty. SIAM J. Control Optim., 37(4):1089–1122, 1999. 196. J. S. Vandergraft. Newton’s method for convex operators in partially ordered spaces. SIAM J. Numer. Anal., 4(3):406–432, 1967. 197. J. S. Vandergraft. Spectral properties of matrices which have invariant cones. SIAM J. Appl. Math., 16(6):1208–1222, 1968. 198. R. S. Varga. Matrix Iterative Analysis. Prentice Hall, Englewood Cliffs, New Jersey, 1962. 199. R. S. Varga. On recurring theorems on diagonal dominance. Linear Algebra Appl., 13:1–9, 1976. 200. W. J. Vetter. Vector structures and solutions of linear matrix equations. Linear Algebra Appl., 10:181–188, 1975. 201. J. C. Willems. Least squares optimal control and the algebraic Riccati equation. IEEE Trans. Autom. Control, AC-16:621–634, 1971. 202. J. L. Willems. Moment stability of linear white noise and coloured noise systems. In B. e. a. Clarkson, editor, Stoch. Probl. in Dyn., Symp. Univ. Southampton 1976, pages 67–89, 1977. 203. J. L. Willems and D. Aeyels. Moment stability of linear stochastic systems with solvable Lie algebras. IEEE Trans. Autom. Control, AC-21:285, 1976. 204. J. L. Willems and J. C. Willems. Feedback stabilizability for stochastic systems with state and control depending noise. Automatica, 12:277–283, 1976. 205. J. L. Willems and J. C. Willems. Robust stabilization of uncertain systems. SIAM J. Control Optim., 21:352–374, 1983. 206. H. K. Wimmer. Inertia theorems for matrices, controllability and linear vibration. Linear Algebra Appl., 8:337–343, 1974. 207. H. K. Wimmer. On the algebraic Riccati equation. Bull. Austral. Math. Soc., 14:457–461, 1976. 208. H. K. Wimmer. Monotonicity of maximal solutions of algebraic Riccati equations. Systems & Control Letters, 5:317–319, 1985. 209. H. K. Wimmer. Linear matrix equations, controllability and observability, and the rank of solutions. SIAM J. Matrix. Anal. Appl., 9:570–578, 1988. 210. H. K. Wimmer. Geometry of the discrete-time algebraic Riccati equation. J. Math. Syst. Estim. Control, 2(1):123–132, 1992. 211. W. M. Wonham. Optimal stationary control of a linear system with statedependent noise. SIAM J. Control, 5(3):486–500, 1967.

References

195

212. W. M. Wonham. On a matrix Riccati equation of stochastic control. SIAM J. Control, 6:681–697, 1968. 213. W. M. Wonham. Random differential equations in control theory. In A. T. Bharucha-Reid, editor, Probab. Methods Appl. Math., volume 2, pages 131–212, New York - London, 1970. Academic Press. 214. W. M. Wonham. Linear Multivariable Control: A Geometric Approach. Springer-Verlag, Heidelberg, 2nd edition, 1979. 215. S. L. Woronowicz. Positive linear maps of low dimensional matrix algebras. Rep. Math. Phys., 10:165–183, 1976. 216. V. A. Yakubovich. A frequency theorem in control theory. Siberian Mathematical Journal, 14:265–289, 1973. 217. W.-Y. Yan, J. B. Moore, and U. Helmke. Recursive algorithms for solving a class of nonlinear matrix equations with applications to certain sensitivity optimization problems. SIAM J. Control Optim., 32:1559–1576, 1994. 218. J. Yong and X. Y. Zhou. Stochastic Controls, volume 43 of Applications of Mathematics. Springer, New York, 1999. 219. G. Zames. Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inverses. IEEE Trans. Autom. Control, AC-26:301–320, 1981. 220. K. Zhou, J. C. Doyle, and K. Glover. Robust and Optimal Control. Prentice Hall, New Jersey, 1995. 221. K. Zhou and P. P. Khargonekar. An algebraic Riccati equation approach to H ∞ optimization. Systems & Control Letters, 11(2):85–91, 1988.

Index

Bounded Real Lemma, 50 bounded uncertainty, 19, 56

LQ-control, 43, 146, 159 Lyapunov function, 21, 27 operator, 13, 23, 63, 75, 85

completely positive operator, 73, 89, 94, 117 concavity, 104 constraint(definite/indefinite), 56, 124 controllability, 24, 151 convex cone, 61

M-matrix, 63, 119

detectability, 29, 65, 94 anti-, 30 disturbance attenuation, 33, 35, 38, 50, 158, 171, 176 duality transformation, 132

observability, 24 order interval, 61 ordering, 61

expectation, 1 extremal, 62

Newton’s method, 106, 162 double steps, 113, 169 modified, 110

perturbation operator, 48 positive operator, 13, 62

Kronecker product, 69

regular splitting, 67 regular system, 53, 133 regularization, 177 resolvent positive operator, 62 Riccati equation, 55, 145, 155 dual, 131, 170 non-symmetric, 118 standard, 120, 130, 164 inequality, 27, 53 coupled pair, 59 dual, 171 operator, 48, 134 dual, 138, 155, 167, 175

Loewner ordering, 181

Schur-complement, 54, 181

Frobenius inner product, 62, 181 Gˆ ateaux differentiability, 104 H∞ -control, 48, 51, 56 Hermitian-preserving, 71 inertia, 181 Itˆ o formula, 6 integral, 3 process, 4

198

Index

spectral abscissa, 63 radius, 63 stability, 10 mean-square, 12, 13 second moment, 12 stochastic, 11 stabilizability, 22 stabilizing matrix, 136, 164 solution, 107, 131, 145, 162 stable equation, 11 equilibrium, 10 internally/externally, 47

matrix, 63 operator, 66, 106 Stein operator, 63, 76, 85 stochastic differential equation, 5 process, 1 Stratonovich integral, 4 interpretation, 4, 7, 10, 17, 18, 21, 27 Sylvester operator, 98 white noise, 2 Wiener process, 2 Z-matrix, 63

Notation Sets, matrices and linear operators N natural numbers {0, 1, 2, . . .} R, C field of real, complex numbers K R or C Re z real part of a complex number z Im z imaginary part of a complex number z i imaginary unit or index, depending on context vector space of matrices with m rows and n columns Km×n equal to Km×1 Km ¯ A complex conjugate of a matrix (or a number) A AT transpose of a matrix A over K conjugate transpose of a matrix A over K A∗ Hn ordered Hilbert space of n × n Hermitian matrices over K n H+ cone of positive semidefinite matrices in Hn n n H− equal to −H+ n B, Bi basis of H (see Section 3.4) >, ≥, <, ≤ order relations in R, or in ordered vector spaces m × m unit matrix Im I unit matrix, whose dimension is clear by context (m) ei i-th unit vector in Km (mn) (m) (n) Eij equal to ei (ej )T (Ipq ⊗ T )(E (mn) ) see equation (3.2) Ker kernel of a matrix or an operator im image of a matrix or an operator rk rank of a matrix or an operator σ(T ) spectrum of an operator or a matrix T R ρ(T ) spectral radius of T , equal to max{|λ| R λ ∈ R σ(T )} β(T ) spectral bound of T , equal to max{Re(λ) R λ ∈ σ(T )} trace trace of a matrix 1X, Y 2 equal to trace XY for X, Y ∈ Hn det determinant In inertia (see appendix) S(M/Q) Schur-complement (see appendix) vec rearranges m × n-matrix in mn-vector (see Def. 3.3.1) ⊗ Kronecker product (see Def. 3.3.1) Lyapunov operator LA : X L→ A∗ X + XA LA PN j 1 N ΠA 0 the operator ΠA0 : X L→ j=1 Aj∗ 0 XA0 , for given A0 , . . . , A0 . ∗ Stein operator SA : X L→ A XA − X SA SA,B Sylvester operator SA,B : X L→ AX − XB L perturbation operator (see Section 2.2)

200

Notation

Stochastic analysis (see Section 1.1) (Ω, F , µ) probability space L2 (Ω, Kk ) Hilbert space of square integrable k-dimensional random variables w (normed) Wiener process E expectation increasing sequence of σ-algebras (Ft )t∈R+ L2w L2 -space of non-anticipating processes (with respect to Ft ) Cw almost surely continuous processes in L2w Banach spaces int, cl B(x0 , ε) X∗ 1x, v2 T ∗ : X∗ → X∗ C∗ [x, y], ]x, y[ f :X→X dom f fx3 R : Hn → Hn G : Hn → Hn dom+ , dom±

and nonlinear operators topological interior, closure of a set ball with center x0 and radius ε dual space of a Banach space X duality product of x ∈ X and v ∈ X ∗ , equal to v(x) adjoint of a linear operator T : X → X dual cone of a cone in a Banach space (see Section 3.1) closed, open order interval, respectively (see Section 3.1) nonlinear mapping on a Banach space X domain of f derivative of f at x (see Section 4.1) Riccati operator (see Section 5.1) dual Riccati operator (see Section 5.1.6) target sets (see Sections 5.1.1 and 5.1.2)