Theory of Difference Equations: Numerical Methods and Applications

= xp +
Applying A to both sides, the above relation becomes p = p

which reduces to (x split into

+ (x + l)Ap +

+ 1)Ap +
-
= 0,

(1.5.6)

~p

and
= -(x + 1)~p.

(1.5.7)

1.5.

13

Difference Equations

Hence, from (1.5.6), p(x) = w(x), where w(x) is an arbitrary function of period one and the corresponding solution of (1.5.5) is

+ ¢(w(x)).

y(x) = xw(x)

The solution of (1.5.7) gives rise to other solutions of (1.5.5) (see problem 1.23 for an example). In the continuous case, it is known that the solutions of the Clairaut equation can be obtained by solving two differential equations corresponding to (1.5.6) and (1.5.7) and that the solution of the second one is the envelope of the solutions of the first one. In the discrete case we cannot speak about envelopes. The solutions of (1.5.7) can have common points with the solutions of (1.5.6), because for equations in the form (1.5.5), the property of uniqueness of solutions may not be true. Analogous to the differential equation of Riccati type, we have the difference equation y(x)y(x

+ 1) + PI(x)y(x + 1) + pix)y(x) + P3(X)

where PI (x), P2(X),P3(X) are arbitrary functions defined on

=

0,

(1.5.8)

J;.

Equation (1.5.8) can be reduced to a linear one by setting y(x) = z(x + 1) . . . z(x) - PI(X). The resulting equation IS z(x

+ 2) + [P2(X) -

+ [P3(X) -

PI(X)]Z(x

+ 1)

PI(X)P2(X)]Z(x) = 0.

(1.5.9)

In the study of iterative processes (see Chapter 5) the following equation arises I

2

ZYn n

1-

By setting Yn = z; -

Zn-I, 2-1

=

I Yn j=O

(1.5.10)

0, equation (i.5.10) reduces to (1.5.11)

which is of second order. Now consider the first order difference equation (1.5.12)

14

1. Preliminaries

Proposition 1.5.1.

Proof.

The solution of (1.5.12) satisfies (1.5.10).

By multiplying (1.5.12) by z; - 1, we obtain

from which it follows that !Z~+l Zn+l -

Zn

=

~z~

-

Zn

Zn+l

+ Zl

+ Zl

= ~(Zn+l - zn)2. Thus we have

~(zn

1 - z;

-

Zn_l)2

1-

•

Zn

Equation (1.5.12) is said to be the first integral of (1.5.12). The solution of (1.5.12) can be written explicitly (see Problem 1.26).

1.6. Comparison Results One of the most efficient methods of obtaining information on the behavior of solutions of difference equations, even when they cannot be solved explicitly, is the comparison principle. In general, the comparison principle is concerned with estimating a function satisfying a difference inequality by the solution of the corresponding difference equation. In this section, we shall present various forms of this principle. Theorem 1.6.1. Let n E N;a' r 2: 0 and g (n, r) be a function nondecreasing with respect to r for any fixed n. Suppose that for n 2: no, the inequalities, Yn+l :5

g(n, Yn),

(1.6.1)

U n+ 1 2:

g(n, un),

(1.6.2)

hold. Then (1.6.3)

implies for all n

(1.6.4)

no.

2:

Proof. Suppose that (1.6.4) is not true. Then, because of (1.6.3) there exists a k E N;o such that Yk :5 Uk and Yk+l > Uk+l' It follows, using (1.6.1), (1.6.2) and the monotone character of g, that g(k,

Uk) :5 Uk+l

<

Yk+l :5

g(k, Yk)

which is a contradiction. Hence the proof.

:5

g(k,

Uk),

•

Usually in applications, (1.6.1) or (1.6.2) is an equation and the corresponding result is called the comparison principle.

1.6.

15

Comparison Results

Corollary 1.6.1. we have

Let n

N;o' k; ;;::: 0 and Yn+l

E

n-l

Yn

Yno

:5

I1

s=no

k,

+

n-l

I

5=no

r.

:5

k;»;

+ p.;

Then.for n ;;::: no,

n-l

I1

(1.6.5)

k..

r=s+1

Proof. Because k n ;;::: 0 the hypotheses of Theorem 1.6.1 are verified. Hence Yn :5 u«, where Un is the solution of the linear difference equation (1.6.6) By (1.5.4), we see that the right-hand-side of (1.6.5) is the solution of (1.6.6). • Theorem 1.6.2. Let g(n, s, y) bedejined onN;o x N;o x Rand nondecreasing with respect to y. Suppose that for n E N;o'

n-l Yn:5

I

g(n, s, Ys) + Pn·

s~o

Then, Yno :5 Pno implies Yn equation

u.; n

:5

n-l u« =

I

s=o

no, where u; is the solution ofthe difference

2':

g(n,

s,

U.) + Pn, Uno

=

Pno'

Proof. If the claim is not true, then there exists a k Yk+l > Uk+l and Ys :5 Us for s :5 k. But, k

Yk+l - Uk+!:5

I

s=o

which is a contradiction.

[g(k •

Yn+l

:5

Yno +

Let n

n

I [ksys + Ps]' s=no

Then, n-l

:5

Yno

I1

s=no

N;o such that

+ 1, s, Ys) - g(k + 1, s, us)] :5 0

Corollary 1.6.2. (Discrete Gronwall inequality).

Yn

E

(l

+ k.) +

n-l

I

s=no

n-l

p,

I1

7'"=s+1

(l

+ kr )

E

N;o' k; ;;::: 0 and

16

Proof.

1. Preliminaries

The comparison equation is n-I

Un

= U"o + l:

5=no

= Y"o·

[ksus + Ps], U"o

This is equivalent to aUn = knun + Pn, the solution of which is n-l

= U"o I1

Un

s=no

(l

+ ks) +

n-l

n-l

l: Ps JT=s+l I1 (l + k s=no

T ) .

The proof is complete by observing that 1 + k, :::; exp(k.). We can prove the following Corollary in a similar way. Corollary 1.6.3.

Let n

E

•

N~o' k; ~ 0 and

Yn+1 :::; Pn+1 +

n

l: ksY., Y"o:::; P"o' s=no

Then, n-l

Yn:::;

n-l

n-l

r; s=no 11 (l + ks ) + s=no l: qs II (1 + k r=s+l

T )

Another form of the foregoing result is as follows. Corollary 1.6.4.

Let n

E

N~o' k;

> 0 and n

Yn+1 s Pn+1

+ l: k»; Y"o:::; 5=no

PlIo'

Then, Yn :::; r;

Proof.

+

n-]

l:

s=no

n-l

Psks

l:

T=s+l

(1 + k

T )

By setting n-I

Vn

= l:

5=no

k.y.,

V"o =

O.

(1.6.7)

17

1.6. Comparison Results

we have (1.6.8)

a to both sides of (1.6.7), a vn = knYn ::s; knPn + k nVn

Applying the operator

we get

from which by Corollary 1.6.1, it follows that n-I

Vn::s;

L

s=no

n-I

k.p,

I1

T=s+l

(1 + k.).

In view of (1.6.8), we obtain the desired estimate.

•

Corollary 1.6.5. Let k(n, s, x): N~o x N~o ~ R+ ~ R+ be monotonic nondecreasing in x and g(n, u):N~o x R+ ~ R+ be monotonic nondecreasing in u. Suppose that (1.6.9) Then

where Un is the solution of Un

=

n-I

L

k(n, s, g(s, us)),

s=no

Proof. The relation (1.6.9) can be written as Yn ::s; g(n, rn), where rn = I::~o k(n, s, Ys)' But this yields rn::s; I::~o k(n, s, g(s, rs))' Now applying Theorem 1.6.2, we obtain r« ::s; Un, which in tum shows Yn ::s; g(n, rn) ::s; g(n, un)'

•

Theorem 1.6.3. Suppose that g 1(n, u) and gi n, u) are two functions defined on N~o x R+ and nondecreasing with respect to u. Let gin, un) es Un+1 ::s; gl(n, un)' Then,

where Pn and rn are the solutions of the difference equations:

18

1. Preliminaries

Proof. Applying estimate. •

Theorem

1.6.1

twice,

we

obtain

the

needed

Theorem 1.6.4. (Discrete Bihari inequality). Suppose that h; is a nonnegative function defined on N~o' M > 0 and W is a positive strictly increasing function defined on R+. If for n ~ no, Yn S Vn where Vn = Yo + M L;:~o h, W(Ys), then, for n E N 1 , Yn S G-

1

(

G(yo)

+M

~nt:o hs)'

where G is the solution of

AG(V)=~ n W(V ) n

and N 1

= {n E N;ol M L;:~o h, S

Proof.

We have A V(n) = Mh nW(Yn)

G( Vn + 1 )

G(oo) - G(xo)}.

S

u», W( Vn). It follows that

S

G( Vn )

+ Mh n ,

from which, in view of Theorem 1.6.1, we get G( V n )

S

G(yo)

+M

n-I

L

s=no

hs '

Theorem 1.6.5. Let an, b; be two nonnegative functions defined on N~o and P; be a positive non decreasing function defined on N no . Let Yn S

r; + snt:o «s, + snt:o asC~:o bkYk).

Then,

Proof.

Since P, is positive and nondecreasing, we obtain

nIl

nIl

nf

Yn S 1 + as Ys + as( b; Yk). P; s~no P, s=no k=no Pk

We let Un

= 1+

L

n-I

s=no

Ys n-I ( n-l Yk) as- + as bk, P, s=no k=no Pk

L

L

Uno = 1,

19

1.6. Comparison Results

so that we have

This in turn yields llun ::;; an[ Un

+ knt~o bkUk

J.

By setting Xn = Un + I::~o bkuk, we obtain llXn = llu n + b.u; ::;; a.x; + b.u; ::;; (an + bn)x n, from which one gets Xn+ 1 ::;; (1 + an + bn)xn, and therefore, n-I

x n::;;

TI

(l

k=no

+ ak + bk).

Consequently, we arrive at n-I

TI

llun ::;; an

(l

k=no

+ ak + bk),

which implies Un ::;; 1 +

n-]

I

s=no

This in turn yields Yn ::;; Pnun. Theorem 1.6.6.

n-1

as

I

k=no

(1 + ak + bk).

•

Let Yj(j = 0, 1, ...) be a positive sequence satisfying Yn+I ::;; g(Yn,

~~~ n.

~~: Yj),

where g(y, z, w) is a nondecreasing function with respect to its arguments. If Yo::;; Uo and

then, for all n z::

°

Proof. The proof is by induction. The claim is true for n is true for n = k. Then, we have YHI ::;;

g(yk,

If we take Uk

:~~ Yj,

= tH

I -

:~: Yj)

::;;

g(

Uk,

~~ Uj,

=

O. Suppose it

:~: Uj) = Uk+I'

•

tk = llt b to = 0, the comparison equation becomes lltk = g(Mk, tk, tk- I).

20

1. Preliminaries

Theorem 1.6.7. inequality

Let Yi (i

= 0,1, .. '.)

be a positive sequence satisfying the

Yn+l =s g(Yn, Yn-l' ..• ,Yn-d, where g is nondecreasing with respect to its arguments. Then,

where Un is the solution of j = 0, I, ... , k:

Proof. Suppose that the conclusion is not true. Then there exists an index m =:: k such that Ym+l > Um+l and Yj =s uj, j =s m. It follows that g(Ym, Ym-l' ... ,Ym-k) =:: Ym+l > Um+ 1

which is a contradiction.

= g( Um' Um- I, ... , Um-k),

•

The next theorem deals with the higher difference of the unknown function. Theorem 1.6.8. Let Yi, Pi, qi be nonnegative sequences defined on N; and hij be a nonnegative sequence defined on N+ x N+. Suppose that, for k =:: 0, Ji kUn =s Pn + qn

k

n-l

I L

hjsJijus'

(1.6.10)

j~Os=O

Then, Jikun=spn+qn where

n-l

n-l

s=O

7=s+1

L
[I+l/IT]'

kf ±(Jijuo)h in( ]. -n .) k-I (n - s. - I)PH +Lh Is=O ] k-l n1-,-1 (n - s - I) qnhkn + L hk-j-I,n L . qs·


+

j=O i~O

I

n-j-I

j=O

l/In = Proof.

k - j - I •n

j=O

s=O

]

From the discrete Taylor formula (1.3.5), we have, for 0 -s j . k-I ( JiJun = I i=j

n).

n-k+j

. ] . Ji'Uo+ s~O I

I -

(nk-'s - I) a kus' -

:I - 1

:5

k - I,

(1.6.11)

21

1.6. Comparison Results

Define R; = L;=o L~:~ hjs6/u" and R o = O. The inequality (1.6.10) can be written as (1.6.12) From the definition of R n , one gets D..Rn =

k

L

j=O

hjnD..jU n = hknD..kU n +

k-I

L

j=O

ju hjn6 n,

and by using (1.6.11) and (1.6.12), it follows that

By exchanging the indices i and j, reversing the summations in the second term and changing indices in the third expression, we obtain

::; hknPn

.J, (

+ j~O ;2:0 k-I

+ [ hknqn +

n).

k-I

n1-,-1

j _ i (6]uo)h in + j~O hk-j-I,n s2:

n1-,-I(n-s-1) ] .L hk-j-I,n L . qs R n, ]=0 s=o J

o

(n - js- 1) Ps

k-l

where the nondecreasing property of R, has been used. By the definitions of the functions 4>n and o/n, we get D..Rn ::; 4>n

+ o/nRn'

The proof is complete by using Corollary 1.6.1, which gives

s;«

n

n-l

n-l

s=O

'7"=5+1

L 4>s

(l

+ o/T)'

and from (1.6.12), it follows that D..kU n ::; v:

+ q.;

n-l

n-l

5=0

l'"=s+1

L 4>s Il o + o/T)'

•

22

1. Preliminaries

1.7. Problems 1.1.

Let Yl = -2, Yz = -2, Y3 = 0, Y4 = 4. Compute Y5 supposing that = O.

~4YI

1.2.

Show that

~ -I

1.3.

ei(ax+b)

=

ei(ax+b) ia

e - 1

+ w(x).

Using the result of problem 1.2, show that

~ cos(ax + b) = -2 sin ~sin( ax + b +~), ~ sin(ax + b) = 2 sin ~cos( ax + b + ~).

tn2(n + If.

1.4.

Show that L.;~I/ =

1.5.

Prove that L~1 cos ql

1.6.

The factorial powers can be defined for values of x that are not integers. Letting m = 1 and n = x-I, in the relation x(m+m) = x(m)(x - m)(nl, we get x(x) = x(x -1)(X-l). Setting y(x) = X(X), the previous expression can be written as y(x) = xy(x -1). The solution of this equation is y(x) = I'(x + 1), where f(x) is the Euler function, which is defined for all real values of x except the negative integers. This allows us to define x(x) for x = O. In fact, we have 0(0) = I'(L) = 1.

n

From

x(-n)

=

•

=

sin(n

+ !)q -

2 . 1 sm"2q

sin !q

.

1 we also have (x+1)(x+2) ... (x+n)

o(-n)

=~. n!

f(x + 1) ( ) fx+l-n

1.7.

Usmg the previous result, show that x' ) =

1.8.

Show that ~ log I'(x) = log x and ~ -I log x = log I'(x) + w(x).

1.9.

Let ljJ(x)

• •

f'(x)

= f(x) ,

n

where f'(x)

1

. , . IS the derivative of I'(x), Show that

The function ljJ(x) has many interesting properties. x Among them we recall lim (ljJ(x) -log(x» = o. ~ljJ(x) = -.

.x-e cc

23

1.7. Problems

1.10. Show that, if p(x) is a degree k polynomial, it can be written in the (i)

form p(x)

=

L~=o ~llip(O), which is the Newton form. /.

1.11. Show that for

q, n

N~, L;~; (-l)j( n rr] q.)

E

the Sterling formula ( 1.12. Show that for q, I, n

=

(qn-1 -1). (Hint: Use

q.) = (qn -rr]~) + ( n+]-l q~ 1 ).

nvi

E N~,

one has

'-l( n+j q )(J)1 = (q n-1 - 1- 1) .

q-n

S(q,l,n)==Lj=1 (-I)'

Note that S(q, 0, n)

= (q -

1) as established in the previous exercise. n -1

1.13. Verify that, if {Yn} and {zn} are two sequences, one has (a) Il(Ynzn) = Yn+lllzn + znllYn

the last formula is called summation by parts. By setting Ilzn = b; we have, ( d ) "N-I L.n~O Yn bn formula.

=

YN "N L./=o bi

- "N-I("n L.n=o L.i~O

b)A h eAb i .:.J.Yn, known as t eI

1.14. Let f(x) E COO[a, b]. Putting EJ(x) = f(x + h), with X,X+hE [a, b], prove that E; = e''" = 1+ Il h , where Ilh!(x) = f(x

+ h)

df(x) - f(x) and Df(x) =~.

1.15. Expand in power series the function f(x) = f(x)

= L~~o B~xn, n.

+-

e - 1

and verify that

where B n are the Bernoulli numbers.

24

1.16.

1. Preliminaries

x X X coshxj2 Prove that - x - - + - = . h j and using the result of problem e -1 2 2 sm X 2 1.13, show that B 2 k + 1 = 0, k = 1,2, ....

B 1.17. Using the result of problem 1.12, show that d- I = L~~o ~Dn-I.

n.

1.18. (Euler-McLaurin formula).

From the previous results, show that

D-1f(x) = ff(X) dx = d -If(x) -

f

B n D n - 1f(x). n=1 n!

This formula can be used either to approximate an integral or to approximate a sum using an integral. 1.19.

Using the result of problem 1.14 prove that D- 1 = d - I + 1/- f2d + ~d2 - ... , from which one obtains the Newton integration formula f f(x) dx

= d -If(x) + 1f(x) -

f2df(x)

+ ~d2f(x) ....

1.20. Show that 1[ 1 2 1 3 ] 1 ~ idi D=- d--d +-d ... =- '- (-0--:-. h 2 3 h ;=1 I

This formula can be used to approximate numerically the derivative of a function. 1.21. (Horner rule).

Find the solution of the difference equation Yn

= ZYn-1 + bn, + Yn

Yo = boo

+ 1.

1.22.

Find the solution of Yn+1

1.23.

Solve the Clairaut equation Y n = 0 there are two solutions.

1.24.

Solve the following nonlinear equation

1.25.

In many applications (for example in computing the cost of Fast Fourier transform algorithm) one meets the following equation C(n) =

2C(~) + fen),

C(2)

= n =

ndy - ¢(dy) and verify that for

= 2.

Yn+l(l

+ Yn) =

Yn'

Transform this equation into a

linear one and find the solution. 1.26. The Newton method to compute the square root of a positive number

Areduces to the difference equation

Zn+1

= !(Zn +

A). Transform

Zn 2 this equation to a linear one and find the solution.

25

1.8. Notes

1.27. Solve the following nonlinear equations (A) -Yn+1

= Y~,

(B) Yn+1

= 2Yn(l

Zo -

- Yn),

(C) Yn+1 = 1

1.28. Find the first integral of Yn+2

= Yn

!y~

- Yn

(l + YnYn+1) 1+ 2 Yn+1

1.29. Suppose that c ~ 0; k; ~ 0, k; < e for n ~ no and Yn ~ c + I;=no ksYs' Show that Yn

~

(1

c n-I k, ) . --exp --Is=n 1-8 l-e 0

1.30. Let be h > 0, M > 0, 5> 0 and 0< Uo:S; 5. Find a bound for

Un

~ h

1/2

n-I M Ij=o (

1 .)1 Uj n - ] /2

+ 8,

Un'

1.31. Suppose that c ~ 0, k n ~ 0 for n ~ no and Yn that Yn ~ c exp(I:=n+1 k.). 1.32. Study the following inequality Y~+I and a < 1.

:s;

ay~

:s;

c + I:=n+1 ksYs' Show

+ bn(Yn + Yn+l) for

a

> 1

1.8. Notes Most of the contents of Sections 1.1 to 1.4 may be found in many classical books on difference equations, such as Jordan [83], Milne and Thomson [116], Fort [49], Brand [16] and Miller [114]. Propositions 1.3.4 and 1.3.5 have been adapted from Agarwal [5]. Condition 1.4.8 can be found in Schoenberg [153]. Theorem 1.6.1 and many other comparison results on difference inequalities are taken from Sugiyama [163]. Theorem 1.6.2 was established by Maslovskaya [13]. Theorems 1.6.3, 1.6.4 and 1.6.5 are due to Pachpatte [133, 134, 135]. Theorem 1.6.6 is adapted from a result of Rheinboldt [147]. Theorem 1.6.7 is essentially new. Theorem 1.6.8 is due to Agarwal and Thandapani [1]. A generalization of this theorem can be found in Agarwal and Thandapani [1,2]. For allied results on difference inequalities, see Popenda [142, 143], Mate' and Nevai [101] and Patula [139].

CHAPTER 2

2.0.

Linear Difference Equations

Introduction

This chapter investigates the essential techniques employed in the treatment of linear difference equations. We begin Section 2.1 with the fundamental theory of linear difference equations and develop the method of variation of constants in Section 2.2. We then specialize our discussions to linear difference equations with constant coefficients, since this is an important class in itself. We exhibit, in Sections 2.3 to 2.5, methods of obtaining solutions of such equations by the use of difference operators as well as generating functions that offer elegant methods of solving difference equations. In Section 2.6, we present theory of stability for difference equations with constant coefficients and discuss in Section 2.7 linear multistep methods as an application of the theory of absolute stability. Section 2.8 considers the scalar boundary value problem that is needed later on. Section 2.9 completes the treatment by filling in the gaps with interesting problems.

2.1.

Fundamental Theory

Since it is not essential to specify the properties of functions with respect to x E C, we shall use in this section N+ or as definition set. We shall denote a sequence by {Yn}, which is the set of all values of the function Y on N~o'

»;

27

28

Definition 2.1.1. Let po(n) = 1, Pl(n), ... ,Pk(n), gn be k defined on N;o' An equation of the form Yn+k

+ Pl(n)Yn+k-l + ... + Pk(n)Yn = gn

2.

Introduction

+2

functions (2.1.1)

is called a linear difference equation of order k, provided that Pk(n)

~

o.

With (2.1.1), the following k initial conditions

Yno+l

=

C2,"" Yno+k-l

=

(2.1.2)

Ck

are associated, where c, are real or complex constants. For convenience, we shall state the following theorem, which is a special case of Theorem 1.5.1. Theorem 2.1.1. The equation (2.1.1) with the initial conditions (2.1.2) has a unique solution. • We shall denote by y(n, no, c) the solution of (2.1.1) and (2.1.2), where k C = (c\, C2," ., Ck) E C . Thus, we have

= 0, 1, ... , k -

j

Definition 2.1.2. If for every n is said to be homogeneous.

E

N;o' g(n)

1.

= 0, then the equation (2.1.1)

Introducing the operator L defined by

LYn =

k

L Pi(n)Yn+k-i i=O

(2.1.3)

equation (2.1.1) assumes the form

LYn = g(n)

(2.1.4)

and the homogeneous one becomes (2.1.5)

LYn = O. It is easy to verify that L is linear, since

Li ay;

+ {3zn)

=

al.y;

+ f3Lzn ,

when a, f3 E C and {Yn}, {z.} are two sequences. Let S be the space of solutions of (2.1.5). Because of linearity of L, we have the following result. Lemma 2.1.1. Any linear combination of elements of S lies in S. Let y(n, no, E 1 ) , ••• ,y(n, no, E k) be solutions of (2.1.5), where E1

= (1,0, ... ,0),

E2

= (0, 1,0, ... ,0), ... , E k = (0,0, ... ,0,1),

and therefore, Ly(n, no, E i) = 0,

i

= 1,2, ... , k.

(2.1.6)

29

2.1. Fundamental Theory

Lemma 2.1.2. Any other solution y(n, no, c) of (2.1.5) can be expressed as a linear combination ofy(n, no, E;), i = 1,2, ... , k.

Proof. If y(n, no, c) is the solution with initial conditions then the sum

Cl> C2,""

Ck,

k

I

z; =

Cjy(n, no, E;}

(2.1.7)

i=l

is a solution of (2.1.5) by Lemma 2.1.1 and also satisfies the same initial conditions z"" = C\, z",,+\ = C2,' " , Z",,+k-\ = Ck' By Theorem 2.1.1 the two solutions must coincide. • Let us now take k functions};( n) defined on N~o and define the matrix f\(n) ft(n + 1)

f2(n)

fin + 1)

K(n) =

Definition 2.1.3. The function };(n), i dent if for all n, k

I

i=l

=

1,2, ... , k are linearly indepen-

O!j};(n) = 0

(2.1.8)

implies a, = 0, i = 1,2, ... , k. Theorem 2.1.2. A sufficient conditionfor the functions fiin), i = 1,2, ... , k, to be linearly independent is that there exists an n ~ no such that det K(n) ;1= O.

Proof.

If (2.1.8) holds, then k

I

O!j};(n) = 0,

j=\

k

I

i=1

k

I

i=l

O!d;(n + 1) = 0,

(2.1.9) O!j};(n + k - 1) = O.

30

2.

Introduction

This linear homogeneous system of k equations in k unknowns has the coefficient matrix K (n). Thus, iffor n = ii, det K (n) ;c 0, the unique solution of the system is a, = 0, i =; 1, .. , k, and the functions are linearly independent. • Theorem 2.1.3. If j;(n), i = 1,2, ... , k; are solutions of (2.1.5), then det K (n) ;c 0, for n 2:: no, provided that det K (no) ;c O. Proof.

From the definition of the matrix k(n), we get

1)

det Ktn., + 1) = det

(

1) ...

f l( no + fl(nO + 2)

f2(nO + f2(n O+ 2)

h(no + k)

f2(n O+ k)

...

fl(n O+

. ..

1))

fk(nO + A(no + 2)

...

A(no + k)

1)

, A(no + 1)

fl(n O+ 2)

A(n o + 1)

fk(n O+ k-1) fk(nO)

fl(n O+ k-1) fl(nO) = (-1)kpk(no) det K(no).

Since Pk(nO) ;c 0, it follows that det K(no + 1) ;c O. In general, for every n E N~o., we have det K(n)

= (-1)kpk(n) det

K(n -1),

and thus by induction it is now easy to see that det K(n) ;c O. Corollary 2.1.1. independent. Proof.

The solutions y(n, no, EJ, i

In this case, det K (no)

=

(2.1.10)

•

1,2, ... , k, are linearly

= 1, since

K(no) = ( ; :

o

~... ~)..

". ". 0 ... "·0".1

In view of Corollary 2.1.1, the base y(n, no, EJ, i = 1,2, ... , k, is said to be a canonical base. Using Lemma 2.1.2 and Corollary 2.1.1, we have the following result. Theorem 2.1.4. dimension k.

The space S of solutions of (2.1.5) is a vector space of

2.1.

31

Fundamental Theory

If a, E c', i = 1,2, ... , k, are linearly independent, the set of solutions y(n, no, aJ can also be used as a base of the space S. In fact, in this case K(n o) = (a], ... , ak) where a, is the ith column of the matrix K(no) and because they are linearly independent, it follows that det K (n) ,e. 0 for all n E N;o' The matrix K (n) is called the matrix of Casorati and it plays in the theory of difference equations the same role as the Wronskian matrix in the theory of linear differential equations. Definition 2.1.4. Given k linearly independent solutions of (2.1.5), any linear combination of them is said to be a general solution of (2.1.5). The term, general, means that such a solution can satisfy any set of initial conditions.

Lemma 2.1.3. The difference between two solutions Yn and Yn of (2.1.4) satisfies (2.1.5). Proof.

o.

•

One has LYn

= gn

and LYn

= gn,

which implies L(Yn - Yn) =

Theorem 2.1.5. Let y(n, no, aJ, i = 1,2, ... ,k, be k linearly independent solutions of (2.1.5) and Yn be a solution of (2.1.4). Then, any other solution of (2.1.4) can be written as Yn = Yn +

k

I

a.yt n, no, aJ.

(2.1.11)

i=1

Proof. From Lemma 2.1.3, one has that Yn - Yn E S and therefore, it can • be expressed as a linear combination of y(n, no, a i ) .

The foregoing theorem also means that the general solution of (2.1.4) is obtained by adding to the general solution of (2.1.5) any solution of (2.1.4). Associated with the operator L, one defines the adjoint operator L * by L*Yn = I~=oPj(n + i)Yn+i and the adjoint equations L*Yn = 0,

(2.1.12)

and (2.1.13) There are some interesting properties connecting the solutions of the equations and its adjoint. We shall consider, however, because of its use

32

2. Introduction

in Numerical Analysis, (see section 5.4) the transpose operator L T defined by

LTy" =

k

I Pi(n - k + i)y"+i i=O

(2.1.14)

and the transpose equation (2.1.15) Theorem 2.1.8. Let u; be the solution of the homogeneous equation (2.1.5) and y" the solution of (2.1.15) with YN+j = 0 for j = 1,2, ... , k and N> k: Then N

j

k-j

I u"g" = j=O I Yj i=O I pJj ,,=0 Proof.

We have N N N I u"g" = I u"L Ty" = I ,,=0

Setting j

k)Uj_i

k

I U"Pi(n - k + i)y,,+i' ,,=0 i=O

,,~o

= n + i and s = min(j, k), one has N

N+k

I u.g; = I n=O

j~O

s

Yj

I

Pi(j - k)Uj_io

Pi"'" 0

for

i

> k:

i~O

Since Yj = 0 for j > N and 'I;~OPi(j - k)uj_ i = 0 for s;::: k, the conclusion follows immediately. •

2.2.

The Method of Variation of Constants

It is possible to find a particular solution of (2.1.4) knowing the general solution of (2.1.5). This may be accomplished by the method of variation of constants. Let y(n, no, c) be a solution of (2.1.5) and y(n, no, E;), i = 1,2, ... , k, be the canonical base in the space S of solutions of (2.1.5). Then, k

y(n, no, c)

=

I

cjy(n, no, Ej).

(2.2.1)

j=l

We shall consider now the cj as functions of n with

cj(no) = cj

(2.2.2)

and require that the functions k

y(n, no, c(n»

=

I

j=l

cin)y(n, no, Ej)

(2.2.3)

33

2.2. The Method of Variation of Constants

satisfies the equation (2.1.4). From (2.2.3) it now follows that yen

+ 1, no, c(n + 1))

k

I

=

cj(n

j=!

+ 1)y(n + 1, no, Ej)

k

I

=

c/n)y(n

j=!

+ 1, no, Ej)

k

I

+

j=!

+ 1, no, Ej ).

Ilcj(n)y(n

By setting k

I

j=!

+ 1, no, Ej ) = 0,

Ilcj(n)y(n

(2.2.4)

we have

+ 1, no, c(n + 1))

y(n Similarly, for i

k

=

I

j=!

+ 1, no, Ej).

c/n)y(n

1,2, ... , k - 1, we can get

=

y(n

+ i, no, c(n + i))

k

I

=

cj(n)y(n

+ i, no, Ej),

(2.2.5)

j~!

if we set k

I

Ilcj(n)y(n

+ i, no, E)

i = 1,2, ... , k - 1.

= 0,

(2.2.6)

j~!

Therefore, in the end, we obtain y(n

+ k, no, c(n + k))

k

=

I

c/n)y(n

+ k, no, EJ

j~!

+

k

I

j=!

Ilcj(n)y(n

+ k, no, Ej).

By substituting in (2.14), there results k

I

pj(n)y(n

+k-

i, no, c(n

+k -

i))

j~O

k

=

I

j~O

k

Pi(n)

I cj(n)y(n + k j=!

i, no, Ej)

+

k

I

Ilc/n)y(n

+ k, no, Ej)

j~!

= gn·

Since y(n, no, Ej) are solutions of (2.1.5), one has k

I

j=!

Ilcj(n)y(n

+ k, no, Ej)

=

gn·

(2.2.7)

34

2. Introduction

The equations (2.2.6) and (2.2.7) form a linear system of k equations in k unknowns cj(n), whose coefficient matrix is the Casorati matrix K(n + 1). The solution is given by

IlCI(n))

Ilc~(n) = K-1(n + 1)

( Ilck(n) Denoting by Mik(n (2.2.8) becomes

(2.2.8)

gn

+ 1) the (i, k) element of the adjoint matrix of K (n + 1),

Ilc.(n) I

(0)f .

Mik(n + 1) = detK(n+1) g n,

i

= 1,2, ... , k,

(2.2.9)

from which it follows that cj(n) = 11

-I

Mik(n + 1) det K(n + 1) gn + Wj,

cj(nO)

= c;

By substituting the values of ci(n) in (2.2.3), we see that z(n, no, c) y(n, no, c(n)) satisfies (2.1.4).

2.3.

=

Linear Equations with Constant Coefficients

If in equation (2.1.1) the coefficients Pi(n) are constants with respect to n, we obtain an important class of difference equations k

I PiYn+k-i = j=O

s-

Po

=

1.

(2.3.1)

The corresponding homogeneous equation is k

I

i=O

Theorem 2.3.1.

PiYn+k-i

= O.

The equation (2.3.2) has the solution of the form Yn = z",

(2.3.2)

(2.3.3)

C, and satisfies (2.3.4).

where z

E

Proof.

Substituting (2.3.3) in (2.3.2), we have

n"L. Pi k

Z

Z k-j

i=O

= 0

from which it follows that k

I

i=O

PiZ k -

i

= O.

(2.3.4)

2.3.

35

Linear Equations with Constant Coefficients

Equation (2.3.4) is a polynomial and it has k solutions in the complex field. Furthermore, it is said to be the characteristic equation of (2.3.2), and the polynomial p(z) = I~~o PiZk-i is called the characteristic polynomial. • If the roots z}, Z2,"" Zk of p(z) are distinct, then z;, ... , z~ are linearly independent solutions of (2.3.2).

Theorem 2.3.2. z~,

Proof. It is easy to verify that in this case the Casorati determinant is proportional to the determinant of the matrix

1

1

ZI

Zk

Z2 2

zi

Z2

k-I ZI

k-I Z2

V(ZI> Z2,"" Zk) =

1

zi

(2.3.5)

k-I

Zk

which is known as Cauchy-Vandermonde matrix (or the Vandermonde matrix). Its determinant is given by det V(z}, Z2, ... , zd

=

n (z; - Zj),

(2.3.6)

i>j

which is different from zero if z, "r:. Zj for all i and j. From Theorem 2.1.4 it follows that if the roots of p(z) are distinct, any solution of (2.3.2) can be expressed in the form k

Yn =

I

;=1

(2.3.7)

ciz7·

When p(z) has multiple roots, the solutions z7 corresponding to distinct roots are linearly independent. But they are not enough to form a base in • S. It is possible however, to find other solutions and to form a base. Theorem 2.3.3.

Let m, be the multiplicity of the root z, of p(z). Then the

functions

(2.3.8)

ys(n) = u,(n)z:,

where Us (n) are generic polynomials in n whose degree does not exceed m s - 1, are solutions of (2.3.2) and they are linearly independent.

Proof.

If z, has multiplicity m s as a root of p(z), we have p(zs) = 0, p'(zs) = 0, ... , and p(m,-I)(zs)

= 0.

(2.3.9)

Let us look for a solution of (2.3.2) of the form Yn

= u,(n)z:.

(2.3.10)

36

2. Introduction

By substituting, one gets k

L PiUs(n + k -

i)z:-i

= O.

(2.3.11)

i~O

By the relation (1.1.4), we get

us(n

+

k- i) kf (k ~ i),~Jus(n) ]

j~O

and, from (2.3.11), we have

it

i

PiZ : -

%~

r:

(2.3.12)

=

i)tijus(n) =

jt

tijus(n)

~~ (k; i)Piz:-i (j)(

k

)

= L tijus(n)z1~.

(2.3.13)

]!

j=O

In view of (2.3.9), one has it that the terms of (2.3.13) corresponding to j = 0, 1, ... , m, -1 are zero for all functions u(n). To make the other k - m, + 1 terms equal to zero, it is necessary that tiju s(n) = 0 for m, :5, j :5, k. This can be accomplished by taking u.(n) as a polynomial of degree not greater than m s - 1. The proof that they are linearly independent is left as an exercise. •

The general solution of (2.3.2) is given by

Corollary 2.3.1. d

Yn where Aij

=L

i=1

= ajcj,

d

a juj(n)z7

=L

i=1

mj - l

aj

L

cjn jz7

j~O

d

mi-l

=L L

i=1 j=O

Aijn

iz7,

and d is the number of distinct roots.

The next theorem is useful in recognizing if a sequence Yn, n solution of a difference equation. Theorem 2.3.4. all n E N;o'

(2.3.14)

A sequence Yn, n

E

E

N;o is the

N;o satisfies the equation 2.3.2 iff, for

Yn D(Yn, Yn+1, ... , Yn+k) = det Yn+1 ( Yn+k

Yn+1 Yn+2 Yn+k+1

... ...

Yn+k) Yn+k+1 Yn+2k

=

0

and moreover, D(Yn, Yn+l, ... , Yn+k-I) '" O. Proof. Suppose that Yn satisfies 2.3.2. One has Yn+k+j = -I:;=1 PiYn+k+j-i' By substituting in the last row of D(Yn, Yn+1, ... , Yn+k), one easily obtains

2.3.

37

Linear Equations with Constant Coefficients

that D(Yn, ... , Yn+k) = O. Conversely, if this determinant is zero, one has, by developing with respect to the first row I~~o Yn+k-Ai(n) = 0, where Ai(n) are the cofactors of the k-i'" elements. If in D one substitutes to the first row, the second one, and then the other rows, one obtains determinants identically zero. By developing these determinants again with respect to the elements of the first row one obtains: k

I Yn+j+k-Ai(n) ;=0

0

=

j = 1,2, ... , k - 1.

The determinant Ak(n) is not zero by hypothesis. One has, setting _ Ai(n) Pi( n ) - Ak(n) k

Yn = -

I Pi(n)Yn+k-i, ;=0 k

Yn+1 = -

I Pi(n)Yn+J+k-b ;=0

Yn+k-I = -

I Pi(n)Yn+2k-I-i. ;=0

n

By setting n + 1 instead of n in the first relation and subtracting the second one, we have k

0= -

I ilpi(n)Yn+k+l-i ;=0

and proceeding similarly for the others, one arrives at an homogeneous system of equations k

I

i=1

ilpi(n)Yn+k+j-i = 0

j = 1,2, ... , k - 1,

whose determinant is not zero by hypothesis. It follows that the solution is ilpi(n) = 0, that is the Pi are constant with respect to n and then the conclusion follows. • Example 1.

Consider the equation Yn+1 - aYn = 0,

a E Co

(2.3.15)

The characteristic polynomial is p(z) = z - a and its unique root is z The general solution of (2.3.15) is then Yn = can.

=

a.

38

2. Introduction

Example 2.

Consider the equation

= O.

Yn+Z - Yn+1 - Yn

The characteristic polynomial is p(z) ZI -

I+VS --v-; Zz

=

=

(2.3.16)

ZZ -

1, which has roots

Z -

I-VS

--2-'

Therefore, the general solution of (2.3.16) is

_ (1 + vs)n + (1 -VS)n

Yn -

Cz

2

CI

2

'

and is known as the Fibonacci sequence. Example 3.

Consider the equation

From Example 1, it follows that the general solution of the homogeneous equation is ca ", Applying the method of variation of constants it follows that (c(n + 1) - c(n))a n+1 = gn and C(

n)

=

A-I

u.

g(n) = n~1 g(j) a

n+1

L.

j=O

a

j+\'

from which, we obtain Yn = Yoan +

n-I

I

g(j)a n- j - I.

j=O

Example 4. The following equation often arises in discretization of second order differential equations, namely,

Yn+Z - 2QYn+1

where q

E

+ Yn = In,

(2.3.17)

C. The homogeneous equation has the general solution Yn =

+ czz~, where

Z\ and Zz are the distinct roots of the second degree equation ZZ - 2qz + 1 = O. It is useful, in the applications, to write the general solution in two different forms. In the first form, the linearly independent solutions

CIZ~

( I)

Yn

n

=

n

ZZZI - ZIZZ ; Zz - Z\

(Z)

Yn

z~ - z~

= Zz -

ZI

are used, which give, as a general solution of the homogeneous equation Yn = CIy~1)

+ czY~Z).

(2.3.18)

2.4.

39

Use of Operators 4. and E

In the second case, one uses the Chebyshev polynomials (see Appendix C) Tn(q) and Un(q) as linearly independent solutions, obtaining (2.3.19) The advantage of using (2.3.18) is that the base y~l), Y~Z) is a canonical one, that is,

y&l) = 1,

Y&Z) = 0,

y~l) = 0,

Y~Z)

= 1,

from which it follows that for the initial value problem (2.3.18) we have

Yn = Yoy~l)

+ YIY~Z),

The advantage of the form (2.3.19) lies in the fact that the functions Tn(q) and U; (q) have many interesting properties that make their use especially helpful in Numerical Analysis and Approximation Theory. The solution of (2.3.17) can then be written in the following form, (I)

Yn = YOYn Example 5.

+ YIYn(Z) +

n-Z '\'

L.

j=O

I"

(Z)

Yn-j-I . J]>

Consider the equation

POYn+Z

+ PIYn+1 + PZYn

=

O.

(2.3.20)

The solution can be written in terms of the roots of the polynomial Pozz + PI Z + P: = 0 as usual. It is interesting, however, to give the solution in terms of Chebyshev and q = (PI) I/Z' Po 2 POP2 One easily verifies that pnTn(q) and pnUn(q) are solutions of (2.3.20). It follows then that Yn = clpnTn(q) + CZpnUn_l(q) is the general solution. polynomials. Suppose POP2 > 0 and let p

.2.4.

= (Pz) I/Z

Use of Operators L1 and E

The method of solving difference equations with constant coefficients becomes simple and elegant when we use the operators ~ and E. Using the operator E equations (2.3.1) and (2.3.2) can be rewritten in the form

p(E)Yn = gn, p(E)Yn

= 0,

(2.4.1) (2.4.2)

40

2. Introduction

where k

=I

p(E)

(2.4.3)

PiEk-i.

;=0

It is immediate to verify that p(E)zn

= znp(z)

and

k

p(E)

= I1

(2.4.4)

(E - z.I),

i=1

where z\> Z2,"" Zk are the zeros of p(z). If there are s distinct roots with multiplicity mj,j = 1,2, ... , s, then p(E) can be written as p(E) = rt., (E - zJ)m, and (2.4.2) becomes s

I1 (E -

zJ)m'Yn = 0,

(2.4.5)

i=1

from which it is seen that the homogeneous equation can be split into s difference equations of order mj • In fact, the commutability of the operators (E - zJ) implies the following result. Proposition 2.4.1.

The solution X n of the equation (E - z/)mjX n = 0

(2.4.6)

is a solution of (2.4.5).

• The problem simplifies further since it is possible to define the inverses of (E - zJ), i = 1, 2, ... , k.

Proposition 2.4.2. Let {Yn} be a sequence and let fez) polynomial of degree m. Then,

= L~~o a.z'

be a

(2.4.7) Proof.

By definition of fez), it follows that f(E)(zn yn)

m

=I

i=O

= z"

aiEi(zn yn) m

I

;=0

m

=I

ai(ziEi)Yn

aiZn+iEiYn

i=O

= znf(zE)Yn-

•

Definition 2.4.1. The inverse of the operator (E - zI) is the operator (E - zI)-1 such that (E - zI)(E - zI)-1 = 1.

2.4. Use of Operators

Theorem 2.4.1.

~

41

and E

Let z

E

Co Then, the inverse of E - zI is given by

(E -

av> = zn- 1a-

1z-

n.

(2.4.8)

Applying (E - zI) to both sides of (2.4.8) and using the result of Proposition 2.4.2, one gets

Proof.

(E - zI)(E - zI)-l

= (E - zI)zn-1a-1z- n = zn-1 z (E - I)a-1zn

Corollary 2.4.1.

For m

= 1,2, ... ,

one has

(2.4.9)

The equation (2.4.9) allows us to find very easily the solutions of (2.4.6) and then of (2.4.1). In fact, from (2.4.6) and (2.4.9) we have But, we know that a-mj. 0 = qj(n), where qj(n) is a polynomial of degree less than mj' Hence, the solution x, is given by x, = z'j-mjqj(n), and this can be repeated for j = 1,2, ... , s. Usually, because mj is independent of n, one prefers to consider the previous solution multiplied by z'j'j. The solutions corresponding to the multiple root Zj are then z'jqj(n), and hence, the general solution of (2.4.1) is Yn = "2.;=1 ajqj(n)z'j, which is equivalent to (2.3.14). In general, to get a solution of the nonhomogeneous equation (2.4.1), one can proceed as described in the previous section by applying the method of variation of constants. Usually this way of proceeding is too long and in some cases can be avoided using the definition of p-1(E). Proposition 2.4.3. f(z) ¥- O. Then

Let f( z) be a polynomial of degree k and z E C with

f Proof.

-1

(E)z

n

z"

= f(z)'

(2.4.10)

By applying f(E) to both sides of (2.4.10), one obtains f(E)f-1(E)zn = f(E) f~;) = zn.

•

42

2.

Introduction

Proposition 2.4.4. Let f(z) be a polynomial of degree k and ZI E C be a root of multiplicity m. Then, setting g(z) = (z - zl)-mf(z), one has n-m (m) f-I(E)z~ = Zl n (2.4.11) g(zl)m!

Proof.

By applying f(E) to both sides and using (2.4.7), one obtains

f( E )f ( E

-I) n

ZI =

z~-mf(ZIE)n(m)

g(z\)m! z~g(zIE)' 1

g(zl)

n"'g_..:...(z...:...IE----'--')(_E_-_I'--)m_n_(m_) g(z\)m!

= ZI-

n

= Zl'

•

Proposition 2.4.5. Let f( z) be a polynomial ofdegree k and Yn be a sequence. • Then for every n E N, we havef-I(E)znYn = znf-l(zE)Yn. These results can be used to obtain particular solutions of the equation p(E)Yn = g(n). Let us consider the most frequent cases: (a) g(n)

=

g constant. If p(l)

0 from (2.4.10) one obtains

;i:

y = p-I(E)g = gp-I(E) .1 n = --L = _g_ p(l)

n

~ L..

i=O

(b) g(n) = I:=I aiz7 with p(za _

Yn

=P

-1

(E)

;i:

O. From (2.4.10) one has

L a.z, = i~1 L a.p S

n

S

i=1

Pi

-1

n

(E)Zi

=

z7 L ap(za i-' n

i=\

(c) Same as in (b), but Zj is a root of p(z) of multiplicity m. From (2.4.10) and (2.4.11) we obtain a.z n zn-mn(m) Yn = L _(_I_I + aj J , jyt.J g(zj)m! f(z) where g(z) = ( )m

za

Z -

zJ

(d) g(n) = e in a • This case can be treated as in (b) or (c) by putting z = e ia • (e) g(n) = cos na, g(n) = sin na. We can proceed as in case (d) taking the real or imaginary part.

2.5.

Method of Generating Functions

The method of generating functions is another elegant method for solving linear difference equations with constant coefficients. Its importance is growing in discrete mathematics.

43

2.5. Method of Generating Functions

Definition 2.5.1. Given a sequence {Yn}, we shall call a formal series generated by it the expression 00

y=

L Yi X i,

(2.5.1)

i=O

where x is a symbol. Only in the case where x will be a complex value, the problem of convergence of (2.5.1) will arise. The formal series are often called generating functions of {Yn}' In the set of all formal series, we can define operations that make such a set algebraically similar to the set of rational numbers. Definition 2.5.2.

Given two formal series Y and Z, their sum is defined by Y

+Z =

00

L (Yn + zn)x n.

n~O

Definition 2.5.3.

The product of two formal series Y and Z is given by (2.5.2)

where en

=

n

n

L Yizn-i = L i=O

ZiYn-i'

(2.5.3)

i~O

We list some simple properties of formal series: (i) (ii) (iii) (iv) (v) (vi)

The product of two formal series is commutative; Given three formal series Y, Z, T we have (Y + Z) T = IT + ZT; The unit element with respect to the product is the formal series I = 1 + Ox + Ox2 + ... ; The zero element is the formal series 0 + Ox + Ox2 + ... ; The set of all the formal series is an integral domain; I Let Y = 0 YiXi. If Yo r6 0 then there exists the formal series yI such that y- Y = 1.

2:::

The polynomials are particular formal series with a finite number of terms. Consider now the linear difference equation k

L PiYn+k-i = 0, i=O

with Po = 1,

(2.5.4)

and we shall associate with it the two formal series

= Po + PIX + y = Yo + Ylx + P

+ p~\ and .

(2.5.5)

44

2. Introduction

P is different from the characteristic polynomial. In fact, one has P

= Xkp(~) == p(x),

(2.5.6)

where p(z) is the characteristic polynomial. The product Q of the two series is (2.5.7) where n

I PiYn-i' ;=0

qn =

(2.5.8)

In view of (2.5.4) it is easy to see that 0 = qk = qk+1 = ... , which means that Q is a formal series with a finite number of terms. Moreover, because P is invertible (Po = 1), one has

y

= P-1Q.

(2.5.9)

If we consider the symbol x as an element z of the complex plane, then (2.5.9) gives the values of Y as ratio of two polynomials Y

= q(z) = p(z)

where

q(z)

(2.5.10)

p (; )'

Zk

p(D is the characteristic polynomial and q(z) = I7:~ q.z'. The roots

of p(z) are Z;-I, where z, are the roots of p(z). It is known in the theory of complex variables that every expression like (2.5.10) is equal to the sum of the principal parts of its poles. The poles of (2.5.10) are the roots..!.. of the denominators, and therefore, z, s

00

'" YnZ n Y =_ L...

_

n=O

=

i=lj=1

mi

s

I I

m,

'\' ' " L... L...

-

aij(-1)

aij( z - z,-I)-j

(2.5.11)

jzl(l-

ZiZ)-j,

;=1 j=1

where s is the number of distinct roots of p(z) and mj their multiplicity. The coefficients aij are the coefficients in the Laurent series of (2.5.10). For Iz;zl < 1, i = 1,2, ... , k, (l - z;z)-j can be expressed as (1 - z.z)

-j

=

I

00

(

n=O

n

z; z

n

)j = I

00

n=O

(

n +.] n

1) z z , n j

n

(2.5.12)

45

2.5. Method of Generating Functions

and substituting in (2.5.10, we get

I

I

Ynz n =

n=O

t I aij(-O-i(n +

z"

i=1 i=1

n=O

If we write q;(n)

=

Im,

i=1

j -1)Z?+i. n

(n + j - 1) (-I)1z?, ..

au

n

we obtain Yn =

s

I

i=1

(2.5.13)

qi(n)z?,

which is equivalent to (2.3.14). Theorem 2.5.1. Suppose that the roots of the characteristic polynomial p (z) are inside the unit disk ofthe complex plane. Then the formal series Y converges inside the unit disk. Proof. The polynomial p(z) has zeros outside the unit disk and q(z)/p(z) has no poles in it. Y must then coincide with the Taylor series in the unit disk. •

Theorem 2.5.2. lfthe characteristic polynomial p(z) has no roots outside the unit disk and those on the unit circle are simple, then the coefficients Yn of Y are bounded. Proof. From (2.5.13), it follows that q;(n) corresponding to Iz;1 = 1 are constants with respect to n. The method of generating functions can also be used to obtain solutions of nonhomogeneous equation (2.1.4). One can proceed as before with the difference that qk, qk+h ... , are not zero. In fact from (2.5.8) and (2.1.4) one has qk = I~=OPiYk+i = go and, in general, for n = 1,2, ... k

qn+k

=I

;=0

PiYn+k-;

= gn,

(2.5.14)

since Pi = 0 for i> k. The series (2.5.7) can be written as k-I n co Q = I zn I PiYn-i + Zk I gnzn =

n=O

;=0

QI(Z)

+ ZkQ2(Z),

n=O

(2.5.15)

where as (2.5.9) becomes

I

;=0

YiZi = QI(Z) ~ ZkQ2(Z). p(z)

•

(2.5.16)

46

2.

Introduction

The polynomial QI (z ) depends only on the initial values Yo, YI, ... , Yk-I , while Q2(Z) is a formal series defined by the sequence {gn}. Proceeding as in (2.5.10), (2.5.11) and (2.5.12) one obtains the solution {Yn}. This procedure can be further simplified by considering that inside the region of convergence, it represents a functionf(z), which is said to be the transformed function of the sequence {Yn}. For example, in the unit disk, the function (1- Z)-I is the transformed function of the constant sequence {l}, since (1- Z)-I =

00 I z'.

i~O

On Table 1, transformed functions of some important sequences are given. Now suppose that Qiz) is the transformed function of {gn}. After doing the necessary algebraic operations, one obtains from (2.5.16), o y.z' = G(z), where G(z) is the function resulting in the right-hand side of (2.5.16). By expanding G(z) in Taylor series and equating the coefficients of the powers of the same orders on both sides, we arrive at the solution {Yn}.

I:

Example. Consider the QI(Z) = 1, Q2(Z) =

equation Yn+1 + Yn = -(n + 1), Yo = 1. Here 1

-I:=o (n + l Iz" = ( l-z )2 and

z

G(z)

1---~

=

(1- Z)2 1+ z

1 - 3z + Z2 (1 + z)(1 - Z)2

From Table 2.1, we find that _1_ l+z (1 -

z

,,00 z? =

"'-n=O

n

nz.

f

There ore,

5

1

1

1 1 z = 4 1 + z - 4 1 - z - 2 (1 - z?'

= I:=o (_1)n zn, _1_ = I:~o z", and l-z

,,00 ,,00 [5( ) 1 1 ] "'-n=oYn z = "'-n~O 4 -1 n - 4- 2n z, n

n

. h we obtain . Y = -5()n 1 -1 - -1 - -n f rom whic n 4 4 2' In some applications, especially in system theory, instead of generating functions defined in (2.5.1), generating functions called Z transform defined

by X(z) therefore,

=

I:=o Yn z- n

are used. It is evident that X(z)

=

yC)

and

2.6.

Stability of Solutions

47 TABLE

1.

Domain of Convergence

Yn

f(z)

n (n + m)(m) n(m) nm kn

(1 - Z)-I Z(1-Z)-2 m!(1 - z )-m-l m Izm(1 - z)-m-I zPm(z)(I- Z)-n-I(*) (l-kz)-1 m!(l- kz)-m-I (1 - eaz)-I

(n

+ m)(m)k n e an

Izi < 1

Izi < k-

Izi < e:" Izl < k-

l-kzcosa

k" cos an

1

1 - 2kz cos a + k 2z 2 kz sin a

k" sin an

1 - 2kz cos a

Bn

z

nl

e' - 1

(;)

+ k 2z2

Izi < 21r Izi < 1

zm(1 - z)-m-I

{I +

C) (-I)'C)

Izi < 1

z)k

Izi < 1 Ixl:5 1, Izi < 1

(1 - Z)k

(1 - 2xz + Z2)-1 (1 - xz)(1- 2xz + Z2)-1

Un(x) (**) Tn(x)

I

(*) Pm(Z) is a polynomial of degree m satisfying the recurrence relationpm+l(z) = (mz + l)Pm(z) + z(I - z)p;"(z), PI = J. (**) Tn(x) and Un(x) are the Chebyshev polynomials (see Appendix C).

where

Q2(;) is the Z transform of {gn} and

p(z) is the characteristic

polynomial. Using the table of Z transforms, everything goes similarly as before.

2.6.

Stability of Solutions

The stability problem will be studied in a more general setting in a later chapter. In this section we shall consider only the stability problem for linear difference equations, which is very important in applications.

Definition 2.6.1. The solution Yn of (2.1.4) is said to be stable if, for any other solution Yn of (2.1.4), the following difference is bounded:

In

= Yn

-

Yn,

n E N~o'

(2.6.1)

48

2. Introduction

Definition 2.6.2. The solution Yn of (2.1.4) is said to be asymptotically stable if for any other solution Yn of (2.1.4), one has limn~oo In = O. Definition 2.6.3. The solution Yn of (2.1.4) is said to be unstable if it is not stable. From Lemma 2.1.3 it follows that the difference In satisfies the homogeneous equations (2.1.5). In the case oflinear equations with constant coefficients we have the following results.

Therorem 2.6.1. The solution Yn of (2.3.1) is asymptotically stable if the roots of the characteristic polynomial are within the unit circle in the complex plane. Proof.

From (2.3.14) we have s

lim IYn - Ynl = lim

n ..... co

mi-l

L L

n_OO i=1 j=O

IAijlnilz?l·

If IZil < 1 one has limn~oo IYn - Ynl = 0 and vice versa.

•

Theorem 2.6.2. The solution Yn of (2.3.1) is stable if the module of the roots of the characteristic polynomial is less than or equal to 1 and that those with modules equal to 1 are simple roots. Proof. From (2.3.14), it is evident that the terms coming from roots with modules less than 1 gives a vanishing contribution for n ~ 00, while the terms coming from roots with unit modules, give a bounded contribution to In' since j = O. • It can happen that for some initial conditions, the solution remains bounded even in presence of multiple roots on the unit circle, as shown in the next example.

Example 1.

Consider the equation

Yn+2 - 2Yn+l + Yn

=

0,

Yo

= Yi = c.

This equation admits the solution Yn = c. Often in applications it becomes necessary to study the stability of a constant solution which exists, as we have seen, if gn is constant.

2.7.

49

Absolute Stability

Example 2.

Consider the equation

Yn+2 - Yn+l

+ ~Yn = 2.

We have p(z) = (z - !)2 and all solutions will be asymptotically stable. In particular the constant solution Y = 8 is asymptotically stable. In fact, the general solution is given by Yn = (c 1 + c2n)Tn + 8 and limn~oo (Yn - 8) = O. From definitions 2.6.1, 2.6.2 and 2.6.3, we see that the properties of stability and instability are usually referred with respect to a particular solution Yn' In the case where all solutions tend to a unique solution Yn as n ~ 00, it is often said (especially in numerical analysis) that the difference equation itself (or the numerical method represented by it) is asymptotically stable. Moreover, in some branches of applications, a special terminology is used that is becoming more and more popular and it is worthwhile to mention it. Definition 2.6.4. A polynomial with roots within the unit disk in the complex plane is called a Schur polynomial. Definition 2.6.5. A polynomial with roots in the unit disk in the complex plane with only simple roots on the boundary is called a Von Neumann polynomial.

Using this terminology, the Theorems 2.6.1 and 2.6.2 can be restated as follows. Theorem 2.6.1. The solution Yn is asymptotically stable, if the characteristic polynomial is a Schur polynomial. Theorem 2.6.2. The solution Yn is stable if the characteristic polynomial is a Von Neumann polynomial.

2.7.

Absolute Stability

One of the main application of linear difference equations is the study of discretization methods for differential equations. The difference equations, as we have seen, can be solved recursively. This is not possible for the differential equations and these are usually solved approximately using difference equations that satisfy some suitable conditions. Let us consider

Y'

= f(t, y),

y(to) = Yo,

(2.7.1)

2.

50

where t

Introduction

[to, T) and suppose that this continuous problem has a unique T solution yet). Let h > and t, to + ih with i 0, 1, ... , N E

°

=

=

=-,;.

Let the discrete problem approximating (2.7.1) be denoted by

r; (Yn , Yn+1,""

= 0, (2.7.2) 1, n + k ~ N. We sup-

Yn+k>fn,fn,··. ,fn+k)

where YI = y(tJ + O(h ) , q> 1 and i = 0, 1, ... , k pose that (2.7.2) has a unique solution Yn' As the discrete problem is represented by a difference equation of order k, it needs k initial conditions, only one of which is given from the continuous problem. The others are approximately found in some way. q

Definition 2.7.1. The problem (2.7.2) is said to be consistent with the problem (2.7.1) if

Fh(y(tn), y(tn+1), ... ,y(tn+d'/( tn, y(tn)), ... ,/(tn+k> y(tn+k))) "" Tn

with p

2::

= 0(h P+ 1 )

(2.7.3)

1.

The quantity Tn is called the truncation error. The equation (2.7.3) can be considered as a perturbation of (2.7.2). Definition 2.7.2. The discrete problem (2.7.2) is said to be convergent to the problem (2.7.1) if the solution Yn of (2.7.2) tends to the solution yet) of (2.7.1) for n ~ 00, and tn - to = nh ~ T.

Since the solution of the continuous problem satisfies (2.7.3), which is a perturbation of (2.7.2), the convergence will occur when (2.7.2) will be insensitive to such a perturbation, that is, when (2.7.2) is stable under perturbation. As a consequence, the consistency is not enough to guarantee the convergence. We shall study the problem in some detail for the main class of methods called linear multistep methods (LMF). These methods are obtained when F h is linear in its arguments, namely, k

I aiYn+i ;=0

I e.c.; = 0, ;=0 k

h

(2.7.4)

with (ik = 1 and coefficients ai, f3i are real numbers. Using the shift operator E and the two polynomials p and a given by k

p(z) =

u(z) =

I

j=O k

I

;=0

a.z',

(2.7.5)

f3i z i,

(2.7.6)

2.7.

SI

Absolute Stability

equation (2.7.4) can be written as p(E)Yn - ha(E)f" = O.

(2.7.7)

The two polynomials p(z) and a(z) characterize the method (2.7.2) uniquely and one often refers to them as (p, a) method. The relation (2.7.3) becomes (2.7.8) Theorem 2.7.1. Suppose that f is smooth enough. Then the quantity Tn is infinitesimal of order 2 with respect to h if the following two conditions are verified: p(q)

=

k

I

CI';

;=0

= 0,

(2.7.9)

and p'(1) - 0-(1)

=

k

k

I

iCl'; -

;~O

I

f3;

= o.

(2.7.10)

;~O

(For the proof of Theorem 2.7.1, see problem 2.18). The conditions (2.7.9) and (2.7.10) are said to be consistency conditions. Iff is nonlinear, then the study of stability of (2.7.7) is in general difficult. Usually, one studies the behavior of solutions of (2.7.7) for particular linear functions f, which are called test functions. The most used test functions are

f(y)

=

0,

(2.7.11)

Re A ::; O.

(2.7.12)

and

f(y)

= Ay,

The use of test function (2.7.11) is justified by considering that in (2.7.7) the values of f are multiplied by h and then, in the limit as h ~ 0, the contribution to solutions of the terms containing fn+i can be disregarded. Also, one sees that the methods give good results when applied to the simple equation y' = O. The use of test function (2.7.12) is justified by considering that in the neighborhood of an asymptotically stable solution of (2.7.7), the first order approximation theorem says that the behavior of any solution is established by the linear part that looks like (2.7.12). Let us first consider the test equation (2.7.11). Then (2.7.7) becomes p(E)Yn

= O.

(2.7.13)

52

2. Introduction

Definition 2.7.3. The method (p, u) is said to be O-stable if the solution Yn = 0, n E N+ of (2.7.13) is stable. As a simple consequence of Theorem 2.6.1, we have the following result.

Theorem 2.7.2. polynomial.

The method (p, u) is O-stable if p(z) is a Von-Neumann

Theorem 2.7.3.

The method (p, o ) is convergent in the finite interval (0, T) O-stable.

iff it is consistent and

Proof. Let us write f(ln, y(tn» - fn = Cnen, where en = y(tn) - Yn' Then, subtracting (2.7.4) from (2.7.8), one obtains the error equation

with q

~

1,

j

= 0, 1, ... , q -

1.

The necessity part of the proof will be left as an exercise (see problem 2.20 and 2.21). Suppose now that the method is O-stable and consistent, we shall prove the convergence. We will use the formal series method, from which we get

where QJ(z)

=

k-J

L

qP)Zi,

i~O

co

Qz(z)

=L

qj2)Zi,

i~O

co

Qiz) =

L

'Yn z n.

n~O

By Theorem 2.5.2 and by O-stability we see that 'Yn are bounded. By

2.7.

S3

Absolute Stability

multiplying the formal series we get 00

QI(Z)Q3(Z) =

I

n=O

8~l)zn,

=I

f q~I)'V/n-".

=

8(1) n

l.J

00

ZkQw(Z)Q3(Z)

n=O

8~2)zn,

8(2) n

By equating the coefficients, we have 18~llkl + 18~zlkl. But 18~llkl

k-I

s

J

i=O

k-I

= min(n, k -l),

O, n = 0, 1, ... , k - 1

=

nr-k:

(Z)

i~O qi

{

en+k =

'Yn-k-i>

n

===

k.

+ 8~zlk and len+kl:5

8~llk

i

k-I

j~O

i~O

i

:5 r I /qP)I:5 r i=LO II aA-jl :5 r H I Ij lajl = r HA i~O

where H

and

= 05;;sk-) max leil,

k-I

i

k-I

I I lajl = I

A =

i=Oj~O

it

(k -

i~O

it

18~Zlkl :5 r Iq\2)1 :5 r(to ITil + h IU(E)Cieil)

r( On + h :5 r( On + hL :5

with

itO

to Il3jIICi+jllei+jl)

itO

to Il3jllei+jl),

On = Ln~o hi and L = 0<;<" max ICi+j/. Moreover, Os;js;k J

18~Zlkl :5 r( On + hL

:t: lesl r~1 Il3 l) r

r( On + hL n:t~llesl rtr, Il3rl + hLll3kllen+kl) :5 r( On + hLB nJ~llesl + hLll3kllen+kl), :5

where O, - n,

r- { s

if s if s

:5 n, ===

n,

s, rz = { k,

if s :5 k, if s > k;

i)/ail,

54

2.

Introduction

. and B = ~k-ll L.i=O f3j'I Finally, len+kl :5

r(

+ On + hLIf3kllen+kl + hLB n:t:

HA

1

lesl)

or (1 -

hLIf3klnlen+kl:5 r( HA + On + hLB n:t:

Supposing now that h <

LI~klr and letting I'" =

1_

1

lesl).

h~lf3klr' we get

n+k-l

len+kl

:5 r*hLB I lesl + r*(AH + On)' s=o

Corollary 1.6.3 is now applicable and we obtain len+kl:5 r- AH er*LB(n+k)h .

+

n+k-l (n+k-l s~o aos exp S~l n+k-l

= r- AH er*LB(n+k)h + I

t-u»

)

ITsl er*hLB(n+k-l-s)

s=O

s

r- AH er*LB(n+k)h + O,.;;s,.;;n+k-l max I I r,

Considering that (n

er*hLB(n+k) _ 1 er*hLB - 1 .

:5 T, we get

+ k)h

r*LBT

Ien+k I :5 r *AH e rr- LB + max ITsI ee

r'LB

- 1. -1

The first term in the right-hand side vanishes in the limit as n ~ 00 because we have supposed lim H = 0 and h :5 T / n. The second term becomes O(h P) h-.O because t, = O(hp+l) and the denominator is O(h). In view of the hypothesis, we have convergence and the theorem is proved. • The previous result holds in the limit when h ~ O. It is not very useful in practice because one must use finite integration steps. Then, it is more useful to use the test function (2.7.12). In this case, the equation (2.7.7) becomes k

I

i=O

(aj - hAf3;)Yn+i

= O.

(2.7.14)

Setting hA

=q

and

'7T(z, q)

= p(z) -

qa(z),

(2.7.15)

2.7. Absolute Stability

55

equation (2.7.14) becomes (2.7.16)

1T(E, q )Yn = 0

with Re q

s;

O. The polynomial (2.7.15) is called Dahlquist polynomial.

Definition 2.7.3. The method (p, u) is said to be absolutely stable if the solution Yn = 0 of (2.7.16) is stable.

As a simple consequence of Theorem 2.6.2, we have the following result. Theorem 2.7.4. The method (p, rr) is absolutely stable Neumann polynomial.

if 1T(Z, q)

is a Von

Definition 2.7.4. The set of values q E C for which the method (p, rr) is absolutely stable is called the region of absolute stability. Definition 2.7.5. If the region of absolute stability of the method (p, rr) contains the negative complex half-plane, the method is said to be A-stable.

Example.

The method defined by Yn+l = Yn + h[(l - 8)fn+l + 8fn]

with 0 s; 0

s;

(2.7.17)

1 is called O-method. One has p(Z) =

Z -

1, and

u(z) = (1 - O)z + 8.

It is easily verified that (2.7.9) and (2.7.10) hold. Moreover,

= (l - q + qO)z 1 + q8 --~1 - q + qO

1T(Z, q)

whose unique root is z =

The locus of points satisfying ( _1 - 1_20 ,

Izi =

1 - qO

1 is the circumference with center

0) and radius (1 - 120 )2' It is seen that for 0

s;

0

< ~, the region of

absolute stability is external to the above circle, while for ~ < 0 ",;; 1, such a region is internal. For 0 =~, the circumference degenerates into the imaginary axis and the region of absolute stability becomes the negative half-plane. As a consequence, we have the following result. Theorem 2.7.5.

The e-method is A-stable for 0",;; 0 ",;; ~.

In the cases 8 = 0, 0 = 1/2, 8 = 1, the O-method is called respectively implicit Euler method, trapezoidal method and explicit Euler method.

56

2.8.

2.

Introduction

Boundary Value Problems for Second Order Equations

Here we shall discuss in some detail the most common cases arising in the discretization of the linear second order differential equations. The general problem will be discussed in Chapter 3. Consider the equation Yn+1 - 2zYn + Yn-\

=

°

(2.8.1)

and the three different boundary conditions Yo = YN

=

(2.8.2)

0,

Yo = 0, YN-\ - ZYN Yo

= YN,

Y\

= 0,

= YN+1'

(2.8.3) (2.8.4)

In order to consider these problems, it is very useful to use the general solution of (2.8.1) in the form presented in Example 4 of Section 2.3. First, let us consider the conditions (2.8.2) that give

(2.8.5) and

(2.8.6) Condition (2.8.5) is satisfied if z is a root of UN-\(z), that is, z;

= cos

hr, N

k = 1,2, ... , N - 1, corresponding to which one has N - 1 solutions y~k) = _. knst: Th C2Un-\ (Zk) = C2 sin N' e parameter C2 is usually chosen by impos~ng a normalizing condition. Let us next consider the mixed conditions (2.8.3), which give rise to C\ = 0, c2( U N- 2(Z) - zUN_\(z)) = and using property (6) of Appendix C,

°

the last equation becomes TN(z)

= 0, which is satisfied if Zk = cos 2k + 1 !!., N

2

k = 0, 1, ... , N - 1, corresponding to which one has solutions (k) _

Yn -

_

C2

(k)

•

Un-\(zd - C2 sm

n(2k + 1)n 2N .

Finally, the periodic conditions (2.8.4) yield c\(l- TN(z)) - C2UN_\(Z) = z) + c2(-1 + UN(z)) = O.

o and c\(TN+1(z) -

57

2.9. Problems

This system is a homogeneous one and it has nontrivial solutions corresponding to the roots of its determinants which, after some manipulation, using the properties of T; and U; given in Appendix C, is equal to 2(1- TN (Z)). This expression is zero for Zk = cos 2~7T, k = 0, 1, ... , N - 1.

One verifies then that the corresponding solutions are =

C(k)

Y(nk ) =

C(k)

Y (nk )

1

1

2k7Tn cos - N '

for k

=

N 0 '2'

2k7Tn Zksm cos - + c( k ) sin -' N

N '

2

for k

=

N 1,2, ... , N, k ;t. 2'

In applications, equation (2.8.1) is usually derived from the discretization d 2y

of dx 2 + AY = 0 and then the parameter z has the form z = 1 -!A (ax )2. The results show that the solution exists only for the values of A given by Ak =

(:x?

(1 - Zk), which are called eigenvalues and the corresponding

solutions are called eigenfunctions.

2.9.

Problems

2.1.

Show that en det k(n) with suitably chosen en is a constant of the motion, that is, a relation among the solutions that remains constant for all n.

2.2.

Solve the equation Yn+2 - 2xYn+1 + Yn = 0, Yo = 1, YI = x, where x is a real parameter and show that Yn(x) is a polynomial of degree n in x having the coefficient of x" equal to 2n - l .

2.3.

For

Ixl :5 1, by setting

x

=

cos t, in the previous problem show that

(a) the solution is Yn = cos nt, (b) the roots of the polynomials Yn (x) are simple and lie in the interval [-1,1]. The polynomial Yn(x) (usually denoted by Tn(x)) are called Chebyshev polynomials and they are of primary importance in Numerical Analysis, see Appendix C. 2.4.

Show that the solutions (2.3.8) are linearly independent.

2.5.

Suppose we deposit a sum So in a bank that will pay at the end of each year an interest proportional to the initial sum. Determine the amount at the end of r lh year.

58

2. Introduction

2.6.

In problem (2.5), if the interest paid is proportional to the sum of the deposit, find the amount at the end of rth year.

2.7.

Suppose we have to pay an initial debt A with a constant rate R, paying an interest rate i. Determine the rate R if the debt has to be paid in 20 years.

2.8.

Equation (2.3.17) deserves a special mention both for its historical importance and for its applications in many different fields. (a) Take YI = 1 and Y2 = 1. Determine C1 and C2 and verify that the sequence obtained is the Fibonacci sequence 1, 1,2,3,5,8,13, ... . Yn+1 1 + J5 (b) Show that hm =- n....oo Yn 2 (This number was called golden section by ancient Greek mathematicians and has been important in the arts through the centuries.) (c) Show that I7=1 Yi = Yn+2 - 1.

2.9.

Find the solution of (a) Yn(l + aYn-l) = 1, and (b) Yn(b + Yn-I) = 1. (

• HIllt: Set Yn

= -Zn-) . Zn+1

2.10. The solutions of the previous two problems can be found by using the continuous fractions in the following way (consider, for example, the second case): Yn=

1

b + Yn-i

1

1

1

b

b + Yn-2

b+--

=-+---

1

1

b+-b+ ... When will the fraction converge? 2.11. The best known methods for finding the roots of a single nonlinear equation f(x) = 0 are the Newton method and the secant method, defined by and

2.9.

59

Problems

If a is the root, denoting the error IXn - al with En, show that, for sufficiently smooth f, E n+ 1 = knE~ and En+1 = k~EnEn-I' respectively k; and k~ uniformly bounded. The order of convergence of an iterative method is defined by p

=

lim log En+l. Show that in the first log En

n->OO

case p

=

i t h e second case p 2,While 1 e In

2.12. Solve the difference equation 2.13.

Zn+!

+ z;

1+ v'5. = --

2

=

-(n

+ 1).

(Bernoulli Method). The solution of the linear difference equation has been obtained considering the polynomial p(z). It also happens that for finding the roots of a polynomial it is useful to consider a linear difference. equation. In fact, supposing that the roots ZI, Z2,"" Zk are all simple and that IZ11 > IZ21 > ... from (2.3.7) we have

from which it follows · Yn+1 Il m - - = n->OO

Yn

ZI'

Solving then the difference equation recursively, the ratio of two successive values of the solution will give an approximation of the first root. (a) How can we approximate the root of minimum modulus? (b) What happens if ZI = Z2? (c) How do we choose the initial conditions in order to avoid the effect of multiple zeros of the characteristic polynomial? 2.14 Suppose one has to perform the sum Sn = l:7~1 a, by using the following algorithm: So = 0, Si+1 = S, + ai+I' If one performs the sum not using the real numbers, but approximation of them (floating point numbers), that is, instead of the number a = m' 10 q , where 0.1 ~ m < 1, one uses the number ii = m' 10 q , where 0.1 ~ m < 1 but m has only t digits, this implies that la - iii < 1m - mj . 10q :S 10 q - ,. Study the behavior of the errors, considering that Sj = (Si-1 + ii;)(l + 10-').

60

2.15.

2.

If

An

-2

1

1

-2

o

O. 1.

O.

=

··0 ...1

o

··0

-2 nXn

·1

Show that D; = det An satisfies the equation D n+2 + 2Dn+1 and that D; = (_1)n(n + 1). 2.16.

Introduction

+ D;

=

0

If

2a

b

0

0

£:

An

=

O. 0 0

··0

·b ·e ··2a

nxn

and D; (A) = dett A, - AI), find the eigenvalues of An in the case a 2 ~ be. 2.17.

If {Yn} has generating functionj(z), for Izl < R, show that v.J:» + 1) has generating function II z J~j(t) dt in the same region. (Hint: integrate term by term.)

2.18.

Prove (2.7.9). (Hint: expand in Taylor series starting from y(tn ) and equate to zero the coefficients of h O and h.)

2.19.

3 Show that the solution of Yn+2 - - Yn+1 2 is unbounded.

1

1

+ - Yn = - - , Yo = 0, YI = 0, 2 n+1

2.20.

Prove that if the method (p, u) is convergent (for all j satisfying the hypothesis stated in the text), then it is O-stable. (Hint: take j = 0.)

2.21.

With the hypothesis of Problem 2.20, prove that the method is consistent.

2.22.

Suppose that the relation (2.7.8) is given by p(E)Yn - hu(E)jn = En' where En is a small bounded quantity but not infinitesimal with respect to h. (This happens in practice when we solve the difference equation on the computer.) Using a similar procedure used in the text prove that len+kl ::5 E 1 + E 2, where E 1 and E 2 have respectively a zero and a pole for h = O. Deduce for this that it is not convenient in practice to use h arbitrarily small.

2.10.

61

Notes

2.23. The linear multistep method Zn+1 - 2z n+1 + Zn = h[fn+1 - In] is consistent. It is not O-stable. Show, however, that for the some set of initial conditions the solutions of p(E)zn = 0 remain bounded. (Hint: try with constant initial conditions.) 2.24. Find the region of absolute stability for the following methods: (a) Yn+2 - Yn = 2hfn+l, (midpoint),

h

(b) Yn+2 - Yn ="3 (fn+2 + 41n+1

.

+ In), (Simpson rule).

2.25. Find the solution of the boundary value problem Yn+2 - 2ZYn+1 0, YI - ZYo = 0; YN-I - ZYN = O.

2.10.

+ Yn =

Notes

The material of Sections 2.1 to 2.4 is classical and can be found in 'many classical books. Theorem 2.1.8 is essentially a compact form of a result in Clenshaw [25] and Luke [99], see also Section 5.4. The content of Section 2.5 is also classical but the notation is adapted from Henrici's book [13]. Section 2.6 consists of a collection of results that are scattered in many publications, essentially dealing with Numerical Analysis. The material of Section 2.7 is based on Dahlquist's papers starting from the fundamental one [33] (see also references given in Chapter 6). The books of Lambert [92] and Gear [54] give more detailed arguments on the subject. Theorem 2.7.3 can be found in Henrici [75]. Section 2.8 deals with material that can be found either in some books on numerical analysis (see for example [150]) or in some books on difference equations, see Fort [49]. The classical reference is the book of Atkinson [9] where both the continuous and discrete cases are treated as well as many applications.

CHAPTER 3

Linear Systems of Difference Equations

3.0. Introduction In this chapter, we shall treat systems of linear difference equations. Some results discussed in Chapter 2 are presented here in an elegant form in terms of matrix theory. After investigating the basic theory, method of variation of constants and higher order systems in Sections 3.1 to 3.3, we shall consider the case of periodic solutions in Section 3.4. Boundary value problems are dealt with in Section 3.5, where the classical theory of Poincare is also included. The elements of matrix theory that are necessary for this chapter may be found in Appendix A. Some useful problems are given in Section 3.6.

3.1. Basic Theory Let A(n) be an s x s matrix whose elements aij(n) are real or complex functions defined on N;o and Yn E R S (or C S ) with components that are functions defined on the same set N;o' A linear equation Yn+l = A(n)Yn

+ bn,

(3.1.1)

where b; E R and Yno is a given vector is said to be a nonhomogeneous linear difference equation. The corresponding homogeneous linear difference equation is S

Yn+l = A(n)Yn'

(3.1.2)

63

64

3.

Linear Systems of Difference Equations

When an initial vector y"" is assigned, both (3.1.1) and (3.1.2) determine the solution uniquely on the set N~o as can be seen easily as induction. For example, it follows from (3.1.2) that the solution takes the form (3.1.3) from which follows the uniqueness of solution passing through y"", because n;:~o A(i) is uniquely defined for all n. Sometimes, in order to avoid confusion, we shall denote by y(n, no, y",,) the solution of (3.1.1) or (3.1.2) having y"" as initial vector. Let us now consider the space S of solutions of (3.1.2). It is a linear space since by taking any two solutions of (3.1.2), we can show that any linear combination of them is a solution of the same equation. Let E lo E 2, ... , E, be the unit vectors of R S- and y(n, no, E i), i = 1, 2, ... , s, the s solutions having E, as initial vectors. Lemma 3.1.1. Any element of S can be expressed as a linear combination of y(n, no, E i), i = 1,2, ... , s. Proof. Let y(n, no, c) be a solution of (3.1.2) with y"" = c E R S • From the linearity of S and from c = L:=I ciE;, it follows that the vector

z, =

s

I

i=1

ciy(n, no, E i)

satisfies (3.1.2) and has c as initial vector. Then, by uniqueness, • coincide with y(n, no, c).

Zn

must

Definition 3.1.1. Let j;(n), i = 1,2, ... , s, be vector valued functions defined on N~o' They are linearly dependent if there exists constants a., i = 1,2, ... , s not all zero such that L:=I a;j;(n) = 0, for all n 2:: no. Definition 3.1.2. The vectors};(n), i if they are not linearly dependent.

=

1,2, ... , s, are linearly independent

Let us define the matrix K(n) = (f1(n)'/2(n), ... ,fs(n» whose columns are the vectors };(n). Also let a be the vector (aI, a2,"" aJT.

Theorem 3.1.1. If there exists an ii E N~o such that det K (ii) "" 0 then the vectors };(n), i = 1,2, ... , s are linearly independent.

3.1.

Basic Theory

Proof.

65

Suppose that for n

2=

no s

K(n)a =

I

aJ;(n) = O.

i=l

Since det K(n) .,e 0, it follows that a linearly dependent. •

=

0 and the functions t(n) are not

Theorem 3.1.2. If t(n), i = 1,2, ... , s, are solutions of (3.1.2) with det A(n) .,e 0 for n E N;o' and if det K(no) .,e 0, then det K (n) .,e 0 for all n E N;o' Proof.

For n

2=

no,

det K(n

+ 1) =

det(f)(n

+ 1),fzCn + 1), ... ,!sen + 1))

= det A(n) det K(n),

(3.1.4)

from which it follows that detK(n)

= C~~ detA(i))

detK(no).

•

(3.1.5)

Corollary 3.1.1. The solutions yen, no, E;), i = 1,2, ... , s, of (3.1.2) with det A(n) .,e 0 for n 2= no, are linearly independent. Proof. In this case det K (no) 3.1.1, the result follows. •

= I,

the identity matrix and by Theorem

Corollary 3.1.2. If the columns of K(n) are linearly independent solutions of (3.1.2) with det A(n) .,e 0, then det K(n) ¥- 0 for all n 2= no. Proof. The proof follows from the fact that there exists an det K(n) .,e 0 and the relation (3.1.4). •

n at

which

The matrix K(n), when its columns are solution of (3.1.2) is called Casorati matrix or fundamental matrix. We shall reserve the name of fundamental matrix for a slightly different matrix, and call K (n) the Casorati matrix. Its determinant is called Casoratean and plays the same role as the Wronskian in the continuous case. The Casorati matrix satisfies the equation K(n

Theorem 3.1.3. dimension s.

+ 1) =

A(n)K(n).

(3.1.6)

The space S of all solutions of (3.1.2) is a linear space of

66

3.

Linear Systems of Difference Equations

The proof is an easy consequence of Lemma 3.1.1 and Corollary 3.1.1. Definition 3.1.3. Given s linearly independent solutions of (3.1.2), and a vector C E R of arbitrary components, the vector valued function Yn = K(n)c is said to be the general solution of (3.1.2). S

c

Fixing the initial condition Y"", it follows from definition 3.1.3 that and

= K-1(no)y""

y(n, no, y",,) and in general, for s

E

=

K(n)K-1(no)y""

(3.1.7)

N;o' Ys = c,

y(n, s, c)

= K(n)K-1(s)c.

(3.1.8)

= K(n)K-1(s)

(3.1.9)

The matrix

(n, s)

satisfies the same equation as K(n), i.e., (n + 1, s) = A(n)(n, s). Moreover, (n, n) = I for all n ;:::: no. We shall call the fundamental matrix. In terms of the fundamental matrix, (3.1.7) can be written as y(n, no, y",,) = (n, no)y"". Other properties of the matrix are (i) (n, s)(s, t) = (n, t) and (ii) if -I(n, s) exists then -I(n, s) = (s, n).

(3.1.10)

The relation (3.1.10) allows us to define (s, n), for s < n. Let us now consider the nonhomogeneous equation (3.1.1). Lemma 3.1.2. The difference between any two solutions Yn and )in of (3.1.1) is a solution of (3.1.2).

Proof.

From the fact that

Yn+i

=

A(n)Yn + bn,

)in+! = A( n) )in+! + bn,

one obtains

Yn+1 - )in+1 which proves the lemma.

=

A(n)(Yn - )in+1),

•

Theorem 3.1.4. Every solution of (3.1.1) can be written in the form Yn = )in + (n, no)c, where y; is a particular solution of (3.1.1) and (n, no) is the fundamental matrix of the homogeneous equation (3.1.2).

Proof. From Lemma 3.1.2, Yn - )in E S and an element in this space can be written in the form (n, no)c. If the matrix A is independent of n, the fundamental matrix simplifies because lI>(n, no) = (n - no, 0). •

3.2.

67

Method of Variation of Constants

3.2.

Method of Variation of Constants

From the general solution of (3.1.2), it is possible to obtain the general solution of (3.1.1). The general solution of (3.1.2) is given by y(n, no, c) = ll>(n, no)c. Let c be a function defined on N~o and let us impose the condition that y(n, no, cn) satisfy (3.1.1). We then have

y(n

+ 1, no, cn + l ) = ll>(n + 1, nO)cn+1 = A(n)ll>(n, no)cn+t = A(n)ll>(n, nO)cn + bn,

from which, supposing that det A(n) ."r:. 0 for all n ;;::: no, we get

c, + ll>(n o, n + 1)b n. The solution of the above equation is Cn + 1

=

n = Cno +

C

n-I

I

j=no

ll>(no,j + 1)bj.

The solution of (3.1.1) can now be written as

y(n, no, Cno ) = ll>(n, no)cno + =

ll>(n, no)cno +

n-I

I

ll>(n, no)ll>(no,j + 1)bj

j=no n-I

I

ll>(n,j + 1)bj,

j=no

from which, setting Cn = Yno' we have n-I

y(n, no, Yno)

= (n, no)yno + I

j=no

ll>(n,j + 1)bj.

By comparing (3.1.5) and (3.1.9), it follows that ll>(n, no) = n~~~~ A(i) = 1. We can rewrite (3.2.1) in the form

y(n, no, Yno)

=

(3.2.1)

rr.; A(i), where

CD: A(i) )Yno + jngoCD~I A(s) )bj.

(3.2.2)

In the case where A is a constant matrix ll>(n, no) = An-no and, of course, (n, no) = ll>(n - no, 0). The equation (3.2.2) reduces to n-I

y ( n, no, Yno )

'\' An-j-1b = A n- n°Yno + j=no ~

j'

(3.2.3)

Let us now consider the case where A(n) as well as b; are defined on N±. Theorem 3.2.1. Suppose that R+, j E N±. Then,

L;=-oo IIK- I (j + 1)11 < +00 and IlbJ <

b, b e

00

Yn = is a solution of (3.1.1).

I

s=o

K(n)K-1(n - s)b n- s -

I

(3.2.4)

68

Proof.

3.

For m

E

Linear Systems of Difference Equations

N± consider the solution, corresponding to Ym n-I

= L

y(n, m, 0)

K(n)K-1(j

j=m

= 0,

+ 1)bj

and the sequence y(n, m - 1,0), y(n, m - 2, 0), .... This sequence is a Cauchy sequence since, for T > 0, e > 0 and m 1 chosen I such that I;'::mt- r IIK- (j + 1)11 < e and Ily(n, ml -

T,

Ilj~~-r K(n)K-1(j + l)b ll

0) - y(n, ml> 0)11 =

j

~ IIK(n)llb~.

It follows that the sequence will converge as m given by

Yn

=

n-I

L

K(n)K-1(j

j=-co

~ -00.

Let Yn be the limit

+ 1)bj ,

which is again a solution of (3.1.1). By setting s = n - j - 1, we obtain 00

Yn

=L S~O

K(n)K-1(n - s)bn- s- I '

In the case of constant coefficients this solution takes the form co

Yn

=L

(3.2.5)

A n_S _ 1> Sb

S~O

which exists if the eigenvalues of A are inside the unit circle.

•

Let us close this section by giving the solution in a form that corresponds to the one given using the formal series in the scalar case. Let (3.2.6) By multiplying with z", with z E C, and summing formally from zero to infinity, one has

Letting co

Y(z)

=

L Yn z n,

n~O

B(z)

=

co

L b.s" n=O

69

3.3. Systems Representing High Order Equations

and substituting, one obtains Z-I(y(Z) - Yo) == AY(z)

+ B(z)

from which

+ zB(z)

(I - zA) Y(z) == Yo

and (3.2.7) When the formal series is convergent, the previous formula furnishes the solutions as the coefficient vectors of Y(z). The matrix R(z-1, A) == (Z-IJ - A)-I is called resolvent of A (see A.3). Its properties reflect the properties of the solution Yn'

3.3. Systems Representing High Order Equations Any k t h -order scalar linear difference equation Yn+k

+ PI(n)Yn+k-1 + ... + Pk(n)Yn

can be written as a first order system in R

Y n == (

Y~:I),

Yo == (

Yn;k-I

k

,

~:

(3.3.1)

== gn

by introducing the vectors

),

a, == (

~)

(3.3.2)

~n

Y:-I

and the matrix

o o

1

o

o 1

o o

o

A(n) ==

(3.3.3)

Using this notation equation, (3.3.1) becomes Yn + 1 == A(n) Yn

+ G;

(3.3.4)

where Yo is the initial condition. The matrix A(n) is called the companion (or Frobenius) matrix and some of its interesting properties that characterize the solution of (3.3.4) are given: (i)

The determinant of A(n)-H is the polynomial (_1)k (A k + PI(n)A k-I + ... + Pk(n». When A is independent of n this polynomial coincides with the characteristic polynomial;

3.

70

(ii) detA(n)

Linear Systems of Difference Equations

and is nonsingular if (3.3.1) is really a k th

= (-1)k pk ( n )

order equation; (iii) There are no semisimple eigenvalues of A(n) (see Appendix A). This implies that both the algebraic and geometric multiplicity of the eigenvalues of A coincide. This property is important in determining the qualitative behavior of the solutions; (iv) When A is independent of n and has simple eigenvalues ZI, Z2, ... , z., it can be diagonalized by the similarity transformation A = VDV-\ where V is the Vandermonde matrix V(Zh Z2,' .. , z.) and D = diagf z., Z2, ... , zs). The solution of(3.3.4) is deduced by (3.2.1), which in the present notation becomes n-I

Yn

= (n, no) Yo + I

j= r1o

(n,} + 1)Gj •

The fundamental matrix <1>( n, no) is given by (n, no) = K(n)K- 1(no),

where the Casorati matrix K (n) is given in terms of k independent solutions fl(n)J2(n), ... Jdn) of the homogeneous equation corresponding to (3.3.4), i.e., fk(n)

f2(n

+k

- 1)

)

f,(n+k-l) .

The solution Yn of (3.3.4) has redundant information concerning the solution of (3.3.1). It is enough to consider any component of Y n for n === no + k to get the solution of (3.3.1). For example, if we take the case Yo = 0, from (3.2.1) we have n-·1

Yn =

I

(n,} + I)Gj ,

(3.3.5)

j=no

where, by (3.1.9), (n,} + 1) = K(n)K- 1(j + 1). To obtain the solution y(n + k; no, 0) of (3.3.1), it is sufficient to consider the last component of the vector Yn +1 and we,get n

Yn+k

= I

E[(n

+ I,} + 1)Gj

E[K(n

+ 1)K- 1(j + 1)Ek gj ,

j=no

==

n

I

j=no

where E k = (0,0, ... , 0, 1) T.

(3.3.6)

3.3.

71

Systems Representing High Order Equations

Introducing the functions

H(n

+ le,j) = EI(n + I)K- I (j + 1)Ek ,

(3.3.7)

the solution (3.3.6) can be written as n

I

Yn+k =

r- no

H(n

+ k,j)gj'

(3.3.8)

The function H(n + le,j), which is called the one-sided Green's function has some interesting properties. For example, it follows easily from (3.3.7) that

H(n+k,n)=l.

(3.3.9)

In order to obtain additional properties, let us consider the identity k

I

=I

i=1

(3.3.10)

EjET,

where E, are the unit vectors in R k and I is the identity matrix. From (3.3.7), one has k

H(n

+ le,j) = I ErK(n + 1)EjETK- I (j + 1)EkJ

(3.3.11)

i=1

which represents the sum of the products of the elements of the last row of k(n + 1) and the elements of the last column of k-l(j + 1). By observing that the elements in the last column of the matrix K-l(j + 1) are the cofactors of the elements of the last row of the matrix K (j + 1), it follows that

H(n+kJ')= 1 , det K(j + 1) fl(j fl(j

+ I) + 2)

fk(j fk(j

+ 1) + 2)

x det

.

ft(j+k-1) fl(n + k)

fin

+ k)

(3.3.12)

fk(j+k-l) fk(n + k)

As a consequence, one finds

H(n H(n

+ k, n)

+k -

(3.3.13)

= 1,

i, n) = 0,

i = 1,2, ... , k - 1,

(3.3.14)

and

H(n+k,n+k)=(-I)

k-l

d

det K(n + k) 1 ( k )=- ( k)' et K n + + 1 P« n +

(3.3.15)

72

3.

Linear Systems of Difference Equations

Proposition 3.3.1. The function H(n,j),for fixed], satisfies the homogeneous equation associated with (3.3.1), that is, k

I

p;(n)H(n

+k

- j,j)

= o.

;~O

Proof. It follows easily from (3.3.12) and the properties of the determinant. •

The solution (3.3.8) can also be written as Yn

n-k =

I

i v n«

(3.3.16)

H(n,j)gj,

with the usual convention L~:no = O. For the case of arbitrary initial conditions together with equation (3.3.1), one can proceed in similar way. From the solution Yn + 1

n

= K(n + I)K- 1(no)Yo + I

K(n

+ 1)K- 1(j + 1)q,

j=no

by taking the k tb component we have Yn+k

= »: K(n + 1)K-

1(n

o)Yo +

n

I

j=no

H(n

+ k,j)gj'

In the case of constant coefficients, the expression for H(n,j) can be simplified. Suppose that the roots Zi of the characteristic polynomial are all distinct. We then have, from (3.3.11), 1 1 k H(n

+ k,j) =

TI z{'+1

i~1

k

TI z{+1 det K(O)

det

i=1

k-2 ZI (n-j)+k-I

ZI k "

z:

= i=1

n-j+k-l,,( Z; "i ZI,""

det K(O)

Zk

)

=

I

k

n-j+k-I

_Z::.. .i_ _ ;=1 p'(z;)-'

(3.3.17)

where V;(ZI,"" zd are the cofactors of the jtb elements of the last row and P'(Zi) is the derivative of the characteristic polynomial evaluated at z.. In this case, as can be expected, one has H(n +k,j) = H(n + k - j, 0). By denoting with H(n + k - j) the function H(n + k - j, 0), the solution

73

3.3. Systems Representing High Order Eqnations

of the equation k

I PJln+k-i = gn i=1 such that Yi

= 0, i = 0, 1, ... , k - 1, is given by Yn =

n-k

I

(3.3.18)

H(n - j)gj'

j~O

The properties (3.3.12), (3.3.13), (3.3.14) reduce to H(k) == 1, H(k - s) s

1

=

0,

.

= 1, ... , k -1, and H(O) = --, respectively. Pk

We shall now state two classical theorems on the growth of solutions of (3.3.1) with gn = O. If lim p.In) = Pi> i

Theorem 3.3.1. (Poincare).

n-+CO

= 1,2, ... , k,

and if the

roots of k

I

i-«

i PiAk- = 0,

(3.3.19)

Po = 1

have distinct moduli, then for every solution Yn,

. Yn+l , II m - = As> Yn

n-+OO

where As is a solution of (3.3.19). Proof. Let pi(n) = Pi + lli(n) where, by hypothesis, lli ~ 0 as n ~ matrix A(n) can be split as A(n) = A + E kll T(n), where

A=

o o

1

o

1

o

The

o

o

o -Pk

o

00.

-Pk-l

-PI

11 T(n) = (-llk(n), llk-l(n), .... , -lll(n)) and E[ == (0,0, ... ,1). Equation (3.3.4) then becomes Y n + 1 = AYn + E kll T (n) Yn' Now A = VA V-I, where V is the Vandermonde matrix made up of the eigenvalues. AI, A2 , ••• , Ak of A (which are the roots of (3.3.19)) and A = diag(A), A2 , ••• , Ak ) . We suppose that IA 11 < IA 2 1 < ... < IAkl. Changing variables u(n) = V-I Y n and letting f n = V- IEk ll T(n)V, we get u(n+1)=Au(n)+r nu(n). The elements of I'(n), being linear combinations of lli(n), tend to zero as n ~ co. This implies that for any matrix norm, we have IIrn I ~ O. Suppose now that

74

3.

max

l'$i:'Sk

IUj(n)1 = lus(n)l.

The index s will depend on n. We shall show that an

no can be chosen such that for n

In fact, we know that for i < j,

IAil1+ e <

IA j

-

e

Linear Systems of Difference Equations

2': no, the

function sen) is not decreasing.

:~;: < 1. Take e > 0 small enough such that

1 and choose no so that for n

2': no, [I'

n

1100

< e. Setting sen + 1) = j,

it follows that

and Iuj(n

+ 01 ~ IAJuj(n)[ + elus(n)1 ~ (IAjl + e)lus(n)1 IUj(n + 1)12': IAjlluj(n)l- elus(n)l·

Consequently, if sen + true:

0 == j were less than lu/n + 01 IAjl us(n + 0 ~ A2

+e

I I I-

I

sen), the following would be

e

< 1,

which is in contradiction with the definition of j. For n > N suitably chosen, the function s(n) will then assume a fixed value less or equal to k. We shall show now that the ratios

IUj(n)1 jus(n)I'

(3.3.20)

j~s

tend to zero. In fact we know that for n > N,

lu(n)1

lu:(n)1 ~

. a < 1. This means

that a is an upper limit for (3.3.20). We extract for n 2': N a subsequence n1, n2, ... , for which (3.3.20) converges to a. Suppose first that j > s. Then luj(np + lus(np +

IAjlluj(np)l_ e ::lus(np)1 01- IAsl + e

01

:>

We take the limit of subsequence, obtaining a lower limit . luj(np + 01 hm p_oo Ius(np + 0 12':

This implies

IAjia - e IAs I+ e .

75

3.4. Periodic Solutions

for arbitrary small a

= O.

E.

Since

I~~II > 1, the

previous relation holds only for

In the case j < s, similar arguments, starting from IUj(np Ius(n p

+ 1)1 <: + 1)1-

IAjlluj(np)1 + e lus(np)1 IAsl- e

lead to the same conclusion that a = O. Consider now the original solution Yn' We have, considering the first two rows of Yn = Vu(n): Yn

=

Ui(n)) L uJn) + us(n) = u.(n) ( 1 + Lse s -(-) , n

tv:s

i

Us

and

One easily verifies, by using the previous results, that Yn+l · -= 11m Yn

n ....OO

\

I\s.

•

We shall now state, without proof, the following theorem due to Perron, which improves the previous one.

Theorem 3.3.2. (Perron).

Suppose that the hypotheses of Theorem 3.3.1 are verified, and moreover Pk(n) ~ 0 for n ~ no. Then k solutions f1 J2' ... J, can be found such that

. j;(n + 1) hm I'() "-+00 Ji n

3.4.

=A

j,

i = 1,2, ... , k:

Periodic Solutions

Let N be a positive integer greater than 1, A(n) real nonsingular s x s matrices and b; vectors of R S • Suppose that A( n) and b; are periodic of period N That is A(n + N) = A(n) and bn + N = b.; A period solution of period N is a solution for which Yn+N = Yn' The trivial example of periodic solution of the homogeneous equation (3.1.2) is Yn = O. For the nonhomogeneous equation (3.1.1), if there exists a constant vector ji such that (A(n) - 1)ji + b; = 0 for all n, then ji is trivially periodic. In this section, we shall look for the existence of nontrivial periodic solutions.

76

3.

Proposition 3.4.1. the relations

Linear Systems of Difference Equations

IfA( n) is periodic, then the fundamental matrix <1> satisfies

+ N, N) <1>(n + N,O)

<1>(n

=

<1>(n, 0)

= <1>(n, 0)<1>(N; 0).

(3.4.1)

Proof. The proof follows easily from the definition of <1> (see 3.2.1) and the hypothesis of periodicity of A(n). • Theorem 3.4.1. If the homogeneous equation (3.1.2) has only Yn = 0 as periodic solution, then the nonhomogeneous equation (3.1.1) has a unique periodic solution ofperiod N and vice versa.

Proof. If Yn is a periodic solution for (3.1.2) and (3.1.1), it must satisfy respectively YN = Yo = <1>(N; O)yo YN

= Yo = <1>(N, O)yo +

N-l

L

j=O

<1>(N,j

+ 1)bj ,

from which it follows that Yo must satisfy

By., = 0 BYN =

(3.4.2)

N-l

L

j=O

<1>(N,j + l)bj ,

(3.4.3)

where B = I - <1>(N, 0). The solution of each of the two equations (3.4.2) and (3.4.3) will give the initial condition for the periodic solutions of the equations (3.1.2) and (3.1.1). If (3.4.2) has the unique solution Yo = 0, it follows that det B '" 0 and this implies that (3.4.3) has a unique nontrivial solution. The converse is proved similarly. Suppose now that det B = 0 and N(B), the null space of B, has dimension k < s. This means that the equation (3.4.2) has k solutions to which will correspond k periodic solutions of (3.1.2). The problem (3.4.3) has solutions if the vector I.'::~l <1>(N,j + l)bj is orthogonal to N(B T ) . Suppose that v(l), V(2), ••• , V(k) is a base of N(B T ) , then we have V(i)T -

V(i)T <1>(N,

0)

= 0,

i

= 1,2, ... , k,

(3.4.4)

from which i = 1,2, ... , k.

(3.4.5)

77

3.4. Periodic Solutions

By imposing the orthogonality conditions we get N-I

V(i)T

I

CP(N,j

j=O

N-I

I

+ 1)bj

=

CP(N,j

+ I),

V(i)T

CP(N,j

j=O

+ 1)bj = O.

(3.4.6)

k,

(3.4.7)

Let XJi)T

=

V(i)T

i = 1,2, ... ,

so that we obtain N-I

~

'--

j=O

x(i)T J

bJ· = 0 ,

•

i = 1,2, ... , k.

(3.4.8)

From this result we get the following theorem. Theorem 3.4.2. If the homogeneous equation (3.1.2) has k periodic solutions ofperiod Nand if the conditions (3.4.8) are verified, then the nonhomogeneous equation (3.1.1) has periodic solutions ofperiod N

Let us now consider the functions defined by (3.4.7) and let xj , j be one of them. Then,

XT-l =

v T CP(N,j) = v T CP(N,j

E

«;

+ I)CP(j + l,j).

The vector xj are periodic with period N and satisfy the equation

XT-l =

(3.4.9)

x]A(j),

which is called the adjoint equation of (3.1.1). The fundamental matrix '1'( t, s) for such equation satisfies the equation

'I'(t - 1, s)

= 'I'(t, s)A(t),

'I'(s, s)

=

1.

(3.4.10)

Using (3.4.9), Theorem 3.4.2 can be rephrased as follows. Theorem 3.4.2'. If the homogeneous equation (3.1.2) has k periodic solutions of period N and if the given vee/or b = (b o , b l , ••• , bN - 1 ) T is orthogonal to the periodic solutions of the adjoint equation, then the nonhomogeneous equation (3.1.1) has periodic solutions of period N. Theorem 3.4.3. Suppose A(n) and b; periodic with period N. If the nonhomogeneous equation (3.1.1) does not have periodic solutions with period N, it cannot have bounded solutions.

78

3.

Linear Systems of Difference Equations

Proof. If (3.1.1) has no period solutions, by Theorem 3.4.1, the equation (3.1.2) has such solutions and of course the conditions (3.4.8) are not verified. Let v T be a solution of v TB = O. Then v T I.j:~l et>(N,j + 1)bj "r:. O. Consequently, for every solution Yn of (3.1.1), we have

VTYN = vTet>(N,O)yo+ v T

N-l

I

j=O

= vTyo+ v T

et>(N,j+ 1)bj

N-l

I

et>(N,j+ 1)bj •

j=O

Moreover, by considering the periodicity of b, and (3.4.1) we get

V

TY2N=V TYN+V T

N-l

I

j=O

et>(N,j+1)bj

N-l

I

= v TYo+2v T

j=O

et>(N,j+1)bj>

and in general, for k > 0, N-l

I

VTYkN=VTYO+kv T

j=O

showing that Yn cannot be bounded.

et>(N,j+1)bj

•

The matrix U"" et>(N, 0) has relevant importance in discussing the stability of periodic solutions. From

et>(n

+ N,O)

= et>(n

+ N, N)et>(N, 0)

and (3.4.1), it follows that:

et>(n

+ N,O)

=

et>(n, 0) U

(3.4.10)

and in general, for k > 0, et>~n

+ kN,O)

= et>(n, 0)

tr.

(3.4.11)

Suppose that p is an eigenvalue of U and v the corresponding eigenvector. Then, Letting et>(n, o)v

et>(n

+ N, O)v = et>(n, 0) Uv = pet>(n, O)v.

= o,

for n

;;=:

0, we get (3.4.12)

This means that the solution of the homogeneous equation having initial value vn , after one period, is multiplied by p. For this reason the eigenvalues of U are usually called muItiplicators. The converse is also true. If Yn is a solution such that Yn+N = PYn, for all n, then in particular is YN = PYo and that means UYo = PYo from which follows that Yo is an eigenvector of U.

79

3.5. Boundary Value Problems

3.5.

Boundary Value Problems

The discrete analog of the Sturm-Liouville problem is the following: (3.5.1) (3.5.2) where all the sequences are of real numbers,

rk

> 0,

ao ¢

0,

aM ¢

0 and

o:s; k :s; M. The problem can be treated by using arguments very similar to the continuous case. We shall transform the problem into a vector form, and reduce it to a problem of linear algebra. Note that the equation (3.5.1) can be rewritten as Let k = 2, .... , M -1,

Y

= (Yl , Y2, ... , YM ) T,

and

A=

al

-PI

0

-PI

a2

-P2

0

0

0

(3.5.3)

-PM-l

0

0

-:PM-l

aM

then the problem (3.5.1), (3.5.2) is equivalent to (3.5.4)

Ay = ARy.

This is a generalized eigenvalue problem for the matrix A. The condition for existence of solutions to this problem is

det(A - AR)

=

0,

(3.5.5)

which is a polynomial equation in A. Theorem 3.5.1.

Proof.

The generalized eigenvalues of (3.5.4) are real and distinct.

Let S = R- 1/2 . It then follows that the roots of (3.5.5) are roots of det(SAS - AI) = O.

(3.5.6)

Since the matrix SAS is symmetric, it will have real and distinct eigenvalues.

80

3.

Linear Systems of Difference Equations

For each eigenvalue Aj , there is an eigenvector v'. which is the solution of (3.5.4). By using standard arguments, it can be proved that if / and are two eigenvectors associated with two distinct eigenvalues, then

y

•

M

(

i j - 0. Y,i RY i) -= '"' L.. rsJ!.Ys

s=1

Definition 3.5.1. R-orthogonaI.

(3.5.7)

Two vectors u and v such that (u, Rv) = 0 are called

Since the Sturm-Liouville problem (3.5.1), (3.5.2) is equivalent to (3.5.4), we have the following result. Theorem 3.5.2.

Two solutions of the discrete Sturm-Liouville problem corresponding to two distinct eigenvalues are R-orthogonal. Consider now the more general problem

Yn+l

=

A(n)Yn

+ bn,

(3.5.8)

where Yn, b; E R S and A(n) is an s x s matrix. Assume the boundary conditions are given by N

L LJ'n, =

i=O

(3.5.9)

w,

where n, E Nt, n, < nj+1, no = 0, w is a given vector in R S and L j are given s x s matrices. Let (n,j) be the fundamental matrix for the homogeneous problem Yn+1

= A(n)Yn,

(3.5.10)

such that <1>(0,0) = 1. The solutions of (3.5.8) are given n-l

Yn

= (n, O)yo + L

j=O

(n,j + l)bj,

(3.5.11)

where Yo is the unknown initial condition. The conditions (3.5.9) will be satisfied if N

L

i=O

LJ'n, =

N

L

i=O

Lj(nj, O)yo +

N

L

i=O

n,-1

i;

L

j=O

(nj,j + l)bj

= w,

3.5.

81

Boundary Value Problems

which can be written as N

"N- I

N

I Lj
I

Lj

j~O

r:/>(nj,j + 1) T(j

+ 1, nJbj = w,

j~O

(3.5.12) where the step matrix T(j, n) is defined by

T(j, n)

By introducing the matrix Q = becomes

Qyo =

W -

I 0

= {

N

"N- I

;=0

j~O

I I

forj f ' or)

S

>

n, n.

I: Lj
the previous formula

Lj
(3.5.13)

If the matrix Q is nonsingular, then the problem (3.5.8) with boundary conditions (3.5.9) has only one solution given by

Theorem 3.5.3.

nN-l

=
y"

s=O

G(n, s)b"

(3.5.14)

where the matrices G(n,j) are defined by: G(n,j)

=


-
I

i=O

Lj
(3.5.15)

Proof. Since Q is nonsingular, from (3.5.13) we see that (3.5.11) solves the problem if the initial condition is given by N

Yo

=

Q-I W

Q-I

-

nN-l

I I j~O

j=O

Lj
(3.5.16)

By substituting in (3.5.11) one has

y"

=
-


N

"N- I

j~O

j=O

I I

Lj
nN-l

+ I
"XI [ Jo

=

L;
+ 1, n;)]bj ,

+ 1, nJbj

82

3.

Linear Systems of Difference Eqnations

from which by using the definition (3.5.14) of G(n,j), the conclusion follows • The matrix G(n, i) is called Green's matrix and it has some interesting properties. For example, (1) for fixed j, the function G(n,j) satisfies the boundary conditions L.~o LjG(nj,j) = 0, (2) for fixed j and n rf j, the function G(n,j) satisfies the autonomous equation G(n + 1,j) = A(n)G(n,j), and (3) for n = j, one has GU + 1,j) = A(j)G(j,j) + I. The proofs of the above statements are left as exercizes. (See problems 3.19, 3.20). If the matrix Q is singular then the equation (3.5.15) can have either an infinite number of solutions or no solution. Suppose, for simplicity, we indicate by b the righthand side of (3.5.13), then the problem is reduced to establishing the existence of solutions for the equation Qyo = b.

(3.5.17)

Let R( Q) and N( Q) be respectively the range and the null space of Q. Then (3.5.17) will have solutions if b E R( Q). In this case if c is any vector in N( Q) and Yo any solution of (3.5.17), the vector c + Yo will also be a solution. Otherwise if b e R( Q), the problem will not have solutions. In the first case (b E R(Q)), a solution can be obtained by introducing the generalized inverse of Q, defined as follows. Let r = rank Q. The generalized inverse QI of Q is the only matrix satisfying the relations (3.5.18) (3.5.19) where P and P l are the projections on R( Q) and R( Q*) (Q* is the conjugate transpose of Q) respectively. It is well known that if F is an s x s matrix whose columns span R(Q), then P is given by F(F* F)-l F*.

By using QI the solution

Yo

of (3.5.17) when b

(3.5.20)

e R(Q)

is given by (3.5.21)

In fact, we have Qyo = QQTb = Pb = b. A solution YM of the boundary value problem can now be given in a form similar to (3.5.14) and (3.5.15) with replaced by QT. This solution, as we have already seen, is not unique.

o:'

3.6.

83

Problems

In fact if C E N( Q), Yk = (n, O)c + Yn will also be a solution satisfying the boundary conditions since N

N

I L;)'n, = ;=0 I (n ;=0

i,

O)c +

n

I L;jn' = Qc + w = w. ;=0

When be R( Q), the relation (3.5.21) has the meaning of least square solution, because Yo minimizes the quantity II Qyo - b 112 and the sequence Yn defined consequently may serve as an approximate solution.

3.6.

Problems

3.1.

Show that the vectorsjjfn) = (1, n)T andf2(n) = (n, n 2)T are linearly independent even if the matrix K (n) = UI(n),f2(n» has determinant always zero. Can the two vectors be solutions of (3.1.2)?

3.2.

Prove that if A(n) is nonsingular for n ~ no, then -I(n, s) exists for n, s G:: no. (Hint: write (n, s) = n~:: A(i) and invert.)

3.3.

Show that if K (n) satisfies (3.1.5), with K (n) ¥- 0 for all n, its column form a set of linearly independentsolutions of (3.1.2).

3.4.

Prove that if A is a constant matrix, (n, no) = (n - no) either directly by the explicit expression of (n, no) or as a solution of (3.1.5).

3.5.

Prove the relations (3.3.14) and (3.3.15).

3.6.

Prove proposition 3.3.1 directly by using the expression (3.3.11).

3.7.

Verify that (3.3.16) satisfies equation (3.3.1).

3.8.

Verify that Yn = I~~-oo ASb n_s_1 is a solution of Yn+1 When does this solution have a meaning?

3.9.

Verify that H(n) is solution of I~=o P;)'n+k-i

=

= AYn + b-.

O.

3.10. Supposing that I;:o H(n - j)gj has a meaning, show that it is the solution of the autonomous linear scalar difference equation. When does it have a meaning? 3.11. As in the previous exercise with the function Yn = I;:'-oo H(n - j)gj, where H(n) is the solution of I:=op;H(n + k - i) = 8;\ find the conditions on the roots of p(z) in order to have Yn as the only bounded solution. 3.12. Consider the second order difference equation Yn+2 + PIYn+1 gn' Construct the function H (n) of the previous exercise.

+ P2Yn =

84

3. Linear Systems of Difference Equations

+

3.13.

Find the function H(n - j) and the solution of Yn+Z - 2(cos t)Yn+l Yn = gn, Yo = Yl = O.

3.14.

By using the onesided Green's function H defined by (3.3.17) find the inverse of the matrix:

f3

a 'Y

o

o

f3 a

'Y

When does the inverse exist for N

~

<X)

NxN

(infinite matrix)?

3.15.

Deduce the adjoint equation in the scalar case as defined in section 2.1 from the Definition 3.4.9.

3.16.

Show that

/).2Yn +

4 sin? 2~ Yn

= 0,

Yo

= 0,

YM

=E

(;eO), has no

solution. It is interesting to notice that the continuous analog of this problem, namely, Y" + 4y sin" 2~ = 0, Yo = 0, y( M) . tion ytr]

= E,

has solu-

sin(2tsin~) =

(

sin 2M sin

2~

)

E.

;z 2

= 0, y(O) = y(M) = 0, for M > 2.

3.17.

Find the solution of a 2Yn +

3.18.

Find the eigenvalues of I1 zYn + AYn = 0, Yo = 0, YM+l = O. Show that the problem is equivalent to finding the eigenvalues of the matrix: 2

A=

-1 0

-1 2

0 -1

Yn

0 0

0

-1 0

0

-1

2

MxM·

3.19.

Show that for fixed j and n ;e i. G(n,j) satisfies the homogeneous equation G(n + l,j) = AG(n,j).

3.20.

Show that for j + 1 and j in the same interval [n i GU + 1,j) = AU)GU,j) + 1.

3.21.

)::0 LiG(ni,j) = O.

Show that for fixed

i.

1,

ni -1] one has

G(n,j) satisfies the boundary conditions

3.7. Notes

85

3.22. Verify that (3.5.14) satisfies the Equation (3.5.8) and Conditions (3.5.9).

3.7. Notes Most of the material of Sections 3.1, 3.2 and 3.3 appears in several different books. The major references are Miller [115] and Hahn [65]. The Poincare and Perron's theorems can be found in Gelfond [57,58]. The periodic solutions are treated in Halanay [69], Halanay and Wexler [68] and Corduneanu [31]. The boundary value problems are treated in Fort [49], Mattheij [104, 105], Agarwal [3,4] and Hildebrand [78]. Theorem 3.5.2 has been taken from Agarwal [3].

CHAPTER 4

4.0.

Stability Theory

Introduction

In Section 4.1, we define various stability notions and give some simple examples. Sections 4.2 to 4.4 are devoted to the theory of stability of linear difference equations. Using norm as a candidate for measure and comparison principle, we develop general results on stability in Section 4.5. Nonlinear variation of constants formula is obtained in Section 4.6 and stability by first approximation is treated in Section 4.7. Sections 4.8 and 4.9 investigate stability theory in terms of Lyapunov functions and comparison principle, and offer several direct theorems of importance. A discussion of converse theorems is the content of Section 4.10. Section 4.11 deals with the concepts of total and practical stabilities that are important in applications to numerical analysis. Several problems are included in Section 4.12 that complement the theory that is presented in the chapter.

4.1.

Stability Notions

Let us denote by B(y, 8) the open ball having center at y and radius 8. If = 0, we shall use the notation B/lo Let Yo E Ba andf: R ~ R be a bounded function in B a • The solution of the difference equation y

S

Yn+l

= fen, Yn),

Yno

= Yo,

S

(4.1.1)

will remain in Ba for all n ~ no such thatf(n, Yn) E B a. Let I no c N~o be the set of all such indicies.

87

88

4. Stability Theory

Definition 4.1.1.

The points ji

E

R S , which satisfy the algebraic equation

f(n, ji) = ji

for all n, are called fixed points (or critical or limit points) of (4.1.1). We shall suppose, for simplicity, that there is a fixed point at the origin. If the fixed point is not at the origin, changing coordinates z" = y" - ji, equation (4.1.1) is transformed into Z,,+1

= f(n, z; + ji) -

f(n, ji) == F(n, z,,),

which has the fixed point at the origin. Definition 4.1.2. (i)

(ii) (iii) (iv)

The solution y = 0 of (4.1.1) is said to be

stable if given e > 0, there is a I) (s, no) such that for any Yo E B; the solution y" E BE' Uniformly stable if it is stable and I) can be chosen independent on no. Attractive if there is I)(no) > 0 such that for Yo E B s one has ,,_00 lim y" =

o.

Uniformly attractive if it is attractive and I) can be chosen independent of no. (v) Asymptotically stable if it is stable and attractive. (vi) Uniformly asymptotically stable if it is uniformly stable and uniformly attractive. (vii) Globally attractive if it is attractive for all starting points Yo E R (viii) Globally asymptotically stable if it is asymptotically stable for all Yo E R (ix) Exponentially stable if there exists I) > 0, a> 0, TJ E (0, 1) such that if Yo E Bs , then S

•

S

•

(x)

Ip-stable if it is stable and moreover for some p 00

L

j=no

(xi) Example.

Ily(j, no, yo)IIP <

> 0,

00.

Uniformly Ip-stable if the previous summation converges uniformly with respect to noConsider the difference equation X"o

= Xo,

(4.1.2)

89

4.1. Stability Notions

where a, are real numbers. The solution of (4.1.2) is

= Xo

Xn

n-l

I1

(4.1.3)

a..

i=no

We then have the following cases: (a) If

IX[ ail

<M(no),

(4.1.4)

then

and it suffices to take

to get stability. (b) If

111 ail ~

M,

(4.1.5)

then it suffices to take ~ = e] M to get uniform stability. This condition is satisfied for example if a, = cos i. (c) If

!~~ 111 ail = 0,

(4.1.6)

then, as this is a particular case of (4.1.5), the stability follows and moreover,

!~~ IX I = IXol !~ IiTI: ai I = 0, n

obtaining the asymptotic stability of the zero solution. This condition is not satisfied when a, = cos j as in the previous case, showing that uniform stability and asymptotic stability are two different concepts. (d) If

lintio ail < aTJn-no,

(4.1.7)

where a > 0 and 0 < TJ < 1, then x = 0 is exponentially stable. There is a sort of hierarchy among all these different kinds of stabilities. For example, uniform asymptotic stability implies asymptotic stability and uniform stability implies stability. Furthermore, asymptotic stability implies stability.

90

4. Stability Theory

Equation (4.1.1) is said to be autonomous iff does not depend explicitly on n. For autonomous equations uniform stability concepts coincide with the stability ones. This can be seen by observing that if y(n, no, Yo) satisfies the autonomous equation y(n + 1, no, Yo) = f(y(n, no, Yo», then y(nno, 0, Yo) satisfies the same equation. Since the two solutions assume the same value for n = no, they must coincide for n E 1"0' This means that for autonomous equations we can always fix no = and if there is stability for no = 0, then it is true for all no, which means that stability is uniform. Attractivity is a different concept than stability, as the following example demonstrates. Consider the system

°

(4.1.8)

where r~ = x~ + y~. The origin of (4.1.10) is globally attractive but unstable (see problem 4.3). Of course lp-stability implies asymptotic stability, because the convergence of the series };lly(j, no, Yo)/IP will imply that y(j, no, Yo) tends to zero. The converse is not true. Exponential stability, however, is sufficient for lp-stability as the next result shows. Theorem 4.1.1. lp-stable.

If the solution y

Proof.

By definition we have:

with 0 <

1]

°

is exponentially stable, then it is also

< 1, and a > O. Therefore I';:nJyJP

1 aPllYollP--, 1 - 1]P

4.2.

=

::5

a" /IYoliP I';: no (1]P)i-no ::5

•

The Linear Case

We shall discuss two results that characterize both uniform stability and uniform asymptotic stability in terms of the fundamental matrix. Consider

Yn+1

= A(n)Yn,

where A(n) is an s x s matrix.

Y"o

= Yo,

(4.2.1)

91

4.2. The Linear Case

Theorem 4.2.1. Let <1>( n, no) be the fundamental matrix of (4.2.1). Then the solution Y = 0 is uniformly stable if there exists an M > 0 such that

1I(n, no)11 < M, Proof. have

for n

~

(4.2.2)

no.

The sufficiency follows from the fact that Yn

= (n, no)Yo. For, we

llYn II::; 1I(n, no)IIIIYoll::; MIIYoll, llYn II < 10 if IIYol1 < 10M- I .

and hence To prove necessity, if there is uniform stability, then

1I(n, no)Yoll < for

IIYoll <

5. Taking

Xo

= 11;:11

1

the previous formula shows that

sup

11-"011=1

1I<1>(n, no)xoll

(4.2.3)

is bounded. But (4.2.3) is just the definition of the norm of (n, no) (see Appendix A). • Theorem 4.2.2. The solution Y = 0 of (4.2.1) is uniformly asymptotically stable if there exists two positive numbers a, TJ with TJ < 1 such that 1I<1>(n, no)11 ::; aTJn-no.

Proof. The proof of the sufficiency is as simple as before. The necessity follows by considering that if there is uniform asymptotic stability, then fixing 10 > 0 there exists 5> 0, N(e) > 0 such that for Yo E B o'

1I(n, no)Yoll < 10 for n

~

no + N( e).

As before, it is easy to see that

1I(n, no)1I <

TJ

for n ~ no + N( e), where this time TJ can be chosen arbitrarily small. Moreover, because the uniform asymptotic stability implies the uniform stability, we obtain 1I<1>(n, no)II is bounded by a positive number a for all n ~ no. We then have for n E [no+ mN(e), no+ (m + 1)N(e)],

1I(n, no)II

::; 1I(n, no + mN( e) II II (no+ mN( e), no + (m

... I (no + N(e), no)11 with TJI

= TJ NtiJ < 1,

at

< an'"

=

M+l aTJ-ITJ N(e) N(e)

=

= aTJ -I and this proves the theorem.

- 1)N( e) II

...

alTJr no •

As a result of this theorem we see that for linear systems, uniform asymptotic stability is equivalent to exponential asymptotic stability.

92

4.

Stability Theory

4.3. Autonomous Linear Systems In this section we shall be concerned with linear autonomous equations because of their importance in applications. Of course, the results of theorems 4.2.1 and 4.2.2 hold true, but we can give more explicit results in this special case. The solution of the homogeneous autonomous equation (4.3.1)

Y"o = Yo, is given by

(4.3.2) From the matrix theory (see Appendix A), we know that Aq

=

kt ~~1

(4.3.3)

(i)ACi(A - AkI)iZk"

where r is the number of distinct eigenvalues of A, mi; is the multiplicity of Ak and Zk, are component matrices of A, which are independent of Ak. From (4.3.3) it follows that if the eigenvalues of A are in the unit disk, then lim An-no = 0 and vice versa. This leads to the following result. n ....oo

Theorem 4.3.1. The solution y = 0 of (4.3.1) is asymptotically stable eigenvalues of the matrix A are inside the unit disk.

iff the

We recall that an eigenvalue Ak is said to be semisimple if (A - AkI)Zk, (see Appendix A).

=0

Theorem 4.3.2. The solution y = 0 of (4.3.1) is stable iff the eigenvalues of A have modulus less or equal to one, and those ofmodulus one are semisimple. Proof.

From (4.3.3) it is easy to see that for semisimple eigenvalues the

term containing q is

(~)

=

1, which does not grow for q

~ 00.

•

If the matrix A is a companion matrix, then it is known (see Appendix A) that there are no semisimple eigenvalues that are not simple. In this case Theorem 4.3.2 assumes the following form, which is very useful in numerical analysis. Theorem 4.3.3. If A is a companion matrix, the solution y = 0 is stable iff the eigenvalues ofA have modulus less or equal to 1 and those of modulus 1 are simple.

4.4.

93

Linear Equations with Periodic Coefficients

Example.

The trivial equation (4.3.4)

where I is the s x s unit matrix is an example where the matrix has a multiple eigenvalue that is 1, but it is semisimple and the zero solution is stable. Let us now consider the nonhomogeneous equation Yn+l = AYn

+ b,

(4.3.5)

where A is an s x s nonnegative matrix and b a nonnegative vector. The critical point y is given by the solution of the equation y = Ay+ b.

(4.3.6)

For such difference equations, there is a relation between the existence of nonnegative solutions of (4.3.6) and the stability behavior. For the notion used in the next two theorems see Appendix A.

°

If A ;;::: 0, b ;;::: and p(A) < 1, where p(A) is the spectral radius of A, then (4.3.6) has a nonnegative solution.

Theorem 4.3.4.

2::

Proof. Since p(A) < 1, (I - A)-l exists and is given by (I - A)-l = 0 A' Aib, which is nonnegative. By assumption from which we obtain y = 0 on p(A), we also see that y is asymptotically stable •

2::

Under stronger assumptions on b, there is a converse of the above theorem. Suppose that b is positive and A;;::: 0, then, if (4.3.6) has a positive solution y, we have p(A) < 1.

Theorem 4.3.5.

Proof. By the Perron-Frobenius Theorem the matrix AThas a real eigenvalue equal to p(A) to which corresponds a nonnegative eigenvector Uo such that A TUO= p(A)uo. Multiplying the transpose of the relation (4.3.6) by Uo, we get [1 - p(A)]yTuo = bTuo from which, since both yTuo and bTu o are positive, one gets p(A) < 1. •

The foregoing results have important applications in the study of iterative methods for linear systems of equations.

4.4.

Linear Equations with Periodic Coefficients

The results obtained in the previous section cannot be extended to nonautonomous equations, as the following example shows. Consider the equation Yn+l = A(n)Yn,

(4.4.1)

94

4. Stability Theory

where A(n)

=.!.( 0" 8 9-(-1) 7

9 + (-1)"7). 0

(4.4.2)

The eigenvalues of A(n) are ±T 1/ 2 for all n, and they are inside the unit disk, but this is not enough to ensure even the stability of the null solution. With no = 0, the fundamental matrix is

T

2

(n,O) = ( 0

"

0)

2"'

(4.4.3)

2") o '

(4.4.4)

if n is even, and

if n is odd. In any case this is a solution that will grow exponentially away from the origin. Consequently there must be an additional condition on A( n) in order to get stability. There is an intermediate case, however, that we can treat, namely, linear equations with periodic matrix A(n). The equation (3.4.11) shows that the central role played by the matrix U == (N, 0) in such cases. Any solution of the equation will have the form (4.4.5) for 0 ~ n ~ N. The behavior of the solution will then be dictated by the behavior of Ujyo. This leads to the following theorem, which is analogous to Theorem 4.2.1. Theorem4.4.1. ThezerosolutionoftheequationY"+1 = A(n)y", whereA(n) is periodic ofperiod N, is asymptotically stable if the eigenvalues of the matrix U are inside the unit disk. When some semisimple eigenvalues are on the boundary of the unit disk, then the solution is stable. In other cases there is instability. The similarity of the results in the two cases of autonomous and periodic equations suggests a more intimate connection between them. This is really the case as the following theorem shows. Theorem 4.4.2. IfA( i) is, for all i nonsingular and periodic, then it is possible to transform the periodic system into an autonomous one.

4.5.

95

Use of the Comparison Principle

Proof.

By hypothesis, the matrix U

== (N, 0)

N-I

=

I1

;=0

(4.4.6)

A(i)

is nonsingular. It is possible to define (see Appendix A) the matrix C such that (4.4.7) P(n) = (n, O)C- n is periodic because P(n + N) = (n + N, O)C-nC- N = (n, O)(N; O)C-nC- N = P(n). Using this matrix

The

matrix

to define the new variable

we

have

P(n

+ l)xn + 1 =

A(n)P(n)xn from

which

we get

Xn+l

=

P~lIA(n)P(n)xn' Simple manipulations reduce this to Xn+1

which proves the theorem.

=

CXn ,

(4.4.9)

•

The solutions having as initial values the eigenvectors of C (or U) have the property

x,

= J.t n v,

where v is an eigenvector and J.t the corresponding eigenvalue. But J.t = P 1/ N, where p is the eigenvalue of U. The corresponding solution of the original equation is Yn

= (n, O)v,

(4.4.10)

and this agrees with what was stated in Chapter 3. The solutions in the form of (4.4.10) are said to be Floquet solutions in analogy with the terminology used in the continuous case.

4.5.

Use of the Comparison Principle

Most of the results on the qualitative behavior of the solutions of difference equations can be obtained using the comparison theorems stated in Section 1.8. This theory is parallel to the corresponding theory of differential equations.

96

4. Stability Theory

Theorem 4.5.1. Let g(n, u) be a nonnegative function nondecreasing in u. Suppose that (1) f: N;o X s, ~ e; P > 0, (2) f(n, 0) = 0, g(n,O) = 0, and (3) IIf(n,y)1I ~ g(n, IIyll)· Then the stability of the trivial solution of the equation

(4.5.1) implies the stability of the trivial solution of (4.1.1).

Proof.

From (4.1.1) we have

and hence the comparison equation is (4.5.1). Theorem 1.6.1 can be applied, provided that IIYol1 ~ uo, to get IIYnl1 ~ u; for n ~ no. Suppose now that the zero solution of (4.5.1) is stable. Then for e > 0, there exists a 5 (s, no) such that for luol < 5 we have lunl < e. This means the stability of the trivial solution of (4.1.1). Since g ~ 0, the trivial solution of the comparison equation (4.5.1) may not be, for example, asymptotically stable and therefore we cannot conclude from Theorem 4.5.1 that the trivial solution of (4.1.1) is also asymptotically stable. This is due to the strong assumption (3), which can be replaced by the following condition

Ilf(n, y)11

(4)

~

Ilyll + w(n, Ilyll),

where g(n, u) = u + w(n, u) is nondecreasing in u. Noting that w in (4) need not be positive and hence the trivial solution of U n+ l = g(n, un) can have different stability properties, we can conclude from Theorem 4.5.1 in case (3) is replaced by (4), that the stability properties of the trivial solution of (4.5.1) imply the corresponding stability properties of the trivial solution of (4.1.1). This version of Theorem 4.5.1 is more useful and we shall denote it by 4.5.1 *. Of course, the proof needs minor modifications. • From Theorem 4.5.1 and 4.5.1 *, we can easily obtain several important variants. Let <1>( n, no) be the fundamental matrix ofthe linear equation

Theorem 4.5.2.

x n+1 Let F:N;o

xR

S

~ R

S

II<1>-I(n

,

= A(n)xn.

(4.5.2)

F(n, 0) = 0 and

+ 1, no)F(n, (n, nO)Yn)II ~ g(n, llYn II),

(4.5.3)

4.5.

97

Use of the Comparison Principle

where the function g(n, u) is nondecreasing in u. Assume that the solutions Un of (4.5.4)

are bounded for n 2= no. Then the stability properties of the linear equation (4.5.2) imply the corresponding stability properties of the null solution of X n+1 =

Proof.

A(n)xn

The linear transformation x,

Yn+l

=

Yn

=

+ F(n, x.),

(4.5.5)

«I>(n, no)Yn reduces (4.5.5) to

+ «I>-l(n + 1, no)F(n, «I>(n, no)Yn).

We then have (4.5.6) IIYn+lll ~ llYn I + g(n, llYn II)· If IIYol1 s Uo we obtain llYn II S Un, where Un is the solution of Un+l = Un + g(n, Un). It then follows that IIxnll ~ «I>(n, no)II llYn II s II«1>(n, no)II Un'

If the solution of the linear system is, for example, uniformly asymptotically stable, then from Theorem 4.2.2 we see that 1I«I>(n, no)11 S a:1]n-n o for some suitable a: > 0 and 0 < 1] < 1. Then and this shows that the solution x = 0 is uniformly asymptotically stable • because Un is bounded. The proof of other cases is similar. We shall merely state another important variant of Theorem 4.5.1 * which is widely used in numerical analysis. Theorem 4.5.3.

Given the difference equation Yn+l

= Yn + hA(n)Yn + f(n, Yn),

(4.5.7)

where h is a positive constant, suppose that (1) f(n,O) = 0 for n (2) Ilf(n, y)1I S g(n,

2=

no, and with g(n, u) nondecreasing in u, g(n, 0)

Ilyll)

= O.

Then the stability properties of the null solution of (4.5.8)

imply the corresponding stability properties of the null solution of (4.5.7). In this form the foregoing theorem is used in the estimation of the growth of errors in numerical methods for differential equations. Instead of (4.5.8), one uses usually the comparison equation (4.5.9)

4.

98

Stability Theory

which is less useful because (1 + hiIAII) > 1. The form (4.5.8) is more interesting because when the eigenvalues of A have all negative real parts, I 1+ hA I can be less than 1. This will happen, for example, if the logarithmic norm: JL(A(n)) = lim

III + hA(n)ll- 1 h

h-O

is less than zero. From the definition it follows that

III + hAil = 1 + hJL(A(n)) + O(h 2A(n)). Letting the comparison equation becomes U n:r1 =

(1 + ,1)u n

+ g(n,

un).

(4.5.10)

The next theorem requires essentially a condition on the variation of 1, but it does not require the a-priori knowledge of the existence of the critical point. Let us consider (4.5.11)

Theorem 4.5.4. Suppose f: Dc R S ~ R S is continuous and g is a positive function nondecreasing with respect to its arguments, defined on J I x J2 X J3 where I, are subsets of R+ containing the origin. Further suppose that for X o E D, the sequence x, is contained in D, (4.5.12) and the comparison equation U n+1 =

g(

n

Un,

n-1)

j~O Uj, j~O u j

(4.5.13)

has an exponentially stable fixed point at the origin. Then (4.5.11) has a fixed point that is asymptotically stable.

Let us say

Proof.

j

j

Ilxn +! - x, II. Then we have Ilxn + 1 Since g is nondecreasing, it follows that

Yn =

I;=o Ilx + 1 - x I :s: I;=o Yj'

n

Yn+1 :S:

g(

By Theorem 1.6.6 we then obtain

Yn,

n-I)

j~O Yj, j~O Yj

•

xoll :s:

4.6.

99

Variation of Constants

where Un is the solution of (4.5.13), provided that Yo S U00 If the origin is exponentially stable for (4.5.13), it follows that for suitable Uo the sequence Un will tend to zero and the same will happen to Yn = [[x n+ 1 - x, II. Moreover for all p ;;:: 0 Ilxn+ p

-

x, II s

p

I

Yn+j'

j~1

Now by exponential stability of the origin of (4.5.13) and by Theorem 4.1.1 it follows that it is also II-stable. Then the series I Yj is convergent and then for suitable n, I%=I Yn+j can be taken arbitrarily small, showing that Xk is a Cauchy sequence. •

4.6.

Variation of Constants

Consider the equation Yn+1 = A(n)Yn

+ f(n,Yn),

Y""

where A( n) is an s x s nonsingular matrix and f: Theorem 4.6.1.

= Yo,

»; x R

(4.6.1) S

~ R

S

•

The solution y(n, no, Yo) of (4.6.1) satisfies the equation n-I

Yn

= (n, no)Y"" + I

j=no

(n,j + l)f(j, Yj),

(4.6.2)

where (n, no) is the fundamental matrix of the equation x n+ 1 = A(n)xno Proof.

(4.6.3)

Let y(n, no, Yo) = (n, no)xn,

x"" = Yo.

(4.6.4)

Then substituting in (4.6.1), we get (n

+ 1, no)x n+ 1 = A(n)(n, no)x n + f(n,Yn)

from which we see that

aXn = -l(n + 1, no)f(n, Yn), and n-I

x,

=

I

j=no

(no,j + 1)f(j, Yj) + x"".

•

From (4.6.4), it follows that I

y(n, no, Yo)

n-]

= (n, no)yo + I

j=no

(n,j + 1)f(j, Yj)'

(4.6.5)

100

4.

Stability Theory

Consider now the equation (4.6.6)

Lemma 4.6.2. Assume that f: N;o x R S ~ R S and f possesses partial derivatives on N;o x R S • Let the solution x(n) == x(n, no, xo) of (4.6.6) exist for n

~

no and let _ af(n, x(n, no, xo)) H ( n, no, X o) . ax

(4.6.7)

Then

(4.6.8) exists and is the solution of (n

+ 1, no, xo)

= H(n, no, xo)(n, no, x o),

(4.6.10)

(no, no, x o) = 1.

Proof.

(4.6.9)

By differentiating (4.6.6) with respect to X o we have aXn + 1 = afaxn axo aXn axo

Then (4.6.9) follows from the definition of <1>. We are now able to generalize Theorem 4.6.1 to the equation Yn+l = f(n, Yn) + F(n,

yJ.

•

(4.6.11)

Theorem 4.6.2. Let f, F: N;o x R S ~ R S and let af/axexistand be continuous and invertible on N;o x R S • If x(n, no, x o) is the solution of X no

(4.6.12)

= Xo,

then any solution of (4.6.11) satisfies the equation y(n, no, x o) = where

x( n, no, Xo + j~~ l/J-l(j + 1, no, vj, vj+ )F (j, Yj)) 1

(4.6.13)

4.6.

Variation of Constants

101

and vj satisfies the implicit equation (4.6.14). Proof.

yen

Let us put yen, no, xo)

•

= x(n, no, vn) and

Vo

= Xo'

Then

+ 1, no, xo) = x(n + 1, no, Vn+l) - x(n + 1, no, vn) + x(n + 1, no, vn) = fen, x(n, no, vn + F(n, x(n, no, vn

»

»

from which we get

x(n

+ 1, no, Vn+l) - x(n + 1, no, vn) = F(n, Yn).

Applying the mean value theorem we have

f f

l aX(n + l , no, sVn+I + (1 - S)Vn) d ( -)- ( ) S Vn+1 Vn - F n, Yn o axo .

and hence by (4.6.8)

cI>(n

+ 1, no, SVn+1 + (1- s)vn) ds(v n+1 -

vn) = F(n, Yn),

which is equivalent to

l/J(n

+ 1, no, Vn, Vn+I)(Vn+1 - vn) = F(n, Yn).

(4.6.14)

It now follows that

Vn+1 - Vn = l/J-I(n + 1, no, Vn, vn+t)F(n, Yn) and n-I

e,

= Vo + L

l/J-I(j + 1, no, vh vj+I)F(j, Yj)

(4.6.15)

j=no

from which the conclusion results.

•

Corollary 4.6.1. Under the hypothesis of Theorem 4.6.2, the solution y( n, no, xo) can be written in the following form.

yen, no, xo) = x(n, no, x o) + l/J(n, no, Vn, xo) n-I

.L

l/J-l(j

+ 1, no, Vj,

vj+I)F{j,

v)

(4.6.16)

j=no

Proof.

Apply the mean value theorem once more to (4.6.13).

Corollary 4.6.2. Proof.

Iff(n,x)

In this case Xn =

= A(n)x,

cI>~n,

•

then (4.6.16) reduces to (4.6.2).

no)xo and

<1>(n, no, xo) == l/J(n, no) == l/J(n, no, Vn, Vn+I),

102

4.

Stability Theory

and therefore, we have Vn+ 1 - Vn = l/J-l(n + 1, no, Vn, vn+l)F(n, Yn) and Vn = Xo + L:;':-~o l/J -l(j + i, no, Vj, vj+l)F(j, Yj) from which follows the claim. •

4.7.

Stability by First Approximation

Consider the equation Yn+l = A(n)Yn

+ f(n, Yn),

(4.7.1)

where y E R A(n) is an s x s matrix, f: N no x B a ~ B, and f(n, 0) = O. When f is small in the sense to be specified, one can consider (4.7.1) as a perturbation of the equation S

,

X n+ 1

(4.7.2)

A(n)xn

=

and the question arises whether the properties of stability of (4.7.2) are preserved for (4.7.1). The following theorems offer an answer to such a question. Theorem 4.7.1.

Assume that (4.7.3)

where gn are positive and L:~=no gn < 00. Then if the zero solution of (4.7.2) is uniformly stable (or uniformly asymptotically stable), then the zero solution of (4.7.1) is uniformly stable (or uniformly asymptotically stable). Proof.

By (4.6.2) we get Yn

=

(n, no)Yo +

n-l

I

j=O

(n,j + 1)f(j, Yj)'

Because of Theorem 4.2.1 we have, using (4.7.3), llYn II ~ MllYol1

+M

n-I

I

j=no

gJyJ.

Corollary 1.6.2 yields

from which follows the proof, provided that Xo is small enough such that MllYol1 exp(ML:;:nogJ < a.

4.7.

103

Stability by First Approximation

In the case of uniform asymptotic stability, it follows that for n > N,

1I(n, no)Yoll < e, for every e > 0, the previous inequality can be written

llYn II :5 e exp ( M ~ gj) from which we conclude that lim Yn

=

0.

•

Corollary 4.7.1. If the matrix A is constant such that the solutions of (4.7.2) are bounded, then the solutions of the equation

Yn+1

= (A + B(n»Yn

(4.7.4)

are bounded provided that co

I IIB(n)11 < 00. n=»o Theorem 4.7.2.

(4.7.5)

Assume that

°

° °

(4.7.6)

where L> sufficiently small, and the solution x, = of (4.7.2) is uniformly asymptotically stable. Then the solution Yn = of (4.7.1) is exponentially asymptotically stable.

Proof.

By using the result of Theorem 4.2.2, we have

I (n, no)11 <

HT/n-no,

0< TJ < 1,

H>O,

and because of (4.7.6), we get llYn I

:5

HT/n-noilYoll + LHTJ n- 1

By introducing the new variable P«

=

Pn:5 HTJ-noilYoil

n-I

I

TJ -jllYjll·

j=no

TJ -n llYn II, we see that n-I

+ LHTJ-I I

Pj'

j=no

Using Corollary 1.6.2 again, we arrive at Pn

:5

HTJ-noilYoil

n-I

I1

(I

+ LHT/-I)

j=no

which implies (4.7.8)

104

If T/

4. Stability Theory

+ LH <

1, that is L <

Corollary 4.7.2. (Perron).

1H

T/

the conclusion follows.

•

Consider the equation Yn+l

= Ay" + fen, Yn),

(4.7.9)

where A has all the eigenvalues inside the unit disk and moreover

lim Ilf(n,y)1I

Ilyll

y .. O

=0

(4.7.10)

uniformly with respect to n, then the zero solution of (4.7.3) is exponentially asymptotically stable. We can similarly prove the following result. Theorem 4.7.3.

Assume that the zero solution of

where A is an s x s matrix, is asymptotically stable. If 00

L

n=no

IIB(n)11 < +00

then the zero solution of (4.7.4) is asymptotically stable. Next theorem concerns instability of the null solution. We shall merely state such a result for completeness leaving its proof as an exercise. Theorem 4.7.4.

· 1im

"->00

Assume that the zero solution ofxn + 1 = AX n is unstable and

Ilf(n, . . .IS unsta . ble fior (471) I Yn)11 I =.0 171en the ongtn .. . Yn

4.8. Lyapunov Functions The most powerful method for studying the stability properties of a critical point is the Lyapunov's second method. It consists in the use of an auxiliary function, which generalizes the role of the energy in mechanical systems. For differential systems the method has been used since 1892, while for difference equations its use is much more recent. In order to characterize such auxiliary functions, we need to introduce a special class of functions.

4.8. Lyapunov Functions

105

Definition 4.8.1. A function 4> is said to be of class K if it is continuous in [0, a), strictly increasing and 4>(0) = o. It is easy to check that the product of any two functions of class K is in the same class and the inverse of such a function is in the same class. Let V(n, x) be a function defined on /"" x Ba , which assumes values in R+.

Definition 4.8.2. The function V(n, x) is positive definite (or negative. definite) if there exists a function 4> E K such that

4>(llxll) ~ for all (n, x)

E N~o

V(n, x)

(or V(n, x)

2:

-4>(llxll»

x Ba •

Definition 4.8.3. A function V(n, x) exists 4> E K such that

2:

0 is said to be decrescent if there

V(n, x)

~

4>(x)

for all (n, x)

E N~o

x Ba .

Let us consider the equation (4.8.1)

Yn+l = f(n, Yn),

where f: N~o x B; ~ R f(n,O) = 0 and f(n, x) is continuous in x. Let y(n, no, Yo) be the solution of (4.8.1), having (no, Yo) as initial condition, which is defined for n E N;o' We shall now consider the variation of the function V along the solutions of (4.8.1) S

,

A V(n, Yn)

= V(n + 1, Yn+l) -

V(n, Yn)'

(4.8.2)

If there is a function w : N~o x R ~ R such that A V(n, Yn) ~ w(n, V(n, Yn),

then we shall consider the inequality V(n

+ 1, Yn+l) ~

V(n, Yn) + w(n, V(n, Yn)

== g(n, V(n, Yn»

(4.8.3)

to which we shall associate the comparison equation U n+1

= g(n, un) ==

Un

+ w(n, Un)'

(4.8.4)

The auxiliary functions V(n, x) are called Lyapunov functions. In the following, we shall always assume that such functions are continuous with respect to the second argument.

106

4.

Stability Theory

Theorem 4.8.1. Suppose there exists twofunction V(n, x) and g(n, u) satisfying the following conditions: (1) g: N;o x R+ ~ R, g(n, 0) = 0, g(n, u) is nondecreasing in u; (2) V:N;oxBa~R+, V(n, 0) =0, and V(n, x) is positive definite and continuous with respect to the second argument; (3) V satisfies (4.8.3).

Then (a) the stability of u = 0 for (4.8.4) implies the stability of Yn = 0; (b) the asymptotic stability of u = 0 implies the asymptotic stability of Yn = O. Proof.

By Theorem 1.6.1, we know that

provided that V(n o, Yo) we obtain for ¢ E K,

:5

Uo' From the hypothesis of positive definiteness

If the zero solution of the comparison equation is stable, we get that Un < ¢ (e) provided that Uo < T/ (s, no), which implies

from which we get

llYn II <

(4.8.5)

e.

By, using the hypothesis of continuity of V with respect to the second argument, it will be possible to find a 8 (e, no) such that llYn"" < 8 (s, no) will imply V( no, Yno) :5 Uo· In the case of asymptotic stability, from

we get lim ¢(IIYnll) n-e-co

= 0 and consequently n_OO lim Yn = O.

•

Corollary 4.8.1. If there exists a positive definite function V(n, x) such that on N;o x Ba , V is continuous with respect to x and moreover, .Il Vn :5 0, then the zero solution of (4.8.1) is stable.

4.8.

Lyapunov Functions

107

Proof. In this case w(n, u) == 0 and the comparison equation reduces to Un+1 = Un, which has stable zero solution. •

Theorem 4.8.2. Assume that there exists two functions V( n, x) and w (n, u) satisfying conditions (1), (2), (3) ofthe previous theorem and moreover suppose that V is decrescent. Then (a) uniform stability of u = 0 implies uniform stability ofYn = O. (b) uniform asymptotic stability ofu = 0 implies uniform asymptotic stability of y; = O. Proof. The proof proceeds as in the previous case except that we need to show that 8(e, no) can be chosen independent on no. This can be done by using the hypothesis that V(n, x) is decrescent, because in this case there exists a Jot E K such that V(n, Yn) :5 Jot (llYn II). In fact, as before, we have

1>(IIYnll):5 V(n,Yn):5 u; provided that V(no, Yn,,) :5 uo:5 7](e). If we take Jot (IIY""II) = I is Ily""II < 8(e) == Jot- ( 1]( e) ), then llYn I < e for all n <:= no· The uniform asymptotic stability will follow similarly.

Corollary 4.8.2. such that

Uo:5 1] (e), that

•

If there exists a positive definite and decrescent function V A V(n, Yn) :5 0,

then Yn = 0 is uniformly stable.

Corollary 4.8.3.

If there exists a function V such that

and where 1>, Jot, II E K and the function w == llJot - I is a Lipshitz function with constant less than one then Yn = 0 is uniformly asymptotically stable. Proof.

Clearly llYn I <:= Jot -I( V(n, Yn)) and by substitution we have A V(n, Yn):5 - Il(Jot -I( V(n, Yn)))

== -w( V(n, Yn)).

The comparison equation is now (4.8.6)

108

4.

Stability Theory

where the righthand side is positive and nondecreasing, because of the hypothesis. The same hypothesis shows that the origin is asymptotically stable for (4.8.6) (see Problem 4.11) and because the equation is autonomous, it follows that the stability is also uniform. With minor changes, one can show that the condition on .:lV can be substituted by .:lV(n, Yn) :S -v(IIYn+1II). If in Theorem 4.8.2 the condition that V is decrescent is removed, the asymptotic stability will remain. • Theorem 4.8.3.

Assume that there exists a function V such that

(1) V:N~o?< B a -7 R+; V(n, 0) = 0; V is positive definite and continuous with respect to the second argument; (2) .:lV(n, Yn) :S -,u(IIYn 11),,u E K. Then the origin is asymptotically stable.

Proof. By Theorem 4.8.1 we know that the origin is stable. Suppose it is not asymptotically stable, then there exists a solution y(n, no, Yo) and a set Jno c N~o such that lIy(n, no, Yo)11 > e > O. We then have

+ l,y(n + 1, no,Yo»:S V(n,Yn) - ,u(e), Jno or, since V is decreasing, V(n + 1, Yn+l) :s V(n, Yn). Summing we V(n

if n E have

V(n

+ 1, y(n + 1, no, Yo» :s

V(no, Yno) - ke,

where k is the number of elements in Jno . Taking the limit we get lim V(n, Yn) = -00, n->oo

which contradicts the hypothesis that V is positive definite. • The next theorem concerns the lp stability and is related to the previous one. Theorem 4.8.4.

Assume that there exists a function V such that

(1) V: N~o x B,

-7 R+, V(n, 0) = 0, positive definite and continuous with respect to the second argument; (2) .:lV(n, Yn) :S -ellYn liP,

where p, c are positive constant. Then Yn = 0 is lp stable.

Proof. By Theorem 4.8.1 we know that Yn = 0 is stable, that means that for n 2:: no, e 2:: 0, there exists 8( s, no) such that for lIyno ll < 8, llYn I < e. Let us define the function G(n) = V(n, Yn) + c L;:~o IlyJP. Then .:lG(n) =.:l V(n,Yn)

+ clIYnIIP:s O.

4.8.

109

Lyapunov Functions

Therefore G(n)

~

G(no) = V(no, Y no) for n ;;::: no and

o ~ G(n) =

yen, Yn)

+c

n-)

I

j=no

llYn liP ~ V(no, Yno)

from which it follows

and 00

I

j=no

llYn liP <

00.

•

The next theorem is a generalization of the previous one. It is the discrete analog of La Salle's invariance principle. Let us consider the solution yen, no, Y no) of Yn+) = fen, Yn),

(4.8.7)

Yno = Yo,

and suppose that it is continuous with respect to the initial vector Yo. Theorem 4.8.5.

Suppose that, for Y E D

c

R S,

(1) there exist two real valuedfunctions yen, y), lUcy) ;;::: 0 both continuous in y, with V( n, y) bounded below such that

(4.8.8)

11 yen, Yn).~ -lU(Yn) (2) yen, no, Y no)

E

D for n ;;::: no·

Then either yen, no, Yno) is unbounded or it approaches the set B

= {x E

15llU(X)

= O}.

(4.8.9)

Proof. By assumption Yn E D for n ;;::: no and yen, Yn) is decreasing along it. Because V is bounded below, it must approach a limit for n ~ 00 and lU(Yn) must approach zero. Then either the limit is finite and must lie in B, or it is infinite. •

Corollary 4.8.4. Suppose that u(x) and vex) are continuous real valued functions such that u(x) ~ yen, x) ~ vex)

(4.8.10)

I

for n e: no. Fixing T/, consider the set D( T/) = {x u(x) < T/}, D) (T/) = {xlv(x) < T/}. Under the hypothesis of Theorem 4.8.5 with D = D(T/), all solutions that start in D) (T/) remain in D (T/) and approach B for n ~ 00.

110

Proof.

4. Stability Theory

Let Yo

E

D 1( TJ), then

u(Yn)

:0;

V(n, Yn):O; V(no, Yo)

showing that u(Yn) < TJ for n Example.

~

no.

:0;

v(Yo) < TJ

•

Consider the equation Yn+l = M(n, Yn)Yn,

(4.8.11)

where M is an s x s matrix. Define V(n, Yn) as (4.8.12)

then L1 V = IIM(n, Yn)Yn II -llYn /I

:0;

(1IM(n, Yn)11 - 1llYn II·

Let u(Yn) == v(Yn) == V(n,Yn). Then D(TJ) == D1(TJ) = {ylllyll < TJ}. For all Y ED, let IIM(n,y)/I < a(y) and w(y) = (1- a(y)/lyll. It then follows that L1 v s -w(y).

If a(y) < 1, for Y E D(TJ), w(y) is positive. The set E is the origin and possibly something on the boundary of D( TJ). Because V is decreasing on D( TJ), it follows that this last possibility cannot occur. Then the solution starting in D( TJ) cannot leave this set and it will tend to the origin. D( TJ) is a domain of asymptotic stability of the origin. Different choices of TJ and of the norm used will give different domains of asymptotic stability. Of course, the union ofall these domains is still a domain of asymptotic stability. If M is independent on n with spectral radius less than one, then it is possible to choose a vector norm such that 12(0) < 1 with a(x) continuous. The previous result shows that it is possible to define a nonempty domain of asymptotic stability. Definition 4.8.4. The positive limit set O(y",,) of a sequence Yn, n E N;o is the set of all the limit points of the sequence. That is, Y E O(y",,) if there exists an unbounded subset J"" c N;o such that Ynj ~ Y for nj E J"". Definition 4.8.5. yen, no, Yo) E S.

A set S c R is said invariant if for Yo E S follows that S

In the case of autonomous difference equation Yn+l = f(Yn),

(4.8.13)

where f is a continuous vector valued function withf(O) = O. Theorem 4.8.5 assumes the following form.

4.9.

111

Domain of Asymptotic Stability

Theorem 4.8.6.

Suppose that for Y

E

D

c R

S ,

(1) there exist two real valued functions V(y), w (y) both continuous in Y, with V bounded below, and ~ V(Yn)

:::; -w(Yn);

(2) Yn E D for n ~ no; then either Yn is unbounded or approaches the maximum invariant set M contained in E. Proof. Same as before and moreover, the positive limit set of any bounded solution of an autonomous equation is nonempty, invariant and compact (see problem 4.12). •

Corollary 4.8.5 can be rewritten as follows. Corollary 4.8.6.

If in Theorem 4.8.6 the set D is of the form

D( 1]) = {x] V(x) < 1]} for some 1] > 0, then all the solutions that start in D ( 1]) remain in it and approach M as n -'; 00.

The next result, again for autonomous equations, imposes conditions on the second differences of the Lyapunov function.

R is a continuous function with Theorem 4.8.7. Suppose that V: R S ~ 2 V(Yn) > 0 for Yn ¥ O. Then for any Yo E R , either y( n, Yo) is unbounded 2 or it tends to zero for n -'; 00. Likewise if ~ V(Yn) < 0, Yn ¥ O. S

-';

Proof.

Put ~2V > 0 in the form V(Yn+2) - V(Yn+l) > V(Yn+l) - V(Yn).

If there exists a k E N;o such that V(Yk+l) - V(Yk) ~ 0, then V(Yn+l)V(Yn) > 0 for n > k, otherwise we get V(Yn+l) < V(Yn) for all n. In both cases V is a monotone function. Suppose that V is not increasing. Consider the positive limit set n(yo) of y(n, no, Yo). If n(yo) is empty then y(n, no, Yo) is unbounded and the theorem is proved. If n(yo) is not empty, V(Yn) must be constant on n(yo) because the limit of a monotone function is unique. But this is impossible because ~2 V> 0, unless n(yo) = {O}. The other cases are proved similarly. •

4.9.

Domain of Asymptotic Stability

All the previous results say that if the initial value is small enough, then the origin has some kind of stability. In applications one is more interested

112

4. Stability Theory

in the domain of asymptotic stability since one needs to know where to start with the iterations. In other words, one needs to know the domain in R S, containing the initial values, starting from which solutions will eventually tend to the fixed point. This problem is a difficult one and will also be discussed in the next chapter. We shall give some results in this direction in the case of autonomous difference equations Yn+l = f(Yn),

where f

E

C[Ba , R

S ] ,

and f(O) =

(4.9.1)

o.

Theorem 4.9.1. Suppose that there exists a continuous function V: Ba ~ R+, V(O) == 0 and il V(Yn) < O. Then the origin is asymptotically stable. Moreover if B a == R S and V(y) ~ 00 as y ~ 00, then the origin is globally asymptotically stable. Proof. The proof of stability follows from the same arguments used in Theorem and Corollary 4.8.1, except that continuity of V is used instead of positive definiteness of V. For proving asymptotic stability, we observe that for Yn E Bel where o< e < CI', V(Yn) is strictly decreasing and it must converge to zero. Again by continuity of V, the sequence Yn itself must converge to zero. Now suppose that the last hypothesis is true. Then it is clear that we are not restricted to take llYn II < CI' and hence the proof is complete. •

As an example, consider (4.9.1) when f is linear,

(4.9.2) and let us take

(4.9.3) where B is a symmetric positive definite matrix. The demand that il V(Yn) < 0 becomes

that is, we must have

(4.9.4) where C is any positive definite matrix. This leads to the following result. Corollary 4.9.1. If there is a symmetric positive definite matrix B such that .(4.9.4) is verified, then the origin is asymptotically stable.

4.9.

Domain of Asymptotic Stability

113

The converse of this corollary is also true. Theorem 4.9.2. Suppose (4.9.2) is asymptotically stable, then there exists two positive definite matrices E and C such that (4.9.4) is verified.

The statement analogous to (4.9.4) in the continuous case is STO

+ as =

-at.

(4.9.5)

where a and 0 1 are symmetric positive definite matrices, and S has the eigenvalues with negative real part. Equation (4.9.5) is called Lyapunov matrix equation. There is, of course, a correspondence between (4.9.4) and (4.9.5). In fact by putting S = (A + I)(A - I)-\ (4.9.5) is transformed into an equation of the form (4.9.4) and vice versa by putting A = (1 - S)-I(1 + S). To find the matrix E, one needs to solve the matrix equation (4.9.4) where A is given and C is chosen appropriately. The methods of solution of the equation (4.9.4) (or alternatively 4.9.5) have been studied extensively in the last ten years. The following theorem also gives the region of the asymptotic stability for the zero solution of (4.9.1) and it is the discrete version of the Zubov's theorem. Theorem 4.9.3. Assume that there exists two functions V and ¢> satisfying the following conditions:

(1) (2) (3) Then

Proof.

V: C[R R+], V(O) = 0, . V(x) > 0 for x s« 0, S ¢>: C[R , R+], ¢>(O) = 0, ¢>(x) > 0 for x,e. 0, V(Yn+l) = (l + ¢>(Yn)) V(Yn) - ¢>(Yn). D = {x I V (x) < I} is the domain of asymptotic stability. S

,

By condition(3), which can be written as 1 - V(Yn+l) = (l

+ ¢>(Yn))(l - V(Yn)),

one obtains 1 - V(Yn)

n-l

= I1

(l

+ ¢>(Yj))(1 - V(Yo))·

(4.9.6)

j~O

Suppose now that Yo lies in the region of asymptotic stability. Then, the left-hand side tends to one by hypothesis, the right-hand side will also be

= n (1 + ¢>(Yj) 00

convergent and will tend to C(1 - V(Yo)), where C

j~O

> 1.

= (C - 1)/ C < 1, which shows that Yo ED. Conversely if Yo ED, then again from (4.9.6) we see that V(Yn) will remain

It follows then that V(yo)

114

4.

Stability Theory

always outside D and V will never be zero. This implies that Yn will not tend to zero. • To use this theorem, one needs to know the solution Yj of (4.9.1), and then from (4.9.6) find the function V, which will define the set D. Unfortunately this can be done only for some cases. Theorems 4.8.5 and 4.8.7 and their corollaries also give the asymptotic stability domains in terms of D( 1]) for the set E. When the maximum invariant set contained in E, reduces to a point, Theorem 4.8.7 gives the asymptotic stability domain for a critical point. Theorem 4.8.7 can also be used to obtain the domain of asymptotic stability. Suppose that the condition d 2 V> 0 holds true only in an open region H containing the origin. Then put d max

= max{d V(x) [x E boundary H}

and Ej

for j

=

{x

E

Hid V(y(j, x)) > d max}

= 0, 1,2, ... , where y(j, x) == y(j, no, x).

Theorem 4.9.4. If the regions Ej are bounded and nonempty, then they are domains of asymptotic stability for (4.9.1). Proof. If E, is not empty and x E Ej , y(j + k, x) decreasing along any trajectory in Ej • Moreover d V(y(j

+ k + 1, x)) - d V(y(j + k; x))

= d

E

H, since d V is not

2V(y(j

+ k, x)) > 0

and d V(y(j + k; x)) > d V( y(j, x)) <:= d max. This means that y(j + k, x) E Ej for all k; from which we see that y(n, x) is bounded. By Theorem 4.8.7 it follows that y(n, x) ~ O. • Similar results hold in the case where d 2 V < 0 in H and the regions

Fj

= {x E

d min

Hid V(y(j, x))} < d min.

= min{d V(x) [x E boundary H}.

As an example, we consider the following system arising in biomathematics 2 Ia I <3 -12. Yn+l=axn-Yn,

4.10.

Converse Theorems

Consider Vex, y) =

115

2a 2x 2

--2

l+a

+ y 2 • One obtains

2

~2V(X, y) =

y --2 (l -

l+a

3a 2 + 4ax - 2y 2) + (ay - (ax _

/?)2.

Here H = {(x, y) 11 - 3a 2 + 4ax - 2y 2 > O} contains the origin. For a one shows that ~ max = -0.0364, and then

{

E o = (x y) ,

4.10.

2

= 1/4,

y

X)2 2x - 15 > 2 -00364 } 17 17 . . I(y2 __4 - -

Converse Theorems

This section will be devoted to the construction of Lyapunov functions when certain stability properties hold. In this construction of Lyapunov functions, however, one uses the solutions of the problem and this implies that this construction is of little use in practice. The real importance of converse theorems lies therefore in the fact that by means of such theorems, it is possible to prove some results on total stability, which is a very important concept in applications. For this reason we shall present only those results that we shall use later. Suppose that the zero solution of (4.8.1) is uniformly stable, then there exists a function V, positive definite and decrescent, such that ~ V ~ 0 along the solutions.

Theorem 4.10.1.

Proof.

Consider the function yen, Yn) = sup k2:.n

Ily(k, n, Yn)ll·

(4.10.1)

As usual, Yn ;;;; yen, no, Yo). From (4.10.1) it is immediate that yen, Yn) ~

llYn"

showing that V is positive definite. From the definition of uniform stability we know thatlly(k,n,Yn)ll<e if llYn I <8(e). Without loss of generality, we shall suppose that 8 E K. (See problems 4.8 and 4.9.) Let ¢ = 8- 1• Then we can write Ily(k, n,Yn)11 ~ ¢(IIYnll)' from which we obtain Yen, Yn) ~ ¢(IIYnID, showing that V is decrescent. On the other hand, for every solution one has sup Ily(k, n,y(n, no,Yo»11 = sup Ily(k, no,y<»11 k~n

k~n

116

4. Stability Theory

and

V(n + 1,Yn+l) = sup lIy(k, no, Yo) II

$:

k2:n+l

sup Ily(k, no, Yo) II k ez n

= V(n, Yn) and this completes the proof.

•

Theorem 4.10.2. Suppose that the zero solution of (4.8.1), where f(n, x) is locally Lipschitzian around the origin, is uniformly asymptotically stable, then there exists afunction V, positive definite and decrescent, such that a V( n, Yn) < -JL(IIYn+ll1) along the solutions, and V is locally Lipschitzian.

r

G'(r) > 0, G"(r) > 0 and let a> 1. Since G(r)

G(;) = LIU du 1

fr

dco ao

f61

IU

G"(v) dv, we have,

=

1 fr dco ao

G"(v) dv < -

0

V(n,Yn) For k

Lr

Consider a function G(r) defined for r> 0, G(O) = 0, G'(O) = 0,

Proof.

=

du

G"(v) dv and

setting u = or]a,

f61 G"(v) dv = -1 G(r). a

0

G(;) =

Define

1 + ak = sup G(lly(n + k, n,Yn)II)--k. k2=O 1+

0, we have

The uniform stability ofthe origin implies that (see previous theorem) there exists a ¢J E k such that

Ily(n

+ k, n,Yn)11 < ¢J(IIYnll)

and therefore

G(lly(n

+ k, n, Yn)ll) <

G(¢J(IIYn II»·

. 1 +ak Consequently, since T+"'k < a, V(n, Yn) $: aG( ¢J llYn II). Furthermore,

asymptotic stability implies that for k e: Nt e], lIy(n + k, n,Yn)1I < e, and hence, we get Ily(n + k, n, Yn)11 <

k, n, Yn)ll) <

II~"

for k e: Ne~lI). Thus, G(lly(n +

G( "~ "), which in turn leads to G(lly(n + k, n,

Yn)ll) \ : :k $:

aG( II~ II) < G(IIYn II) $: V(n, Yn). This shows that it is.sufficient to consider

4.10. Converse Theorems

k

E

(0,

117

N( "~ II)) obtaining V(n,Yn)

=

G(lly(n

sup OsksN(IIYnllla)

1 + lea + k, n,Yn)ID--. 1+k

(4.10.2)

Let k 1 be the index where the sup is achieved so that

V(n,Yn)=G(lIy(n+k 1,n,Yn)ll)

1 + ak,

1+

k'

(4.10.3)

1

Then

V(n + 1, Yn+1)

=

1 + ak, G(lIy(n + 1 + ki> n + 1, Yn+l)II)----.:. 1 + k1

= G(lly(n + 1 + k«, n + 1, Yn+l)lJ) 1 - - -] x 1 + ex (k 1 + 1) [ 1 - - -ex-- 1 + (k 1 + 1) (l + k 1 )(1 + a + exk t )

~ v(n,Yn)[ 1- (l + kl)(~;~(kl + 1)]. from which V(n

+

1,y.+,} - V(n, Y.) "- [1 + N("y;.II)] [1 + a+ aN("Y;dl)] -1)G(IIYnID [ 1 + N(I'Y:+llI) 1 + ex + exNeY:+llI) ] (a

< -

-s

-s

(ex

J[

-1)G(~1l0 IIYn+lll)

[1 + N("Y:+1II) J[1+

ex + exN("Y::llI) ]

-JL(IIYn+lID,

where Lilo is the Lipschitz constant. The last inequality follows from I/Yn+111 Ilf(n,Yn)1I ~ LdYnll· The function f.L

E

K, because it is strictly increasing being

=

N( "~ II) a

decreasing function, and G(O) = O. To complete the proof, we must show that we can choose a function G such that V is Lipschitzian. By hypothesis

118

4.

Stability Theory

of uniform asymptotic stability it follows that for r> 0, there exists a lJ(r) such that for y', y"E B/;(r) and y(n, no, y'), y(n, no, y") E B r. Because f is Lipschitzian one has

We let q < 1,

and O(r)

=A

J:

(4.10.5)

qNOf,I/a ds,

(4.10.6)

where N(r) is the same function defined before. O(r) satisfies the condition required because N(r) is decreasing and lim N(r) = 00. r~O

We have seen that V(n,y') = O(lly(n

+ lc., n,y')II)

1 + ak, k' 1+ I

(4.10.7)

where (4.10.8) For simplicity, Ily(n + k«, n, y")II.

let us Suppose

r l = Ily(n + k.; n, y')11 and r2 = r2 ~ r l. Then 0 ~ O(rl) - 0(r2) ~ r2)' But r l - r2 ~ Ily(n + k«, n, y')-

put

O'(rl)(rl - r2) ~ A qN(6(r)/a)(r l + ki, n, y")11 ~ L~'lIy' - y"ll. By substituting, we have

y(n

o~

O(rl) - 0(r2) ~ A q N(6(r)/ a) . L~IIIy' -

y"11 ~ AllY' - y"II·

(4.10.9)

The last inequality follows from (4.10.8) and (4.10.5). Multiplying (4.10.9) 1 + ak l by k we get 1+

I

o ~ V(n, y') - O(IIy(n + k.; n, y")II)

1 + ak

1+

k

I

I

~ aAllY' - y"ll

from which, because of the fact V(n, y") > O(lIy(n + k«, n, y")ID we obtain V(n, y') - V(n, y") -s aAllY' y' and y" one gets similarly

y"II.

1 + ak, 1+

k' I

By interchanging the roles of

V(n, y") - V(n, y') ~ -aAllY' - y"ll,

4.11.

119

Total and Practical Stability

which shows that

IV(n, y') proving the theorem.

- V(n, Y")I ::; aAIIY' -

y"ll,

•

The following theorem is the converse in the case of lp-stability. Theorem 4.10.3. Suppose that the trivial solution of (4.10.1) is lp-stable and Ily(n, no, Yo) I ::; gn,B(IIYoll) with,B E K and L~=no s, = g. Then there exists a function V: N~o x B, ~ R+ such that it is positive definite, decrescent and ~ V(n, Yn) ::; -llYn liP.

Proof.

Let 00

V(n,Yn)

=

I

k=O

Ily(n

+ k, n,Yn)lIp •

It follows that

V(n, Yn) 2: llYn liP, which shows that V is positive ,BP(IIYnll) L~~no s; = g,BP(IIYnll) and 00

Ll V(n, Yn) =

I

k=O

Ily(n

+ 1 + k, n + 1, Yn+l)IIP

Ily(n

+ 1 + k, no, Yo)IIP

00

=I

k=O

=

Moreover, 00

I

-

k=O

00

-

I

k=O

Ily(n

lIy(n

V(n, Yn) ::;

+ k, n, Yn)IIP

+ k, no, Yo)[JP

-lly(n, no, Yo) liP,

which completes the proof.

4.11.

definite.

•

Total and Practical Stability

Let us consider the equations

Yn+l

=

f(n, Yn) + R(n, Yn)

(4.11.1)

Yn+l

=

f(n, Yn),

(4.11.2)

where R is a bounded, Lipschitz function in B a and R(n, 0) = o. We shall consider (4.11.1) as a perturbation of equation (4.11.2). Suppose that the zero solution of (4.11.2) has some kind of stability property, we want to know under what conditions on R the zero solution preserves some stability.

Definition 4.11.1. The solution Y = 0 of (4.11.2) is said to be totally stable (or stable with respect to permanent perturbations) ir'for every e > 0, there

120

4.

Stability Theory

exists two positive numbers 8 1 = 8 1 (e) and 8 2 = 8i e) such that every solution y( n, no, Yo) of (4.11.1) lies in Be for n 2= no, provided that

IIYol1 <

81

and

Theorem 4.11.1. Suppose that the trivial solution of (4.11.2) is uniformly asymptotically stable and moreover

Ilf(n, y') - f(n,

y")11 ~

Lrlly' -

y"ll,

y', y" E B,

C

Ba •

Then it is totally stable. Proof. Let y(n, no, Yo) be the solution of the unperturbated equation. The hypothesis of uniform asymptotic stability implies that for 0 < 80 < a, there exist 8(80 ) > 0,8(80 ) < a such that Yn E BIJ(lJ o), lim y, = 0 and moreover there exist the functions a, b, c E K and V such that for n E N~o' (a)

a(llfll) ~

V(n,

(b) 11 V(n, Yn) ~

y) ~ b(llfll),

-c(IIYn+111),

and (c) Iv(n, y') - V(n, y")1 ~ MIY' - y"l,

for y',

y"

E BIJ(lJ o)'

M>O'

Let 0 < e < 8(80 ) , Choose 8 1 > 0, 82 > 0 such that

b(28 1 ) < a(e),

82 ~ 8 1 ,

c(8 1 ) 82 < M'

(4.11.3)

Suppose that e is sufficiently small so that Lee + 8 2 < 8(80 ) , Let IIR(n, y)11 < 8 2 for y E Be. One then finds for Ilyll ~ e, Ily(n + 1, n, y)11 = If(n, y)1 ~ Lee, Ily(n + 1, n,y) - y(n + 1, n,y)11 =:' IIR(n,y)1I < 82 , and Ily(n + 1, n,y)11 ~ Lee + 8 2 < 8(80 ) , Thus, for Ilyll < e, we have V(n + l,y(n + 1, n,y)) - V(n,y) = V(n + l,y(n + 1, n,y)) - V(n,y)

+ V(n + l,y(n + 1, n,y)) - V(n + l,y(n + 1, n,y)) ~ -c(lly(n + 1, n, y))11 + Mlly(n + 1, n, y) - y(n + 1, n,y)1I ~ -c(lly(n + 1, n, y)11) + M8 2 < -c(lly(n + 1, n, y)ll) + c(8 1 ) . (4.11.4)

121

4.11. Total and Practical Stability

Now suppose that there is an index n\:?: no and a y; E Bat such that y(nj, no, y') e Be and y(n, no, y') E Be for n < n\. It then follows that V(nj, y(nl> no, y'»

:?:

a(e) > b(28\)

:?:

b(8\

+ 82 ) ,

and V(no, y') < b(8\). Then there exists an index n2 E [no, n\ - 1] such that V(n2,y(n2' no,y'»::s

s(s, + 8z) (4.11.5)

and V(n2 + 1, y(n2 Thus

+ 1, no, y'»

:?:

b(8\

+ 82 ) ,

Ily(n2 + 1, no, y')11 :?: 8\ + 8z from which we get 11Y'(n2 + 1, nz , y(n z , no, y')11 :?: IIy(nz + 1, no, y')11 -lly(n z + 1, »«, y(n2, no, y'» - Y(n 2 + 1, »«. y(n z , no, y'))II :?:

8\ + 8z - 82

= 8\.

Then from (4.11.5) and (4.11.4), it follows that

o~

+ 1, y(nz + 1, no, y'» - V(n z , y(n z, no, y'» < -c(IIY'(n2 + 1, n«, y(n2' no, y')II) + c(8\) ::s 0, V(nz

which is a contradiction.

•

Corollary 4.11.1. Suppose that the hypothesis of Theorem 4.11.1 is verified and moreooer.for y; E B; one has IIR(n, Yn)II < gnilYn II with gn ~ 0 monotonical/y. Then the solution of the perturbed equation is uniformly asymptotically stable. Proof.

From (4.11.4) one has

-c(IIY'(n + 1, n, Yn) II) + Mg nllYn II. Suppose 0 < r < 8(80), and r < IIY(n + 1, n, Yn)11 < 8( 80), by the hypothesis on gn, it can be chosen an n\ E N:o such that for n :?: n\ one has Mg; llYn II < ~ V(n, Yn) -s

2-\c(r) and then

Ll V(n, Yn)::S

Then apply Theorem 4.8.5.

-!c(IIY'(n + 1, n, Yn).

•

Connected to the problem of total stability is the problem of practical stability of (4.11.2) and (4.11.1). In this case we no longer require that R(n, 0) ¥- 0 so that (4.11.1) does not have the fixed point at the origin, but it is known that IIR(n, 0)11 is bounded for all n. This kind of stability is very important in numerical analysis, where certain kinds of errors cannot be made arbitrarily small.

122

4.

Stability Theory

Definition 4.11.2. The solution Y = 0 of (4.11.2) is said to be practically stable, if there exists a neighborhood A of the origin and fl ~ no such that for n ~ fl the solution y(n, no, Yo) of (4.11.2) remains in A.

Theorem 4.11.2. Consider the equation (4.11.1) and suppose that in a set D c R the following conditions are satisfied (1) IIf(n, y) - f(n, y')11 ~ Lily - y'll,

L< 1,

(2) IIR(n, y)11 < 8. Then the origin is practically stable for (4.11.2).

Proof. Let Yn and Yn be the solution of (4.11.1) and (4.11.2) respectively. Set m; = llYn - Yn II then by hypothesis mn+1 -s Lm; + 8 from which it follows n-I 8 j llYn - )in II ~ L n IIyo - Yoll + 8 j~O L = L n II Yo - Yoll + 1 _ L' If Yo = Yo, we see that the distance between the two solutions will never exceed _8_. Thus choosing n > fl suitably, both of the solutions will l-L remain in the ball

B(O, _8_) and the proof is complete. 1- L

•

The next theorem generalizes the previous result. Theorem 4.11.3. Consider the equation (4.11.1) and a set D c R S • Suppose there exists two continuous real valued functions defined on D such that, for all XED (1) V(x) ;:::0,

(2) .:l V = V(f(n, x)

+ R(n, x» -

V(x)

~

w(x)

~

a,

for some constant a;::: O. Let S = {x E 151w(x);::: O}, b = sup{V(x)lx E S} and A = {x E 15 IV(x) ~ b + a}. Then every solution, which remains D and enters A for n = ii; remains in A for n ;::: fl.

Proof. Suppose that Yn = y(fl, no, Yo) E A, then V(Yn) ~ b + a and V(Yn+I) -s V(Yn) + w(Yn). If w(Yn) is less than zero, Yn e S, then V(Yn+I) ~ b + a from which it follows that Yn+1 E A. If Yn E S, then, because V(Yn) ~ b, again it follows that V(Yn+I) ~ b + a. The proof is complete by induction. • Corollary 4.11.2. If 8 = sup{ w(x) IXED - A} < 0 then each solution y(n) of (4.11.1), which remains in D enters A in a finite number of steps.

4.12.

123

Problems

Proof. From 11 v s w(y) we get V(Yn):S; V(Yn) + 1:;=no w(Yj»:S; V(Y"o) + I)(n - no), from which it follows that V(y(n» ~ -<X) as n ~ 00, and this is a contradiction because V(Yn) 2': b + a for Yn E D - A. •

4.12.

Problems

4.1.

Show that if in the equation X n+1 = anxn, the product bounded, then the zero solution is uniformly stable.

4.2.

Consider the following equation, Yn+1 = n

In:no ail is

~ 1 y~. Compute, by using

a calculator, the two solutions starting from Yo = 10- 2 \ n = 0 and Yo = -1, no = 210 2 • Conclude that the zero solution is not uniformly stable. 4.3.

Show that for the system (4.1.8), the origin is the only fixed point that is attractive but not stable.

4.4.

Let Y = (x,

z) be such that Yn+1

=

(x~2x- z~).

Show that the set

nzn

D = {(x, z) Ix 2 + Z2 :s; 1} is the region of asymptotic stability of the origin. (Hint: Apply Theorem 4.9.3. Take (y) = x 2 + Z2 and show that if there is convergence, then which V(Yo) 4.5.

=

x~

n:}

1

(1 + (Yj»

=

1

1-

2

Xo -

2

Zo

from

+ z~).

. , . log(n + 2) . Consider the linear scalar equation Yn+1 = ( ) Yn' Find the log n + 3 solutions and show that the zero solution is asymptotically but not exponentially stable. (Hint: Put z; = Yn log(n + 2».

4.6.

Show that the zero solution of the previous problem is not Ip stable for any p 2': 1.

4.7.

Show that the function 8(e, no) in the definition of stability can be taken of class K. (Hint: Take 81(10 ) = sup 8(10, no). The function 1)1(e) is positive and strict increasing. Take l/J E K such that l/J( e) :s; I)I(e). This function can be used instead of I) in the definition.)

4.8.

Show that an alternative definition of stability is: The trivial solution is stable if there exists E K and nl E N;o such that for n 2': nl, Ily(n, no, Yo) I :s; (1IYoll, no). (Hint: Take as the inverse of rJ! defined in the previous problem.)

4.9.

Using Theorem 4.8.5 study the behavior of the solutions of the equation Yn+1 = y~2.

124

4. Stability Theory

4.10. Consider the equation Yn

o

+ b, where

AYn-l

=

o 0 0

A=

0 0 1

-2:

o

o

o

SXS

and study the stability of the critical point. The problem arises in economics treating oligopoly systems. The case s = 2 was treated by Cournot in finding asymptotic stability. 4.11. Show that the function g(n, u) = u; - w(un ) in Corollary 4.8.3 is not decreasing and prove that the origin is asymptotically stable for (4.8.6). 4.12. Show that O(Yo), the limit set of Yo E R 4.13.

.

•

=

Consider the equation Yn+1 of

1]

1] -

h~

1 - Yn

S ,

is invariant and closed.

.

, Yo = 1]. Determine the values

for which Yn converges and find the limit point.

4.14. Consider the system Xn + 1 =

Yn

Yn+1 = x,

+ f(x n),

»

and suppose that a(xJ(xn > 0 for all n. Show, by using Theorem 4.8.7 that the solutions are either unbounded or tend to zero. (Hint: Take Vn = xnYn.)

4.13.

Notes

The definitions in Section 4.1 are modified versions of those given for differential equations, for example, see Corduneanu [30] and Lakshmikantham and Leela [90]. Theorem 4.1.1 is adapted from Gordon [61]. The

4.13.

Notes

125

results of Section 4.2 are adapted from Corduneanu [29]. Most of the material of Section 4.4 has been taken from Halanay [68,69,70], see also [46]. The contents of Sections 4.5, 4.7, and 4.8 have been adapted from Halanay [68] and Lakshmikantham and Leela [90]. For Theorem 4.8.4, see Gordon [61]. Theorems 4.8.5 and 4.8.6 are due to Hurt [81] (see also LaSal1e [88]). Theorem 4.9.3 is the discrete version of Zubov's theorem and can be found in O'Shea [120] and Ortega [125]. Theorem 4.9.4 is due to Diamond [43,44]. The converse theorems are adapted from Lakshmikantham and Leela [90] and Halanay [68]. The total stability in the discrete case is treated by Halanay [68] and Ortega [125]. Practical stability is discussed in Hurt [81] and Ortega [125].

CHAPTER 5

5.0.

Applications to Numerical Analysis

Introduction

Despite the fact that the two theories have been developed almost independently, the connections between numerical analysis and the theory of difference equations are several. In this chapter, we shall explore some of these connections. In Sections 5.1 to 5.3, iterative methods for solving nonlinear equations are discussed and the importance of employing the theory of difference inequalities is stressed. Sections 5.4 and 5.5 deal with certain classical algorithms from the point of view of the theory of difference equations. Sections 5.6 and 5.7 are devoted to the study of monotone iterative techniques, which offer monotone sequences that converge to multiple solutions of nonlinear equations. This study also includes extension of monotone iterative methods to nonlinear equations with singular linear part as well as applications to numerical analysis. In Section 5.8 we provide problems of interest.

5.1.

Iterative Methods

By iterative methods one usually means methods by which one is able to reach a root of a linear or nonlinear equation. Let us consider the problem of solving the algebraic equation of the form F(x) = 0,

(5.1.1)

127

128

5.

where F: R

S

~

R

S ,

Applications to Numerical Analysis

which is usually transformed into the iterative form:

Xno = Xo. (5.1.2) The function f(x) is called the iterative function and is defined such that the fixed points of f are solutions of (5.1.1). There are many ways to define the iterative function f. The essential criterion is that the root x* of (5.1.1), which we are interested in, is asymptotically stable for (5.1.2), although it may not be sufficient to ensure rapidity of convergence. This, however, will imply that if Xo lies in the region of asymptotic stability of x*, the sequence will converge. If f is linear, we know that asymptotic stability can be recognized by looking at the eigenvalues of A (the iterative matrix) and indeed this is what is done in the study of linear iterative methods such as Jacobi, Gauss-Seidel, SOR, etc. Moreover, in the linear case, asymptotic stability implies global asymptotic stability. For nonlinear equations the situation becomes more difficult. In this case there are essentially three kinds of results that one can discuss: Xn+l

= f(x n ) ,

A. LOCAL RESULTS. These results ensure asymptotic stability of x*, but they have nothing to say on the region of asymptotic stability. Usually theorems in this category start with the unpleasant expression "If Xo is sufficiently near to x* ...." B. SEMI LOCAL RESULTS. The results in this category verify that an auxiliary positive function (usually the norm of the difference of two consecutive terms of the sequence) is decreasing along the sequence itself. The sequence is supposed to (or one proves that it) lie in a closed set D c R Then one can infer that the auxiliary function has a minimum in D and this minimum is located at x*. We shall see that this usually requires that x* is exponentially stable. Connected to these types of results are those requiring the stronger condition of contractivity off in a closed set D c R Contractivity implies exponential stability of the fixed point x*. C. GLOBAL RESULTS. These results say that the sequence x, given by (5.1.2) is almost (except, on a set of measure zero) globally convergent. S

•

S

•

Of course, results in the class (C) are very few in the nonlinear case. One such result is due to Barna and Smale, which says that for a polynomial with only real root, Newton's method is almost globally convergent, which we shall not present here. We shall, however, discuss in the next two sections some of the most important results in the classes (A) and (B).

5.2. Local Results

"

Let us begin with the following main result. Here f: R x* = f(x*).

S

~

R and x* satisfies S

5.2.

129

Local Results

Theorem 5.2.1. Suppose that f is differentiable with Lipshitz derivative, and the spectral radius off'(x*) is less than one. Then x* is asymptotically stable. Proof. By the transformation eTj difference equation en + 1

= f(x n) - f(x*) = f'(x*)en + =

where

f'(x*)en

=

xTj - x*, from (5.1.2) we obtain the

L

{f'(x*

+ s(xn -

x") - f'(x*)} ds en

(5.2.1)

+ g(en)

l~~ II~~:II)II = 0,

(see problem 5.2). The equation (5.2.1) is in the

form required by Corollary 4.7.2, which assures the exponential asymptotic stability of x*. Note that the exponential asymptotic stability does not imply in general that /lxl - x*11 < Ilxo - x"], unless (see section 4.2) aT] < 1, where the constant a depends on the norm used. The result established in Chapter 4 can be used to prove the next theorem on the convergence of nonstationary iterative methods defined by (5.2.2) where q : N;o x D c R ~ R , XO E D. It is obvious that nonstationary iterative methods are, in our terminology, nonautonomous difference equations. • S

Theorem 5.2.2.

S

Consider the difference equation

yno = X o ,

(5.2.3)

where f: D ~ R S , and f is locally Lipshitzian. Suppose that y* E D is an asymptotic stable solution for (5.2.3) and that the solutions y(n, no, x o) of (5.2.3) and x(n, no, xo) of (5.2.2) are in D for n E N;o' Assume further that

Ilq(n, x) - f(x)11 ~ Lnllx where L; ~ 0 for n ~ suitably chosen.

Proof.

y*ll,

(5.2.4)

Then the solution x(n, no, xo) ~ y*, when

00.

Xo

is

Rewrite (5.2.2) as Xn+I

= f(x n ) + R(n, x,)

(5.2.5)

and consider (5.2.5) as the perturbed equation 4felative to (5.2.3) with (5.2.6)

130

5.

Applications to Numerical Analysis

as the perturbation, which tends to zero as x, 4.11.1. •

~

y*. Then apply Corollary

Other results of local type can be obtained by using the comparison principle given in section 1.6. For example: Theorem 5.2.3.

Let f: D c R S ~ R S continuous, g: R+ ~ R+. Suppose that

(1) there exists x*

E

Int D such that for all x

E

B(x*, ll) < D, II

> 0,

(5.2.7) Ilf(x) - x*11 :5 [x - x*11 + g(llx - x*Ii); u + g(u) is nondecreasing with respect to u and g(O) = 0;

(2) G(u) = (3) the trivial solution of

Un + 1 =

G(u n ) ,

Uo = Ilxo - x*ll,

is asymptotically stable, with U o in the domain of asymptotic stability, and u, :5 Uo for all i. Then x* is an asymptotically stable fixed point for the equation

(5.2.8) Proof. Let Xo E B(x*, ll). Then by Theorem 1.6.1, we have [x, - x*11 :5 u; and, because Un is decreasing to zero, all x, E B(x*, 8) and the sequence will tend to zero. •

The next two theorems are applications of LaSalle's Theorem 4.8.6. Here we study the convergence of secant and Newton methods on the real line. These two methods are widely used in numerical analysis to find roots of equations and their generalizations are almost countless. On the real line, these two methods are defined by (5.2.9) and (5.2.10) respectively. Iff(zk) ¥- f(Zk+I), the difference equation (5.2.9) is well defined. Suppose that f is differentiable and that a is the simple root of f( z) = O. Using the transformation (5.2.11) and the mean value theorem f(a

+ e)

= f'(a)

+ g(a,

e)e 2 ,

(5.2.12)

S.2.

Local Results

131

we write the new difference equation corresponding to (5.2.9) having the fixed point at the origin in the form (5.2.13) where _ g(a, en+l)en+ l - g(a, en)e n M( a, en, en+ l ) ( . / a + en+!) - /(a + en)

(5.2.14)

If a is a simple root and g( a, e) is continuous and bounded, then Mt;«, ek, ek-l) is continuous and bounded if ek, ek-l are small enough. The second order difference equation (5.2.13) can be transformed into a first order system by introducing the variables x, = en and Yn = en+ l • Then the system is

= Yn, Yn+l = M(a, xn,Yn)xnYn' Lyapunov function V(x, y) = Ix[q + Iylq, X n+ l

By using the

q

2':

LlV(xn, Yn) = -(1 -IM(a, x n, Yn)Ynlq)lxnl

1, one obtains

q,

which is negative if W(x,y);;;: (l-IM(a,x,y)yj) > O. Let D(1) = {(x,y)I(!xl q + Iylq)l/q < 1)}. Since y = 0 implies W(x, y) = 1 > 0, there exists some 1) > 0 such that W(x, y) 2': 0 in D( 1). If the initial values Zo and ZI are such that (x o, Yo) and (Xl> Yl) E D( 1), then (x n , Yn) will remain in D (see Corollary 4.8.5) and will converge to the maximum invariant set contained in the set E = {(x, y) E 15(1) Ix = O}. The only invariant set with Xl = 0 is the origin, so we obtain (xn, Yn) ~ (0,0) for n ~ 00. Corollary 4.8.5 can be used to study the convergence of (5.2.10). Because there is no more difficulty involved we shall consider the case where (5.2.10) is defined in R S • The root a, which we are interested in, is simple. The change of variable similar to (5.2.11) allows us to write the method in the form where M l ( en) -_ [d/(a It can be

+ en)]-l and M ( en) -_ [d/(a + en)] . 2

ax shown that for I enII <

IIMI(en)[Mz(en)en

ax

1), -

/(a

+ en)] II =:; k(1)lle~ll.

132

S.

Then, taking Vee)

= IIell, one

Applications to Numerical Analysis

can obtain

AV(e):s -(1- k( 17)llell)llell

;:: - W(llell),

showing that A V :s 0 for k( 17) II en II :s 1. Using Corollary 4.8.5, we get a region of convergence D( 170) = {x Illx - a II < 170} with 170 = mine 17, 1/ k( 17». The parameter 17 can be chosen to maximize 170 and consequently the region of convergence.

5.3. Semilocal Results Results of semilocal type are obtained either by requiring contractivity or some of its generalizations on a compact D c R or by requiring the weaker condition that some auxiliary function decreases over a sequence. We shall consider some cases of the latter type. We begin discussing the simplest of such results in some detail to bring out clearly the arguments used. S

Theorem 5.3.1. Suppose that all x, obtained by (5.1.2) lie in a compact Do c R S and that

Ilxn+2 - xn+111:s allxn+l - xnll, with a < 1. Then the sequence converges to x* Proof.

E

(5.3.1)

Do.

Setting (5.3.2)

we have, because of (5.3.1), We can now apply Theorem 1.6.1 to obtain solution of Un + 1

Yn:S Un'

where

Un

is the (5.3.4)

= aU n

provided that Yo :S Uo. Now the equation (5.3.4) has the solution u; = 0 exponentially asymptotically stable, which means (see Theorem 4.1.1) that I;:l uj is bounded. In 00

fact from (5.3.4) we get Ij=o uj =

1 I-a

--Uo.

It then follows that

Uj,j = 1,2, ... , if we choose Uo = Ilxl IIxn+p

-

Xn

II :S IIxn+p

-

Ilxj + 1 -

Xjll

:S

- xoll. Moreover, for p ~ 1, we obtain x n+p - 1 + x n+p - 1 - X n- p-2 + ... + X n+1 - X nII

p-l

:S j=O L Ilxn+j+l -

p-l

xn+J

:S j=O L un+j •

133

5.3. Semllocal Results

Because the right-hand side can be made less than an arbitrary positive quantity when n is chosen properly, this inequality shows that x, is a Cauchy sequence and thus must converge to x* in Do- From this result we can also find the distance between Xo and x*. In fact, from n-l

n-l

j=O

j=O

[x, - xoll ~ I IIXj+l - xjll ~ I

Uj>

it follows that

Ilx* -

xoll

~

1

00

I Uj j~O

=

--Uo· 1- a

(5.3.5)

Using this property one can avoid the a priori assumption that all x, lie in Do. Also, if we assume thatfis continuous, thenf(x*) = x*. Furthermore, if we assume more than (5.3.1), namely, x, y ED, Ilf(x) - f(y) II ~ k < 1, then the solution x* is unique in Do. • Using similar arguments, one can also prove the following result. Theorem 5.3.2. Suppose that xo, Xl lies in Do and Bt x«, L;'o Uj) c Do. Then the sequence converges to x* E B(xo,LJ:=o Uj)' The next theorems are on the convergence of Newton's method defined by (5.3.6) where F is the same function as in (5.1.1). The main result was first proved by Kantorovich and is usually called Newton-Kantorovich theorem. Theorem 5.3.3. ofR s,

Assume that, for X o, x, Y E D, where D is a convex subset

(l) F is differentiable in D; (2) IIF'(x) - F'(y)11 ~

(3)

II F'(xo)-lll

(4) f3'YTJ (5)

'Yllx - yll; 11F'(xo)-1 F(xo) II

~ f3,

=

TJ;


B( Xo, f31") c

D.

Then the sequence defined by (5.3.6) is well defined and converges to a root x* of F(x) = 0 contained in B(xo, I/f3'Y)· Proof.

For every

X E

Bt x«,

II {3'Y) we have

IIF'(x) - F'(xo)11 ~

'Yllx - xoll

< {3-I.

134

5.

Applications to Numerical Analysis

A bound of IIF'(x)-111 can then be obtained by using the Banach lemma (see Corollary A5.2), which yields <: IIF'(xo)-111 <: f3 x - I - IIF'(xo)-IIIIIF'(x) - F'(xo)ll- 1 - f3'Yllx - xoll'

IIF'( )-111

From this estimate and from hypothesis (4), it follows that for x E B(xo,I/f3'Y), F'(x) is nonsingular and the Newton's iterative function is defined. Moreover,

and (see problem 5.2) IIF(xn+1)11 = IIF(xn+l) - F(xn) - F'(xn)(xn+ 1 :s; hllxn+1 - x n11

2

-

xn)11

(5.3.8)

•

Substituting for F(x n + l ) in (5.3.7), we then find

IX

2

n+2 -

f3'Y Ilxn+1 - xnl1 Xn+1I :s; - 2 1 - f3'Y II.Xn+1 - Xo II'

Suppose for a moment that the sequence X n lies in B(xo, 1/ f3y), then we . gUn, ( "n "n-I) = f3'Yu~n ' The can apply Theorem 4.5.4 with L.j=1 Uj, L.j=1 U j

2(1 - f3y

Lu

j )

j=O

comparison equation is therefore f3Yu~

(5.3.9)

whose solution is Un = (,8'Y)-lr n- l. The origin is thus exponentially stable for (5.3.9). All x, lie in B(xo, 1/ ,81') because [x, - xoll < (f3'Y)-I2:;::11 2-

j

-

1

<

;1"

We may apply the

same arguments used

b ef ore. In f act, II xn+ p _ Xn II <: - "p-I L.j~O II xn+j+ 1 _ xn+j

II

<: -

"p-I _ (f3'Y)-1 L.j=O un+j 2

x 2:;::~ r n - j = (f3'Y)-lr n(l - r p ) , which shows that X n ~ x* and also gives the error estimate IIx* - xnll:s; (f3'Y)-lr n. The proof that x* is a root of F(x) follows from F(xn) = F'(Xn)(Xn+1 - x n) = [F'(xo) - (F'(x n) - F'(xO))](Xn+1 - x n) and then IIF(xn)ll:s; [IIF'(xo)11 + 'Yllxn - xoll]llxn+1 - xnll:s; [IIF'(xo)11 + 'Y(f3'Y)-I]llxn+1 - xnll. Taking the limit as n ~ 00 one has, by the continuity of F, F(x*) = O.

5.3.

Semilocal Results

135

Hence by Theorem 4.5.4, the conclusion follows. • The conclusion of Theorem 5.3.3 has been easily obtained because the solution of (5.3.9) can be given in the closed form. More refined estimates can be obtained if we assume in (5.3.9) that Uo < !(f~y)-l. In this case, the solution of (5.3.9) is more difficult to compute but it can be done. In fact, let us define the new sequence Zn by dZn- 1 = {3YUn, L l = O. Then . (5.. 3 9) IS . transtorme .. d'mto Zn+l = Zn + -1 (zn - Zn_l)2 , W h'IC h equation 2

1 - z;

possesses the first integral, that is, the sequence {z.} satisfies the first order . Zo - !z~ . of equation Zn+l = , where Zo = {3yuo . (See (1.5.12).) The solution 1 - Zn this equation is z; = 1 - (1 - 2Z0 ) 1/ 2 coth k;", (see problem 1.27) where k is a constant depending on the initial condition. (See problem 5.8 for its determination and for a more refined bound.) This solution exists if Zo s ! and then Uo s !({3y)-l. The case of equality has already been examined. Suppose that Ul < !({3y)-I. Then it follows that lim z; = 1 - (1 - 2Z0 ) 1/ 2 == z* and since for n~OO

• , we obtain z; - z* = 2(1 - 2z o) 1/2 e" k2 + . • Zn - z* Consequently, we get !~~ (Zn-l _ z*? = 1, which shows that the order of

large n, coth k2 n

= 1 + 2 e" k2

n+l

n

1

convergence of the sequence z.,, to the limit point, is quadratic. From

the

comparison

_l_(Zn - Zn-l) and hence {3y

[x, of z.;

xoll s l:;':-~ Uj S

Ilx

n -

principle

it

x*11 < ~(1 {3y

follows

that

Ilxn+1 -

Xn

II <

1

2Z0 ) 1/ 2 e- k 2 n + • Also for n > 0,

_l_ Zn_ 1 s·_l- z * because of the increasing behavior

{3y

(3y

•

Summing up these considerations we see that we have proved a variant of Theorem 5.3.3, which we state below. Theorem 5.3.3(a).

Assume that the hypotheses (1), (2), (3) of Theorem 5.3.3

hold. Suppose further that (4) f3yY]


Z*)

and (5) B( Xo, f31 c: Dare y satisfied. Then the sequence x, is well defined and converges quadratically to

x* contained in B( xo, (31y z*). Newton's method is invariant under affine transformation, that is, it is invariant under the transformation G = AF where A is an invertible matrix,

136

S. Applications to Numerical Analysis

whereas the hypotheses in the Newton-Kantarovich theorem are not. In fact, one has 0- 10 = F-1A-1AF = F- 1 F, while for example, the hypothesis (3) is not invariant under the same transformation, The following variant of Theorem 5.3.3 takes care of this situation. Theorem 5.3.3(b). and suppose that

Assume that the hypotheses (1) of Theorem 5.3.3 holds

II F'(XO)-I F(xo) II :s; a; IIF'(xo)-I(F'(y) - F'(x»11 :s; wllx - yll;

(2) (3) (4) (5)

aw:s;!; B(xo, w-1z*)

E

D, where z* = 1 - (1- 2wa)I/Z:s; 1.

Then the sequence x, remains in B(xo, w-1z*) and converges quadratically to a root x* of F(x).

Proof.

We have

Xn+Z - Xn+1 = -F'(Xn+I)-IF(xn+l)

= -F'(Xn+I)-I[F(xn) + F(xn+ 1 )

-

F(xn)]

-F'(Xn+I)-I{ F(x n) + F'(Xn)(Xn+1 - x n)

=

+ = -

f

[F'(x n + t(xn+1 - x n» - F'(x n)] dt(x n+1 - x n)}

F'(X n+1)-1 F'(xo){I F'(XO)-I[F'(xn + t(xn+1

-

»

xn

- F'(xn)] dt(x n+1 - Xn)}, from which it follows that Ilxn+z - xn+lll -s IIF'(xn+I)-1 F'(xo) II . ~ Ilxn+1 - x,

liZ,

Since F'(XO)-IF'(xn+ l ) == F'(XO)-I F'(Xn+l) - F'(xo)-I F'(xo) + F'(XO)-I F(xo), using hypothesis (3), we get 11F'(xO)-I(F'(xn+l) - F'(xo» II < wllxnn - xoll < z*:s; 1, if Xn+l E B(xo , w-1z*). Then, by the Banach lemma it follows that F'(x n+ l) is invertible and II F'(Xn+I)-1 F'(xo) II :s; 1-

II 1

W Xn + 1 -

II' Finally, we arrive at

Xo

IIXn+Z -

w Ilxn+1 - xnf xn+lll :s; -2 1 _ I _ II' w Xn+1 Xo

137

5.3. Semilocal Results

which is the same as before with f3y replaced by w. All the previous results then apply, and we are done. • The next result also shows the familiar quadratic decay of the errors in the Newton method, but it requires more assumptions on the inverse ofthe Jacobian.

Assume that F: Do c R S Moreover, suppose that

Theorem 5.3.4. (1) (2) (3) (4) (5)

~

R S and Do c R S is a convex set.

for x E Do, F is differentiable, for x E Do, F'(x) is invertible and IIF'(x)-111 for x, y E Do, 11F'(x) - F'(y)11 :5 yllx - yll, for Xo E Do, IIF'(xo)-IF(xo)II:5 TJ, and (\' = 4f3YTJ < 1.

:5

f3,

Then the Newton iterates remain in ei«; ro), where ro converge to a solution x* of F(x) = O. Proof.

= TJ 2:~~0 (\'ZH

and

From

IIx n+z -

xn+III =

IIF'(xn+ I)-IF(x n+I)11

:5

f3IIF(xn +1 ) II

and from the estimate of II F(xn + l ) I obtained in the previous theorem, we get (5.3.10) The comparison equation is therefore

f3y

Z

Un + 1 = TUn,

Uo

=

(5.3.11)

TJ,

whose solution is u; = TJ(\'zn_ l • Now applying comparison Theorem 1.6.1 we get (see problem 5.4)

[x, - xn+11 1:5

Un

= "1(\'zn_ l.

(5.3.12)

Since (\' < 1, the trivial solution 01 (5.3.11) is exponentially stable and hence it follows that x, is a Cauchy sequence. Furthermore, we also have [x, xoll :52:';:0 Uj = ro, which means that all the iterates remain in Bt x«, ro) and converge to x*. • Corollary 5.1.1.

The error [x" - Xn II satisfies the inequality

Ilx* - x, I :5 en IIxn -

Xn-I

liz,

for n

= 1,2, ....

(5.3.13)

As an application of Theorem 1.6.7, let us consider the two-step iterative method

138

5.

Applications to Numerical Analysis

Theorem 5.3.5. Suppose that G: D x D c R x R set Do c R the following inequality holds true S

S

~

R S and on some closed

S

,

IIG(x,y) - G(y,

z)ll:5 allx - yll + ~lly - zll, let there exist x o, XI E D such

where a + ~ < 1. Furthermore, that Xk E Do for k e: 1. Then the iteratives converge to the unique fixed point of G(x) = G(x, x). Proof. Setting Yk = Il xk + l - xkll, we obtain Yk+l:5 aYk + ~Yk-l' By Theorem 1.6.7, we then get Yk :5 Uk, where Uk is the solution of Uk+l = auc + ~Uk-I' which is Uk = coz~ + CIZ~, where ZI and Z2 are the roots of Z2 - az - ~ = O. Because of the assumption on a and ~, the two roots ZI and Z2 are less than 1 in modulus, which means that the zero solution of the comparison equation is exponentially stable. Using similar arguments as in the previous theorems, we can conclude that the sequence Xk is a Cauchy sequence. Since

IIG(x*) - Xk+111 :5IIG(x*, x") - G(x*, Xk-I)II + IIG(x*, Xk-I) - G(Xk' Xk-I)II :5 ~llx* - xk-t11 + allx* - xkll, the convergence of Xk shows that Xk+l ~ G(x*). The uniqueness of x* now • follows very easily and the proof is complete. The effect of perturbation on the iterative methods are of two different kinds. If the perturbations are small and tend to zero as n tends to 00, then the theorems on total stability will ensure that the fixed points continue to have asymptotic stability. In numerical analysis, however, perturbations may remain bounded for all n (roundoff errors, for example). In this case, a more convenient concept is the practical stability. If an iterative procedure is practically stable, the sequence of iterates will not tend to the solution x* but to a ball B(x*, B) surrounding x*. Inside B, the solution may oscillate but what is important is that it never leaves B. Of course it would be nice to have B as small as possible. Let us consider for example the iterative method

= Xo'

(5.3.14)

Ilf(x) - f(y) II :5 allx - yll

(5.3.15)

Xn+ 1 = f(x n ) ,

We assume that for x, y

E

Do c R

x na

S ,

with a < 1. Suppose that the errors in the computations perturb (5.3.14) by a bounded perturbation R n , that is

xn + 1 = f(x n ) + R n ,

xna = xo,

(5.3.16)

5.4.

139

Unstable Problems

with X n , xn E Do for all n. Then Theorem 4.11.2 can be applied. The conclusion is that the difference of the two solutions [x, - xn II will not exceed Rj(l- a), where R

B(Xn,~) I-a

=

sup Rn- The condition that all the balls

must be contained in Do may be very restrictive. In fact if a

is close to one, it follows that these balls are very large and the method is considered a bad one.

5.4.

Unstable Problems: Miller's, Olver's and Clenshaw's Algorithms

The situation that we shall analyze in this section and in the next is in a certain sense the opposite of that treated in the previous ones. In fact, there the limit point was important (and indeed in the theory of iterative processes the "solution" is the limit point), where the intermediate values z; are considered an unavoidable noise. Here the limit point will not be important (generally will not exist) and the important part ("the solution") will be a finite subset of the sequence itself. These problems are very typical in that part of numerical analysis which concerns the study of special functions, orthogonal polynomials, quadrature formulas, and numerical methods for ordinary differential equations. Consider the homogeneous scalar linear equation (see (2.1.5)) of order k. (5.4.1) It has k linearly independent solutions/l(n)'/in), ... '/k(n). Suppose that the solution Yn = 0 (n ~ 0) is unstable, and

. II(n) lim 1"( n ) = 0,

i = 2,3, ... , k.

(5.4.2)

n~oo.li

When this happens, the solution II(n) is said to be minimal. The problem is how to find II or a multiple of it. Even if we know the exact initial condition that generates the minimal solution, small errors in the calculations will, as usual, lead us to solve a perturbed equation LYn = en, whose solution (see (2.1.11)) will contain all the j;(n) and this will destroy the process (see for example [52], [175]), because the h(n) (i ~ 2) grow faster than II(n). The same problem appears, of course, in the nonhomogeneous case, where it is not excluded a priori that there exists a bounded solution even if all the hen) are unbounded. One is often interested in following this solution.

140

5.

Applications to Numerical Analysis

Consider, for example, the following first order equation Yn+l - (n

+ 1)Yn

= -1,

Yo = e.

(5.4.3)

The solution of the homogeneous part is Yn = en!, while the complete solution is Yn = n!( e -

2:.;=oh) ~ 2:.:=1 (n +l s

) (S) ,

which is bounded for

n ~ 00. If one tries to follow this solution, a small error in the initial data e (always existing because it has infinite digits), will be amplified by the factor n!. A simple idea is to find the way to look for the solution of the problem backwards (for n ~ -00), because in this case the origin becomes asymptotically stable for the homogeneous equation and the errors will be damped. This idea indeed works, as we will show in a simple case, which is the original proble~ to which it was applied (see [52] for references). Consider (5.4.6) where an, en ¥- 0 for all n, and the nonhomogeneous problem LYn = gn'

(5.4.7)

Suppose that two linearly independent solutions ¢n and l/Jn of the homogeneous equation are such that ¢n is minimal, that is

· ¢n 0. IIm-= "_00 t/Jn

(5.4.8)

Of course all solutions that are multiple of ¢n are minimal and vice versa. That means that only one appropriate initial condition is needed to determine the minimal solution we are interested in. The general solution of the nonhomogeneous problem is then (5.4.9) where

Yn

is a particular solution that will be supposed minimal too; that is,

· Yn 0 IIm-= I/Jn '

n_OO

(5.4.10)

and Yo = O. We are interested in the solution Yn

=

Yo,/, +_ ¢o't'n Yn'

A class of methods are based on the following result.

(5.4.11)

5.4.

141

Unstable Problems

Theorem 5.4.1.

Consider the boundary value problem Ly
= gn,

y~N)

= Yo,

Y<:')

= 0,

(5.4.12)

where N is a positive integer, and suppose that conditions (5.4.8) and (5.4.10) are verified. Then the problem (5.4.12) has a solution and moreover, for fixed n, y~N) ~ Yn as N ~ 00.

Proof. Since Yn is a particular solution of the nonhomogeneous equation, any other solution of the same equation will be y~N)

= c~N)cPn + ciN)ljJn + Yn'

(5.4.13)

The boundary conditions give for C~N) and ciN) the values YN C(N) _ I

-

IjJN

ljJo

It follows that C~N) and ci

(N) __ cPo (N)

.

cPo _ cPN'

C2

-

ljJo

CI'

IjJN

N)

tend to zero as N ~ 00 and the claim follows. Note that the condition Y<:') = 0 can be replaced by Y<:') = k where k is any fixed value (for example an approximation of YN if available). In the homogeneous case y~N) can be obtained starting from j<:') = 0, j~NlI = 1, and then using the difference equation backwards ~(N)

Yn-I

an _

b., :

c,

en

= --Yn+1 --Yn

obtaining a value y~N) different from Yo. Multiplying then the sequence by the scale factor Yolj~N), one obtains y~N). This is the Miller's algorithm. A disadvantage of this algorithm is that one does not know a priori which value of N must be used. A modified version of it, which works in the nonhomogeneous case is also obtained by considering that the boundary value problem (5.4.9) is equivalent to the system of equations Ay(N) = b, (5.4.14) where

bl

0

0

al

CI

A=

0

0 aN-I

0

0

bN - I

CN-I

and Y

(N) _

-

b

«(N)

Yt

= (gt

(N)

"'YN-t

)T

,

- cIYo, ... , gN-t) T.

142

5.

Applications to Numerical Analysis

Some clever method to solve this system can be used to avoid the growth of the errors (see [23], [24]). By solving the system in an appropriate way, it is also possible to determine the best value of N (Olver's algorithm). • Similar in motivation to the previous algorithms and indeed related to them is the following, known as Clenshaw's algorithm. Here the problem is to evaluate the sum L~=o bnun , where b; is a given sequence and u.; is supposed to satisfy a k th linear difference equation LUn = O. The algorithm uses the result stated in Theorem 2.1.8, that is, one considers the transposed equation LTYn = bn, YN+j = 0 (j = 1,2, ... , k), obtaining N

I

n=O

k-l

b.u;

=I

j=O

j

Yj

I

i=O

Pi(j - k)Uj_i'

Suppose, for example, we want to compute IN = Ln~o bnTn(z), where Tn(z) are Chebyshev polynomials satisfying the equation Tn + 1 - Zz'T; + T; = O. The transpose equation is N

Yn - 2ZYn+l YN+1

+ Yn+2

=

bn,

n

= N, N

- 1, ... , 0

= YN+2 = 0,

which can be solved recursively obtaining Yl and Yo. From the quoted result it then follows IN = Yo + Yl [T1(z) - 2zTo(z)] = Yo - ZYI' It is interesting to note that the sum can be obtained without the knowledge of the polynomials Tn(z) except for To(z) and T1(z) and also the reduction of the operations involved.

5.5.

Unstable Problems: the "SWEEP" Method

The topic discussed in this section is very similar to the previous one, but historically it has different sources. It arises typically in that part of numerical analysis that concerns the approximation of solutions of ODE and PDE. We shall consider, for simplicity, the autonomous case. The more general case is not, in principle, more complicated. Let us consider Yn+l - 25Yn + Yn-l

= gn,

y(O) = a,

Yk

= {3,

(5.5.1)

with 5 > 1, k> O. This problem can be solved in many different ways. It can be solved rewriting it as a system of k - 1 equations similar to (5.4.14). In order to control the growth of errors it is better to treat the problem as a difference equation.

5.5.

143

Unstable Problems

The characteristic polynomial of (5.5.1) has two positive roots, one less than one and the other greater than one. This means that the origin is not asymptotically stable for the homogeneous part and this, as usual, gives troubles because the errors are amplified. One way to avoid this difficulty is to transform the linear second order equation (5.5.1) into nonlinear first order equations (see problem 5.12). 1 Xn+1 = 2i> _ x ' (5.5.2) n

and (5.5.3) Furthermore, one verifies that Yn-l

= XnYn + z.:

(5.5.4)

The solution is then obtained by computing recursively the sequences {x n } , {zJ using (5.5.2) and (5.5.3) with XI = 0 and ZI = a up to n = k - 1 and then computing {Yn} using (5.5.4) from n = k to n = 1 with Yk = (3. The advantage is that (5.5.2) has a limit point asymptotically stable and the sequence x, converges monotonically to this point, while the sequences (5.5.3) and (5.5.4) remain bounded.

Theorem 5.5.1. If Xo = 0, 8> 1, the sequence {x n } converges monotonically to the first root p of the characteristic polynomial of (5.5.1) and, moreover, 0:$ x, < p. The critical points of (5.5.2) are the roots of the equation x 2 - 2i>x + 1 = 0, which is the characteristic polynomial of (5.5.1). Let p be the root less than one. Changing the variable h; = p(p - x n ) , equation (5.5.2) becomes Proof.

p2 hn + 1 = --h-hn • 1+ n

(5.5.5)

One shows at once (for example using the Lyapunov function Vn = h~) that the half line h > 0 is contained in the asymptotic stability region of the origin, and moreover 0 < h; :$ p2, from which it follows that -p :$ x, - P < O.

•

(5.5.6)

The next theorem is not really necessary in this case, but it could be useful in the more general case.

s.

144

Applications to Numerical Analysis

Theorem 5.5.2. In the hypothesis of Theorem 5.5.1 the solution x, = P is practically stable for (5.5.2). Proof.

The conditions of Theorem 4.11.2 are satisfied for x, < p.

•

The last result ensures that for equation (5.5.2) the errors will not be amplified. Let us consider the equations (5.5.3) and (5.5.4). Theorem 5.5.3. In the hypothesis of Theorem 5.5.1, the sequence {z.} remains bounded, even if the two equations are perturbed with a bounded perturbation. Proof.

From

p2 - 20p

+ 1 = 0, it follows that 20' - p = 1/ p and = P - gn( + Zn ) and f rom (5.5.6 )

. ( ) can be wntten 5.5.3 as Zn+1

IZn+tl:::; p(lgnl + IZnl)·

I - p xn - p

Using Corollary 1.6.1, it follows that IZnl:::; pna + L;:~ pn-s+1lgsl from which we see that if'[g.] is bounded z; is bounded. Clearly if we add to gn a bounded perturbation Tn (any sort of errors), the statement remains true. • Similar arguments can be applied to (5.5.4) since it follows that IYn-tl :::; plYnl + IZnl· The method just described is known as the "SWEEP" method (see [28]).

5.6. Monotone Iterative Methods In this section, we shall develop a general theory for a broad class of monotone iterations. This class of iterations includes Newton's method as well as a family of methods, which are called Newton-Gauss-Seidel processes that are obtained by using the Gauss-Seidel iteration on the linear systems of Newton's method. As before, we are interested in finding the solutions of

0= f(u), where f

E

C[R

n

;

R

n

].

(5.6.1)

Let us first split the system (5.6.1) as

0= j;(u i , [u]p" [u]q,),

(5.6.2)

where, for each i, 1:::; i:::; n, Pi + qj = n - 1 and u = (Ui, [u]p" [u]q')' Let v, W E R" be such that v :::; w. Then v, ware said to be coupled quasi lower and upper solutions of (5.6.1) if

o:::;j;(v [v]p" [w]q,), o-z.j;(w j, [w]p" [v]qJ i,

145

5.6. Monotone Iterative Methods

Coupled quasi extremal solutions of (5.6.1) can be defined easily. Vectorial inequalities mean the same inequalities between the components of the vectors. We are now in a position to prove the following result.

Theorem 5.6.1. Assume that f E C[R n , R n ] and possesses a mixed quasimonotone property, that is, for each i,};(uj, [u]p" [u]q') is non decreasing in [u]p, and nonincreasing in [u]q,. Suppose further v, ware coupled quasi lower and upper solutions of (5.6.1) and

j;(Uj, [u]p" [u]q) - };(iij, [u]p" [u]q.) 2:= -Mj(uj - iij) whenever v ~ ii -s U ~ wand M, > O. Then there exist monotone sequences {v n }, {w n } such that Vn ~ p, Wn ~ r as n ~ ex) and p, r are coupled minimal and maximal solutions of (5.6.1) such that v ~ p ~ u ~ r ~ wfor any solution u of (5.6.1). Proof.

For any

771,772 E [v, w] = [u E R" : v ~ u ~ w].

Consider the

system

u, = Mi l};( 77ti, [77I]P,' [772]q.)

+ 77ti,

i

= 1,2, ... , n.

Clearly U can be uniquely defined, given 771, 772 E [v, w]. Therefore, we can define a mapping A such that A[ 771, 772] = U. It is easy to show that A satisfies the properties: (i) 0 ~ A[v, w], 02:= A[w, v]; (ii) A is mixed monotone on [v, w]. Then the sequences {v n }, {w n } with Vo

= v, Wo = w can be defined as follows

Furthermore, it is clear that {v n } , {w n } are monotone sequences such that v ~ V n ~ W n ~ wand consequently lim V n = p, lim W n = r exist and satisfy the relations n_OO n_OO

By induction, it is also easy to show that if (Ul> U2) is a coupled quasi solution of (5.6.1) such that v ~ Ul> U2 ~ w, then V n ~ Ul> U2 ~ W n and consequently (p, r) are coupled quasi extremal solutions. Since any solution U of (5.6.1) is a coupled quasi solution, the conclusion of the theorem follows and the proof is complete. • If f does not possess the mixed quasi monotone property, we need a different set of assumptions to generate monotone sequences that converge to extremal solutions. This is the content of the following results.

146

S.

Theorem 5.6.2.

Applications to Numerical Analysis

Assume that

(i) there exist v, W E R" with v :s; W such that 0 :s; f( v), 0 =:: f( w); (ii) there is a n x n matrix M such that f(y) - f(x) =:: -M(y)(y - x), whenever v :s; x :s; y :s; w. Then the sequence {w n}, given by Wn + 1

= Wn + Bnf(wn ) ,

(5.6.3)

where B; is any nonnegative subinverse of M( wn), is well defined, and {w n} is monotone non increasing such that lim Wn = r. If there exists a nonsingular n~OO

matrix B =:: 0 such that lim inf B; =:: B, then r is a maximal solution of (5.6.1). n-e co

Set v = Vo and W = Wo' From B o > 0 and f( wo) :s; 0 it follows that Using the fact that B o is a subinverse of M (w o), we find for any [v, w],

Proof.

WI :s; W o'

u

E

u

+ Bof(u) =

WI -

:s; WI -

Hence, in particular, vo:S; Vo f( WI)

:s;

(w o - u) - Bo[f(w o) - f(u)J

[I - BoM(wo)](wo - u):s;

+ Bof( vo) :s;

f( wo) + M( woH Wo -

WI) =

WI'

WI'

Similarly, we obtain

[I - M( wo)BoJf(wo) s

o.

Proceeding similarly we see by induction that f(w n ) =:: 0,

n = 1,2, ....

(5.6.4)

Consequently, as a monotone nonincreasing sequence that is bounded below, {w n } has a limit r 2: Vo. If u is any solution of (5.6.1) such that u E [v, w J, then u = u + Bof( u) :s; WI' then by induction u :s; W n for all n. Hence u :s; r. Finally, the continuity of f and the fact lim inf B; =:: B, where B =:: 0 is nonsingular, (5.6.4) yields n

= -(lim inf B; )f( r) =:: -

Bf( r)

2:

0,

n~OO

which implies f(r) = 0 completing the proof.

•

By following the argument of Theorem 5.6.2, one can prove the next corollary.

5.7.

147

Monotone Iterative Methods

Corollary 5.6.1.

Let the assumptions of Theorem 5.6.2 hold. Suppose that M(y) is monotone non increasing in y. Then the sequence {v n } with Vo = v, given by V n+1

= Vn + Bnf(vn)

is well defined, monotone nondecreasing such that lim minimal solution of (5.6.1). n-+OO

Vn

= p, and is the

The case of most interest is when p = r, because then the sequences {v n }, {w n } constitute lower and upper bounds for the unique solution of (5.6.1). The following uniqueness result is of interest.

Theorem 5.6.3. Suppose that f(y) - f(x):5 N(x)(y - x), where N(x) is nonsingular matrix and N(X)-l ~ O. Then if (5.6.1) has either maximal or minimal solution in [v, w], then there are no other solutions in

[v, w]. Proof. Suppose r E [v, w] is the maximal solution of (5.6.1) and u is any other solution of (5.6.1). Then

0= f(r) - f(u)

:5

E [v, r]

N(u)(r - u).

Since N(U)-l ~ 0, it follows that r ~ u and hence r = u. A similar proof holds if the minimal solution exists. The proof is complete. •

5.7.

Monotone Iterative Methods (Continued)

Consider the problem of finding solutions of Ax = f(x),

(5.7.1)

where A is an n x n matrix and f E C[R n, R n ] , which arises as finite difference approximation to nonlinear differential equations. If A is nonsingular, writing F(x) = f(x) - Ax = 0, one can study existence of multiple solutions by employing the method of upper and lower solutions and the monotone iterative technique, described in Section 5.6. In this section we extend such existence results to equation (5.7.1) when A is singular. For convenience let us split the system (5.7.1) and write in the form (5.7.2)

148

5.

Applications to Numerical Analysis

where for each i, 1 ~ i ~ n, u = (Ui' [U]Pi' [u]q') with Pi + qj = n - 1 and (Au); represents the i l h component of the vector Au. As before, a function f E C[R", R"] is said to be mixed quasi-monotone if, for each i, /; is monotone non decreasing relative to [u ]p, components and monotone nonincreasing with respect to [u ]q, components. Let v, w E R" be such that v ~ w. Then v, ware said to be coupled quasi lower and upper solutions of (5.7.1) if

Coupled quasi-extremal solutions and solutions can be defined with equality holding. We are now in a position to prove the following result. Theorem 5.7.1. Assume that (i) f E C[R n , R n ] and f is mixed quasi-monotone; (ii) v, ware coupled quasi lower and upper solutions of (5.7.1); (iii) /;(u;, [u]p" [u]q') - /;(u;, [u]p" [u]q,) ~ -Mj(u j - uJ whenever v ~ u ~ u ~ wand M, > 0, for each i; (iv) A is an n x n singular matrix such that A + M = C is nonsingular where M is a diagonal matrix with M, > 0 and C- I ~ O. Then there exist monotone sequences {v n }, {w n } such that V n ~ p, W n ~ r as n ~ 00 and p, r are coupled quasi-extremal solutions of (1.1) such that if u is any solution of (1.1), then v :5 P ~ u ~ r ~ w.

Proof. For any YJ, I-t system

E

[v, w]

=

[x

E

Rn:v

~

x

~

w], consider the linear

(5.7.3)

Cu = F( YJ, I-t),

where C=A+M and for each i, Fj(YJ,I-t) =/;(YJk, [YJ]p,,[I-t]q,) + MjYJj. Clearly u can be uniquely defined given YJ, I-t E [v, w] since the matrix C is nonsingular. Furthermore, we have Av ~ F(v, w), Aw ~ F(w, v), and F is mixed monotone. Consequently we can define a mapping T such that T[ YJ, I-t] = u and show easily, using the fact that C- I ~ 0, that (a) v ~ T[v, w], w ~ T[w, v]; (b) T is mixed monotone on [v, w]. Then the sequences {v n } , {wn } with follows:

Vo

= v,

Wo

=w

can be defined as

S.7. Monotone Iterative Methods

149

Furthermore, it is evident from the properties (a) and (b) that {vn } , {w n } are monotone such that Vo ~ VI ~ ••• ~ vn ~ Wn ~ ••• ~ WI ~ Wo0 Consequently, lim e, = p, lim Wn = r exist and satisfy the relations n-JoOO

n-JoCO

By induction, it is also easy to prove that if (u l , U2) is a coupled quasisolution of (5.7.1) such that v ~ UI, U2 ~ w, then V n ~ ul , U2 ~ Wn for all nand hence (p, r) are coupled quasi-extremal solutions of (5.7.1). Since any solution u of (5.7.1) is a coupled quasi-solution, the conclusion of the theorem follows and the proof is complete. • The case of most interest is when p

=

r, because then the sequences {vn },

{w n } constitute lower and upper bounds for the unique solution of (5.7.1).

The following uniqueness result is thus of interest. Corollary 5.7.1. for each i, that

If, in addition to the hypotheses of Theorem 5.7.1, we assume (5.7.5)

where B is an n x n matrix such that (A - B) is nonsingularand (A - B)-I O. Then u = p = r is the unique solution of (5.7.1) such that v ~ u ~ w. Proof.

Since p

~

r, it is enough to prove that p

2::

2::

r. We have by (5.7.5)

[A(r - p)]j ~j;(ri' [r]pi' [p]q) - j;(Pj, [P]Pi' [r]q) ~ [B(r - p)]j.

Consequently, (A - B)(r - p)

~

0, which implies r ~ p.

•

Remark 5.7.1. If qi = 0 for each i, then Theorem 5.7.1 shows that r, pare maximal and minimal solutions of 5.7.1 and Corollary 5.7.1 gives the unique solution.

If f does not possess the mixed quasi monotone property, we need a different set of assumptions to generate monotone sequences that converge to a solution. This is the content of the next result. Theorem 5.7.2. Assume that (iv) of Theorem 5.7.1 holds. Further, suppose that V, W E R" such that v ~ W, Av

~f(v)

- B(w - v),

-B(x - y)

~f(x)

Aw 2::f(w) - f(y)

~

+ B(w -

B(x - y)

v),

(5.7.6) (5.7.7)

150

5.

Applications to Numerical Analysis

whenever v ~ y ~ x ~ w, B being an n x n matrix of nonnegative elements. Then there exist monotone sequences {v n }, {w n } that converge to a solution u of (5.7.1) such that

provided that (A - B) is a nonsingular matrix. Proof.

We define F(y, z)

= ![f(y) + fez) + B(y -

z)).

(5.7.8)

It is easy to see that F(y, z) is mixed monotone and

-B(z - z)

~

F(y, z) - F(y, z)

whenever z, z, y, Y E [v, w] and i

~

~

B(y - y)

(5.7.9)

z, Y ~ y. In particular, we have

F(y, z) - F(z, y)

= B(y -

z).

(5.7.10)

From (5.7.8) we obtain -B(w - v) ~ F(v, w) - f(v), which yields Av ~ f(v) - B(w - v) ~ F(v, w).Similarly Aw ~ F(w, v). Finally, it follows from (5.7.8) that F(x, x) = f(x). Consider now for any Tl, JL E [v, w], the linear system given by Cu = O( Tl, JL) where C = A + M, M being the diagonal matrix with M; > 0 and for each i, OJ( Tl, JL) = F; ( Tl, JL) + MjTli' Proceeding as in proof of Theorem 5.7.1, we arrive at Ap = F(p, r), Ar = F(r, p). Using (5.7.9), we see that A(r - p) = F(r, p) - F(p, r) = B(r - p) and this implies u = r = p is a solution of (5.7.1). The proof is complete. • Corollary 5.7.2. If in addition to the hypotheses of Theorem 5.7.2 we also have f(x) - fey) ~ C(y)(x - y) for x, y E [v, w], where C(y) is an n x n matrix such that [A - C(y)] is nonsingular and [A - C(y)r 1 ~ 0, then u is the unique solution of (5.7.1).

Proof.

If ii is another solution of 5.7.1, we get A(u - ii) = feu) - f(ii)

which yields u (5.7.1). •

~

ii. Similarly, ii

~

~

C(ii)(u - ii),

u and hence u is the unique solution of

Equations of the form (5.7.1) arise as finite difference approximations to nonlinear partial differential equations as well as problems at resonance where the matrix is usually singular, irreducible, M-matrix. For such matrices the fol1owing result is known.

5.8.

151

Problems

Theorem 5.7.3. Let A be an n x n singular, irreducible, M-matrix. Then (i) A has rank (n - 1); (ii) there exists a vector u > 0 such that Au = 0; (iii) Av ~ 0 implies Av = 0; and (iv) for any nonnegative diagonal matrix D, (A + D) is an M-matrix and (A + D)-t is a nonsingular M-matrix, if d jj > 0 for some i, 1 ~ i ~ n. As a consequence of Theorem 5.7.3, if we suppose that A in (5.7.1) is n x n singular, irreducible, M-matrix, then (A + M) is a nonsingular M-

matrix. Therefore assumption (iv) of Theorem 5.7.1 holds since nonsingular M-matrix has the property that its inverse exists and is greater than or equal to zero. Furthermore, in some applications to partial differential equations, the function f in (5.7.1) is of special type, namely feu) = (ft(Ut),/zCU2),'" '/n(u n». We can find in this special case lower and upper solutions such that assumption (ii) of Theorem 5.7.1 holds whenever we have lim sup};(uJ Sig(uj) < 0 IuJ... co

for each i.

In fact, by Theorem 5.7.3, there exists a ~ > 0 such that Ker A = and hence we can choose a A > 0 so large that f(A~) ~

0

Span(~)

and f( -A~) ~ O.

Letting v=-A~, w=Ag, we see that v~w, Av~f(v) and Aw~f(w). Assumption (iii) is simply a one-sided Lypschitz condition and in assumption (5.7.5), B is a diagonal matrix.

5.8. Problems 5.1.

Show that p( G'(x*» = 0 where G(x) is a simple root of F(x) = O.

5.2.

Show that if F: R S

=x -

(F'(X»-l F(x) and x*

R has a Lypshitz derivative then

~

IIF(x) - F(y) - F'(x)(y -

x)11 ~ hllx - y112.

5.3.

Show that if in the Newton-Kantarovich theorem one supposes IIF'(x)-tll bounded for all x E R S , then [x, - x*II ~ cllxn - 1 - x*II2•

5.4.

Show that in the hypothesis of Theorem 5.3.4, the error satisfies the inequalities (5.3.12) and (5.3.13).

5.5.

Suppose that

x, E R

S ,

n

~ 0 and Il.Ilxnll ~

n

.Ilt

.Il

n- t

lI.1lxn-t1I, where

a positive converging sequence with to = 0 and lim tn n~CO

that

Xn

converges to a limit x".

= r:

tn is

Show

152

5.

Applications to Numerical Analysis

IlllxnI s; ( Iltn)'Y (1IIlxn-lll)Y Ilt

5.6.

As in the previous exercise suppose that

5.7.

Show that the solution of

5.8.

Find the constant k in the previous problem and deduce that for Newton's method one has:

Zo -

n- I

~z~

IS given by Zn = 1 - Zn 1 - (1 - 2zo) 1/ 2 cotght cz"), with k appropriately chosen.

'I

X

*-

Xn

Zn+1

I s; -(1 2 f3y

=

2'

2zo)1/2 - -(J2 - " 1 - (J

where 1 - Zo - (1 - 2Z0) 1/ 2 (J = ---=--'-----=..:--:-= 1 - Zo (1- 2Z0 ) 1/ 2 '

+

5.9.

Obtain the result of Theorem 5.3.3 by considering the first integral with Zo = ~. (Hint: in this case the first order equation becomes Zn+1 = ~(1 + zJ.)

5.10. Consider the iterative method

= JL[AG(x n ) + (1- A)G(xk, Xk-I)] + (1- JL)Xk, A, JL E [0,1] and G, G are defined as in Theorem 5.3.5. Show Xk+1

where that JL and A can be chosen such that the convergence becomes faster. 5.11. Solve equation (5.5.1) directly and show the growth of the errors. 5.12. Obtain the relations (5.5.2), (5.5.3) and (5.5.4). 5.13. Show that the sweep method is equivalent to the Gaussian elimination of the problem Ay = g when the problem 5.5.1 is stated in vector form, see (5.4.16).

5.9. Notes The discussions of Sections 5.1 to 5.3 have been introduced only to show examples of application of difference equations to numerical analysis. More details can be found in the excellent books by Ortega and Rheinboldt [127] and Ostrowsky [132]. Theorem 5.3.3 is a simplified version of the original one, see [129], [127], while the estimates given before 5.3.4 (b) seem to be

5.9.

Notes

153

new. For other estimates of the error bounds for the Newton's method and Newton-like methods, see also [41], [109], [111], [179], [144]. For the effect of errors in the iterative methods, see Urabe [168]. A very large source of material on the problem discussed in Section 5.4 can be found in Gautschi's review paper [52]. Theorem 5.4.1 has been taken from Olver [123]. A detailed analysis, in a more general setting, of the Miller's algorithm is given in Zahar [180]. Applications to numerical methods for ODE's can be found in Cash's book [23], while a large exposition of applications as well as theoretical results are in Wimp's book [56], see also Mattheji [104]. The "sweep" method can be found in the Godunov and Rayabenki's book [59], where it is also presented for differential equations. An improvement of the Olver's algorithm can be found in [169]. An application of the sweep method to the problem of Section 5.4 can be found in Trigiante-Sivasundaram [167]. The contents of Section 5.6 are adapted from [89, 126, 127], while the material of Section 5.7 is taken from [91], see also [85, 87, 89].

CHAPTER 6

6.0.

Numerical Methods for Differential Equations

Introduction

Numerical methods for differential equations is one of the very rich fields of application of the theory of difference equations where the concepts of stability playa prominent role. Because of the successful use in recent years of computers to solve difficult problems arising in applications such as stiff equations, the connection between the two areas has become more important. Furthermore, in a fundamental work Dahlquist did emphasize the importance of stability theory in the study of numerical methods of differential equations. We shall consider in this chapter some of the most relevant applications. In Section 6.1 we discuss linear multistep methods again and show that the problem can be reduced to the study of total or practical stability when the roundoff errors are taken into account. In Section 6.2 we deal with the case of a finite interval where we shall find a different form of Theorem 2.7.3. Section 6.3 considers the situation when the interval is infinite restricting to the linear case. The nonlinear case when the nonlinearity is of monotone type is investigated in Section 6.4, while in Section 6.5, we show how one can utilize nonlinear variation of constants formula for evaluting global error. Sections 6.6 and 6.7 are devoted to the extension of previous results to partial differential equations via the method of lines, which leads to the consideration of the spectrum of a family of matrices in order to obtain the right stability conditions. The problems given in Section 6.8 complete the picture.

155

156

6.

Numerical Methods for Differential Equations

6.1. Linear Multistep Methods We have already seen that a linear multistep method (LM), which approximates the solution of the differential equation y'

= f(t, y),

(6.1.1)

is defined by

p(E)zn - hu(E)f(tn, zn) = 0,

(6.1.2)

where p(E) and u(E) are defined in section 2.7. For convenience we shall suppose that f is defined on IR X IR and it is continuously differentiable. The general case can be treated similarly but for some notational difficulties. The values of the solution y( t) on the knots tj = to + jh satisfy the difference equation

p(E)y(tn) - hu(E)f(tn, y(tn))

= Tn,

(6.1.3)

where Tn is the local truncation error. The global error

In

=

y(tn) - z;

(6.1.4)

satisfies then the difference equation

p(E)ln - hu(E)[f(tn, z; + In) - f(tn, Zn)]

= Tn·

(6.1.5)

In practice, equation (6.1.2) is solved on the computer and instead of (6.1.2) one really solves the perturbed equation (6.1.6) where en represent the roundoff errors. The equation for the global error then becomes (6.1.7) where (6.1.8) It is worth noting the different nature of the two kinds of errors. If the method is consistent Tn depend on some power of h, which is greater than one, while en is independent on such a parameter (it depends on the machine precision). For simplicity, we assume that en is bounded. (We note that en can grow as a polynomial in n, see problem 1.14). The equation (6.1.7) can be considered as the perturbation of the equation

(6.1.9) which has the trivial solution In = 0. The problem is then a problem of total stability or of practical stability if the roundoff errors Cn are taken in account. One needs to study equation

157

6.1. Linear Multistep Methods

(6.1.9) and verify whether the zero solution is uniformly asymptotically stable and then apply, for example, a theorem similar to 4.11.1 in order to get total stability. This has been done indirectly in the linear case. Recently some efforts have been devoted to certain nonlinear cases. The techniques used to solve the problem are different. One method is to use the Z-transform (with arguments similar to those used in the proof of Theorem 2.7.3 (See for example [117]) and the other is to transform the difference equation of order k to a first order system in R". We shall follow the latter approach. Let us put f(tn, Zn

+ In) -

(6.1.10)

f(tn, Zn) = cnln·

Equation (6.1.9) can be written (we take ak = 1) as In+k - h{3kcn+k1n+k

k-I

+I

j=O

(aj - h{3jCn+j)ln+j =

W n,

where b\n) = {3jCn+j - {3k a jcn+k J 1 - h{3k Cn+k .

By introducing the k-dimensional vectors En

= (1m, In+l, ... , In+k-I) T,

Wn = b(n)

=

(0, 0, ... ,1 -

(6.1.12)

hWn ) T (3kCn+k b(n) = (b~n), b~n), •.. , b~~I)T;

and the matrices

o o

1

o

o o

A=

o

o

1

(6.1.13)

IS8

6.

Numerical Methods for Differential Equations

° Bn =

(6.1.14)

equation (6.1.7) reduces to the form E n+ 1

=

(A + hBn)En + Wn.

(6.1.15)

The problem is now to study total stability for the zero solution of this equation, which depends crucially on the nonlinearity off contained in the matrix Bn • Historically this problem has been studied under some simplifying hypothesis that we shall summarize as follows: finite interval, infinite interval with f linear, infinite interval with f of monotone type.

6.2.

Finite Interval T

If the interval of integration (to, to + T) is bounded, one takes h = Nand

n = 0, 1, ... ,N This implies that h becomes smaller and smaller as N increases and the quantitiy hBnEn, like the term Wn, can be considered as small perturbation of the equation (6.2.1) The eigenvalues of the matrix A are roots of the characteristic polynomial p(z) and the requirement of the stability of the null solution of (6.2.1) is equivalent to O-stability. We cannot assume asymptotic stability of the zero solution and apply theorem 4.11.1 because one of the conditions of consistency impose p(l) = 0, which means that at least one of the eigenvalues of A is on the boundary of the unit disk in the complex plane. The analysis of the stability of the zero solution of (6.1.15) needs to be done directly. This is not difficult if we use (4.6.2). In fact n-l '\' An-j-l(W) + hE-E.) E n = AnE0 + L. (6.2.3) )). j~O

Taking the norms, we get

liEN I ~ IIA NII I Eoll + I IIA N N-l

- j -

j~O

111(II"'ili

+ hIIBjIIIIEJ).

(6.2.4)

Now suppose that the zero solution of (6.2.1) is stable (that is the same to demand that the method is O-stable). This means that the spectral radius

159

6.2. Finite Interval

of A is one and moreover the eigenvalues on the boundary of the unit circle are simple. It follows then (see Theorem 12.4) that the powers of A are bounded. Let us take for simplicity that IIAjl1 = 1,j = 1,2, ... , N, (see also Theorem A5.2). The inequality (6.2.3) becomes

N-l liEN I ~ IIEol1 + j=O L (II "'ill + hIIBjIIIIEJ).

By Corollary 1.6.2 we obtain liEN I

~ I Eoll ex p( h ;tolIIBjll) + JOl I "'ill ex p( h T:~:l IIBTII) ~

IIEol1 e

NLh

+

N-l

L I "'ill

(6.2.5)

ehL(N-j-l),

j~O

where L

=

max

}:$T::sN

IIBTII. We let max

O::SJsN

I "'ill = 8 + r(h),

(6.2.6)

Then (6.2.5) becomes e

LT

-1

liEN I ~ IIEol1 e L T + e hL -1

(8 + r(h)).

(6.27)

Usually the initial points are chosen such that lim

N->oo

Hence, taking the limit as h lim liEN I h->O

~

IIEol1 = o.

(6.2.8)

0, we have

~ (e L T

-1) lim h->O

e

e

~L r(h), - 1

from which it is clear that if 8 ,e 0 (the roundoff errors are considered), the right-hand side is unbounded. It is possible to derive for I EN I a lower bound with a similar behavior. This means that the zero solution is not practically stable. If the roundoff errors are not considered, and if the method is consistent (see definition 2.7.1) then 8 = 0 and r(h) = O(h P + l ) , P ;::: 1 and therefore lim h->O

IIEnl1 = 0

(6.2.9)

and this implies that the null solution is totally stable. It is possible to give a more refined analysis of (6.2.3) by considering in the limit only the components of the matrix A (see Appendix A), corresponding to the eigenvalues on the unit circle, that have nonzero limit for n ~ 00 (see problem 6.1).

160

6.

Numerical Methods for Differential Equations

This analysis allows us to weaken the conditions that we impose on the local error. If, as usually happens in the modern packages of programs designed to solve the problem, one allows the stepsize to vary at each step, the difference equation for the global error is no longer autonomous even for autonomous differential equations. Gear and Tu obtain the difference equation (6.2.10) where the matrix S(n) takes into account many other factors that define the method. It is shown that (6.2.11) Suppose that the two conditions hold: (1) (2)

1I
where
EN

=


j=no


+ "'1],

and hence it follows that N-I

IIENII < kollEol1 + ko j=no I (hjklllEJ + I "'111)· By Corollary 1.6.2 we obtain

IIENII :5 kollEol1 exp( kok l :~~ hj) + k o :~~ I Wsil exp( kok l j:~:1 hj) :5

k o eV1T[IIEoll

from which follows that if method is convergent.

+ :~~ I Wsll]'

I Eoll

and I:~~ I W

S

I

tend to zero as h

--'»

0, the

6.3.

Infinite Interval

161

6.3. Infinite Interval In practice one uses methods with h bounded away from zero. This means that it is not true that h ~ 0 as n-tends to infinity. The term hBnE n cannot be considered as a perturbation, especially when the norm II B; II is very large (stiff equations). For such problems it is necessary to consider the quantity hBnE n as a principal component of equation (6.1.15). Because h is taken either fixed or bounded away from zero, the study of the qualitative behavior of (6.1.15) as n ~ 00 implies that the interval of integration is infinite. When the term hli.B; is taken in account, the equation is no longer linear because the elements of the matrix B; depend on the solution. The problem becomes a difficult one. It has been studied extensively in the linear case f(y) = Ay when ReA < O. In this case, the matrix B; becomes independent on n, namely,

A

-T

BA..b - 1 - hAf3k 'I'k

,

(6.3.1)

where (6.3.2) and the equation (6.1.15) becomes (6.3.3) whose solution is, taking no = 0,

En

=

ere; +

n-l

I

C n- j + 1 "'} .

(6.3.4)

j=no

The behavior of En will be dictated by the eigenvalues of the matrix C, which are the roots of the polynomial 7T(Z, q) defined in Section 2.7. The set D of values of q = hA for which the zero solution of the unperturbed equation (6.3.5) is stable is said to be the absolute-stability region of the method. As in the previous case, since the roundoff errors Ej are only bounded it is more desirable, according to Theorem 4.11.1 to require the asymptotic stability of En = 0 in (6.3.5). This implies that q must be strictly inside the absolute stability region. A very large effort has been made to study the absolute stability for different Cij and f3j, that is, for different classes of methods.

162

6.

Numerical Methods for Differential Equations

Special importance is given to methods for which the absolute stability region is unbounded and contains the complex left half plane (A-stable methods) because in this case no restrictions are to be imposed on the parameter h in order to restrict h): to be inside that region. We cannot go into the details of this work. We are only interested in the techniques used relative to difference equations. Recently the case of nonautonomous linear equations has also been discussed. This is very important because in the modern packages of software to solve differential equations, one allows the step-size to vary at each step. In this case, the difference equation is no longer autonomous and the considerations based on the eigenvalues of the matrices C(n) are no longer enough to ensure the asymptotic stability (compare the counter example given in Section 4.4). One need to impose some conditions on the norms of matrices II Cn II in order to get information on the fundamental matrix <1>(n, no) of the equation (6.3.6) Once this has been achieved, we can use the formula (4.6.5) and obtain En = <1>(n, no)Eo +

n-I

I

j=no

<1>(n,j + 1) «j.

(6.3.7)

Suppose that the zero solution of (6.3.6) is uniformly asymptotically stable, then by Theorem 4.2.2, there exist a, TJ > 0 and TJ < 1 such that (6.3.8) It then follows that

liEn II -s a77"-"oIIEoll + a

n-I

I

77"-j- 1 11«jll,

j=no

from which one derives at once the stability of En = 0 for the perturbed equation. We have treated so far the case of a single equation. The case of autonomous systems y'

= Ay,

y(to) = Yo,

(6.3.9)

can be treated similarly. In fact, suppose that the matrix A can be reduced to the diagonal form by means of a similarity transformation A = T-1AT where A = diag(A 1 , A2 , ••• , A). Then the system (6.3.9) reduces to uncoupled scalar equations and the condition becomes that each hs, must be inside the absolute stability region.

163

6.4. Nonlinear Case

6.4.

Nonlinear Case

The study in the nonlinear case of equation (6.1.15) is difficult and has been made for only nonlinearities of a special kind, namely, monotone. The techniques used require essentially to bound norms of the matrix A + hBn. We shall be content to present only the case of the implicit Euler method and the one-leg methods, since they are good examples of the use of the equations employed here. Let f: R x R ~ R and suppose that for u, v E R, S

S

(6.4.1) with p., :::; O. The difference equation (6.1.15) becomes in this case In+1 - In - h[f(tn+l'

Zn+l

+ In+1) - f(t n, zn)] == W n·

(6.4.2)

By multiplying both sides by 1~+1 we have

II In +1 11 2 -s 1~+1 . In + hu. Il/n+t11 2 + 1~+1 W n (6.4.3) where the Cauchy-Schwarz inequality (6.4.4) has been used. From (6.4.3) one gets

Il/n+t11 :::; 1 _

1 hp., II In I

+1_

1 hp.,

II W n II·

(6.4.5)

By using Corollary 1.6.1, we then obtain (6.4.6) Since the quantity TJ ==

~h is less than one by hypothesis, the 1 - p.,

uniform

stability of the solution In == 0 follows. The previous arguments can be generalized to the class of methods defined by (6.4.7) This class of methods was proposed by Dahlquist who gave the name of one-leg methods. There is a connection between the solutions of the difference equation (6.1.2) and the solutions of (6.4.7) (See problems 6.6,

164

6.

Numerical Methods for Differential Equations

6.7). For the sake of simplicity, we shall suppose f: R x R ~ R. In this case the quantities in (5.4.1) are scalars. The error equation for (6.4.7) is p(E)ln - h[f«(J"(E) tn, (J"(E)zn

+ (J"(E)ln) - f«(J"(E)t n, (J"(E)zn)] = W n· (6.4.8)

Multiplying by (J"(E)ln one obtains (J"(E)ln' p(E)ln ~ hP,I(J"(E)lnj2 + W n ' (J"(E)ln'

Let us switch now to the vectorial notation (See 6.1.12) and suppose that there exists a symmetric positive definite matrix a such that is the Lyapunov function satisfies the inequality (6.4.9) It then follows that

Vn+ 1 - Y n ~ 2(J"(E)ln' p(E)ln 2

~ 2hp,1(J"(E)ln 1

+ 2Iwn(J"(E)lnl

2+ 2I(J"(E)lnllwnl.

~ 2hp,1(J"(E)ln 1

By considering that (here 1
=

11·11: is used.)

1(0,0, ... , /3dEn+1 + (/30, /31' ... , /3k-I)Enl

~ (to /3 i) 1/2(IIEn+11I + liEn II),

k 2)1/2 . ("\' we h ave, b y setting f..,i=Of3i

Vn + 1 - Vn

--

~. 2'

~ <5(IIEn+111 + IIEnll)lwnl +~<52p,(IIEn+111 + IIEnll)2. v (6.4.10)

Since

IIEnl1 ~ Ila-1/2111Ial/2Enll = lIa-' /211V~/2, the

Vn + 1 -

If

v, ~ <5IIG-I/211(V~~1 + v~/2)lwnl + h; <5

<5lla- I / 211 = a, one

relation becomes

21Ia- I 2 2 / 11

obtains

I 2 / V n+l _ y ln/2

< -

h a2 alwn 1+~(VI/2 2 n+l

+ V n1/2)

(V:!; , + V:!2)2.

165

6.5. Other Techniques

from which follows the estimate

I/ 2 V n+1

<

-

alwnl 1- hJ.La

2

2

+

2 rlIa _ 1 + h_ 2

1- hJ.La 2

2

1/2

Vn

•

Therefore,

V~/2 (1+ h~)"-~ V, $

1 - h J.L;

+

a,

"f (1+ h~)"-HIWA'

1 - h J.L; j~O

1 _ h J.L;

which gives a bound for liEn 110' The existence of the matrix G can be established if the method is A-stable. See Dahlquist [36].

6.5 Other Techniques The formula (4.6.16) can be used to evaluate the global error. In fact if x(n, no, xo) is the solution of the unperturbed problem corresponding to (6.1.2) and yen, no, xo) is the solution of the perturbed problem corresponding to (6.1.3), then the difference En = yen, no, xo) - x(n, no, xo) defined by (4.6.17) gives the global error. We have already used this result in the linear case (see (6.2.3)). In applications one uses a similar formula obtained by using the method of variation of parameters of the original differential equation. Theorem 6.5.1.

Let

(1) yet) be the solution of (6.1.1);

(2) Yo = Zo, ZI,"" z; be some approximation of yet) at the points to,t1 , ••• ,tn; (3) pet) be a sufficiently smooth function that interpolates the points Zi so that p(tJ = z, i = 0,1, ... , n. Then En = y(tn) - z; satisfies the relation En

=

-f

In


to

where
ayet, to, x o) . axo

(6.5.1)

166

6. Numerical Methods for Differential Equations

Proof. Let s E [to, tnJ and y(t n, S, p(s)) be the solution of (6.1.1) with initial value (s, p(s)). It is known in the theory of differential equations that ay(tn, s,p(s))

= -(tn, s, p(s) )f(s, p(s)),

-"---'-..:.:.:..--~-.:..:..

as

and ay(tn, s,p(s)) = m(t

ap

'¥

n, S,

( ))

P s .

Consider the integral

This integral also is equal to

f

In

[ay(tn, s, p(s))

as

to

=

J

In

+ ay(tn, s,p(s)) pl(S)] ap

ds

(tn, s, p(s) )[p'(s) - f(s, p(s))] ds,

10

which completes the proof.

•

The formula (6.5.1) can be used to obtain methods with higher order, to find the order of known methods and to study new methods. See [140, 171J.

6.6.

The Method of Lines

Consider the following problems au(x, t)

a2u(x, t)

at

ax

--'---'- =

2'

u(O, r) = u(l, t) = 0, u(x, 0) = g(x),

(6.6.1)

and au(x, t) at

au(x, t)

----'-~, u(O, t)

ax

= 0,

u(x, 0)

= g(x),

x> O.

(6.6.2)

Let us discretize the interval (0,1) by taking Xi = ih, i = 0, 1, ... , N + 1 and consider the vectors U(t) = (u(XI> t), ... , U(XN' t))T, G = (g(xd, ... , g(XN)) T, where u (Xi, t) are approximations of the solution along 2

the lines x =

Xi'

We approximate the operators

!.., and ax-

~

ax

by central

6.6. The Method of Lines

167

differences and backward difference respectively. By introducing the matrices

AN =

-2 1 0

0

1 -2

1

0

0 ... 0

0

0 1 -2 NxN

0

(6.6.3)

and 0 1

1 -1 BN =

0

0

(6.6.4)

0

-1 1 NxN,

0

the two problems can be approximated by 1

dU

dt = ax 2 ANU, U(O) = G,

(6.6.5)

and 1

dU

dt = - ax BNU,

U(O)

= G.

(6.6.6)

We have approximated the two problems of partial differential equations with two problem of ordinary differential equations. If we discretize the time, using for example, the explicit Euler method and try the results on absolute stability, we get two different results, namely, one correct for problem (6.6.1) and one wrong for problem (6.6.2). In fact, the eigenvalues of AN and B N are

ksr IJ-k

= 1, k = 1,2•... , N,

(6.6.7) (6.6.8)

and the region of absolute stability for the explicit Euler method is D

=

{z

E

c liz + 11 :S I}.

(6.6.9)

168

6.

Numerical Methods for Differential Equations

In order to have stability of the method we must have /).t

Ak :0; 0

(6.6.10)

-2:0; - - Jl.k :0; 0

(6.6.11)

-2:0;

-2

/).x

for the first case and /).t

/).x

for the second case. From (6.6.10) and (6.6.7) it follows that the condition of stability, for the first problem is /).t 1 -
(6.6.12)

and for the second one is /).t

< 2. /).x-

(6.6.13)

It happens that (6.6.12) agrees with the Courant, Friedrichs and Lewy condition, while (6.6.13) does not agree with the correct condition, which is /).t

<1 . /).x-

(6.6.14)

The reason of the discrepancy lies in the fact that the ordinary differential equations (6.6.5) and (6.6.6) depend on N, which tends to infinity when /).X tends to zero. This implies that when we study the error equation, for /).X and at tending to zero, the dimension of the space R N and therefore of the matrices AN and EN increases. We shall see in the next section how to obtain the right conditions.

6.7.

Spectrum of a Family of Matrices

Consider a family of matrices {Cn}nEN+' Definition 6.7.1. The spectrum of a family of matrices {Cn} is the set of complex numbers A such that for any e > 0, there exists n E N and x E en, x ;c such that (6.7.1) I CnX - Axil :0; e [x]:

°

The spectrum of {Cn } will be denoted by S( {Cn } ) , or S when no confusion will arise. It is obvious that the set L of all eigenvalues of all matrices of the family is contained in S. We now list some properties of S, without proof.

6.7. Spectrum of a Family of Matrices

169

Proposition 6.7.1.

S is a closed set.

Proposition 6.7.2. closure of ~.

If the matrices are normal, then S = ~, where ~ is the

Proposition 6.7.3. then

IfPis a compact set ofthe complexplane and ifP sup

nEN+ zeP

IIUz -

Cn)-lll <

1\

S

=

cP,

(6.7.2)

00

Proposition 6.7.4. Let S be compact and Sen where n is an open simply connected set of C. Moreover let f be an analytic function in n. Then (6.7.3) Proposition 6.7.5.

Let

q = sup IAI.

(6.7.4)

Ae5

If q < 1, one has for every n, mE N+

IIC;:'II
(6.7.5)

where k is independent on m and n. Let us now find the spectrum of the family {Cn } : a f3

o Cn

o

'Y 0 a 'Y

=

o

(6.7.6)

'Y

o

Of3

where a, x , 'Yare real and we shall suppose for simplicity that 1f3/ 'YI < 1. Let be Q = S\~. The elements of Q will be called quasi-eigenvalues.

Theorem 6.7.2. The set of complex numbers A such that the zero solution of the difference equation 'YXi+l

+ (a

- A)xi

+ f3Xi - 1 = 0 A-a

X2=--

'Y

is asymptotically stable, is contained in S( {Cn }) .

(6.7.7)

170

Proof.

6.

Numerical Methods for Dilferential Equations

The characteristic polynomial of (6.7.7) is yr

2

+ (a

-

A)r + {3

(6.7.8)

= 0

and to get asymptotic stability it must be a Schur polynomial (see Theorem 2.6.1). Suppose it is, then taking as vector x the vector made up with the first n components of the solution of problem (6.7.7), one has CnxAx = (0,0, ... , YXn+I)T. It follows that IICnx - Axil = lyllxn+11 = lyllclr~+1 + c2r~+11, where rl and r2 are the roots of (6.7.8) which, for simplicity, are supposed simple and C I, C2 are deduced from the initial conditions, that is

We then get 1 Ilr~+1 + r~+1lllxll, rl - r2 where we have considered that Ilxll;;:: 1. The quantity multiplying [x] can be made arbitrary small if Irll and Ir21 are less than 1. Similar considerations can be made if Irll = hi. One can show that if Irll or Ir21 are greater than

/ICnx -

Axil :5

1

1, the corresponding A is no part of the spectrum and A ~ 6.9). •

~

(see Problem

The next theorem explicitly furnishes the boundary of S. Theorem 6.7.2.

The boundary of S is given by

x = a + ({3 + "I) cos where A

= x + iy, and

0 :5 e

e,

y = ("I -

(3) sin

e,

(6.7.9)

:5 7T.

Proof. The proof reduces to find for which values of A the polynomial (6.7.8) is a Schur polynomial. This can be done easily by using Theorem Bl of Appendix B, obtaining the conditions (6.7.9). •

The set S has then as boundary, the ellipses with center (a, 0) and semiaxes ({3 + "I) and ("I - (3). By considering that the eigenvalues Akn ) of the matrices C; are given by . Akn ) = a + 2«(3"1)1/2 cos -br - , It follows that Akn ) E S.

n+l

Particular cases are: (a) a = -2, (3 = "I = 1; in this case the family {Cnl coincides with the family {An} defined by (6.6.3). The spectrum S({An}) is then the segment 0:5 e:5 7T. -2 + 2 cos e, (6.7.10)

171

6.7. Spectrum of a Family of Matrices

(b) ex = 1, f3 = -1, 'Y = 0; in this case the family {en} coincides with the family {En} defined by (6.5.4). The spectrum S( {En}) is then the circle defined by x

=

1 ~ cos 8, Y = -sin 8.

(6.7.11)

Once the spectrum of the family has been obtained, the correct stability conditions can be obtained. If the case where the ordinary differential equations are discretized by using the Euler method for example, instead of (6.6.10) and (6.6.11), one imposes !:!..t !:!..x 2 S({A n } ) cD

(6.7.12)

for the first problem and (6.7.13) for the second problem. In fact we have: Theorem 6.7.3. Suppose that the problems (6.6.5) and (6.6.6) are solved with the explicit Euler method. If (6.7.12) and (6.7.13) hold, then the resulting difference equations are stable. Proof. Let VB s = 0, 1, ... be an approximation of U(ts), the Euler method will produce the difference equations

and Us+ 1 = (I - : ; En) Us

respectively. When (6.7.12) and (6.7.13) hold, by proposition 6.7.3, it follows that the spectrum of the families of matrices

{I + ::2 An} and {I -:; En}

are contained in the unit circle. This implies, by Proposition 6.7.4, that all the powers of the matrices of the families are uniformly bounded and this implies the stability. One should consider, however, that the condition of consistency for the space discretization implies ex + f3 + 'Y = 0 and the origin is a point of the boundary of both S and aD. This implies that (6.7.12) and (6.7.13) cannot be strictly satisfied. For some class of matrices, for example

172

6.

Numerical Methods for Differential Equations

normal matrices, this does not create any problem. Note that condition (6.7.12) is not more restrictive than the one obtained by using only the eigenvalues because in this case, the matrices are normal and the spectrum is only the closure of the set of all the eigenvalues. Condition (6.7.13) is more restrictive because the spectrum is a disk in the complex plane, while the set of eigenvalues is made up of only one point. More generally to each method will be associated a region of absolute stability D defined by the discretization of t and a region S, which is a function of the spectrum of a family of matrices defined by the discretization of the space variables. For the stability the second region must be contained in the first. This approach is also useful because it permits the choice of appropriate discretization of the time as consequence of the discretization of the space variables. •

6.8.

Problems

6.1.

Suppose thatthe matrix A has the eigenvalues 1 = Al > A2 :2: ••• 2= Ak . Using the decomposition (A2.2) of the matrix, give a bound for" En" in (6.2.3).

6.2.

Show that for the Euler method Yn+1 = Yn + hf; the region of absolute stability is the circle with center at -1 and radius 1.

6.3.

Show that the region of absolute stability of the trapezoidal method Zn+1

h

.

= z; +"2 Un + fn+l) IS the

left-hand plane.

6.4.

Find the one-leg correspondent of the trapezoidal method. It is known as the implicit midpoint method.

6.5.

Show that the region of absolute stability of the midpoint method Zn+2 = z; + 2hfn+1 is the segment [-i, i] of the imaginary axis.

6.6.

Suppose that Yn satisfies the one-leg equation p(E)Ynhf(u(E)Yn) = O. Show that Yn = u(E)Yn satisfies the equation p(E)Yn - hu(E)f(Yn) = O. (Hint: the operators p and a commute.)

6.7.

Suppose that {Yn} satisfies the equation p(E)Yn - hu(E)f(Yn) = 0 and P(z) and Q(z) are polynomials satisfying P(z)u(z)Q(z)p(z) = 1. Then show that Yn = P(E)Yn - hQ(E)f(Yn) satisfies the one-leg equation p(E)Yn - hf(u(E)Yn) = o.

6.8.

Suppose that en < -JL (JL > 0) for all n, f3n ,e 0 in (6.1.10) and the roots of u(z) are inside the limit disk. Show that the error en tends to zero.

173

6.9. Notes

6.9.

Show that in Theorem 6.7.1 if one of the roots of (6.7.7) has modulus greater than 1, then A does not lie in Q.

6.10.

Show that for the matrix (6.7.6) it is IICnx - Axil = l,8pn- 1 Un(q)l, where p = (,8h)I/2 and q = (A - a)/2(,8'Y)1/2 and Un(q) is the Chebyshev polynomial of second species. (Hint: use the results of Section 2.3.)

6.11.

Show that the midpoint method cannot be used for the time discretization of 6.6.5.

6.12.

Consider the second order hyperbolic equation Utt - Uxx = 0, U(x,O) = ft x), U(O, t) = u( 7T, t) = 0 and discretize the space varia bl e

0 btaini tammg

0f

a system

first order equations, . dw = -d t

1 D 2 N w,

ax

-2

w(O) = l/J, where w = (U, V) T, l/J = (<1>,0), U, V, <1> ERN, N is the number of discrete points in (0, 7T), and

D (0AN 2N

6.13.

6.9.

=

2 aX

0

I).

Show that D~N is normal and find its spectrum. Using this result, show that the midpoint method can be used in this case.

Notes

The problem of numerical methods for differential equations has been extensively studied in the last 30 years. For important works in this field see Henrici [74, 75] and Dahlquist [33]. The study of nonlinear case was also initiated systematically by Dahlquist [39,34,35,36,37,38]. Important contributions have also been made by Odeh and Liniger [11], Nevanlinna and Liniger [118], Nevanlinna and Odeh [117]. Forthe class of Runge-Kutta methods, not treated in this book, see Butcher [20], Burrage and Butcher [17], [18]. For applications of the theory of difference equations to methods for boundary value problems see Mattheij [105]. Material on the spectrum of family of matrices can be found in Bakhvalov [10], Godunov and Ryabencki [59], Di Lena and Trigiante [45]. A general treatment in Hilbert space setting of the spectrum of infinite Toeplitz matrices can be found in Hartman and Wintner [72], as well as in the Toeplitz original papers [165]. Applications of the enlarged spectrum of symmetric tridiagonal matrices (Jacobi matrices) are usual in the theory of orthogonal polynomials, see for example Mate and and Nevai [101] and the references therein.

CHAPTER 7

7.0.

Models of Real World Phenomena

Introduction

In this chapter we offer several examples of real world models to illustrate the theory developed, to show how versatile difference equations are and to demonstrate how often the results of discrete models can explain observed phenomena better when compared to continuous models. Furthermore, the contents of this chapter are also intended to help the practitioners who can look first at the examples and the proceed to read necessary theory of difference equations. Section 7.1 deals with linear models of population dynamics, sketches the development and discusses the appearance of population waves. We devote Section 7.2 to the study of nonlinear models of population dynamics including bifurcation and chaotic behavior. In Section 7.3, we consider models arising in the distillation of a binary ideal mixture of two liquids while in Section 7.4, we discuss models from theory of economics. Models from queueing theory and traffic flow are investigated in Section 7.5 and in Section 7.6 we collect some interesting examples dealing with various models.

7.1.

Linear Models for Population Dynamics

The dynamics of populations have been extensively studied since Malthus. The object of study is the evolution in time of a group of individuals who follow the simplest and essential life events: they are born, mature, reproduce

175

176

7.

Models of Real World Phenomena

and die. The group can be of bacteria or of animals or of humans. For simplicity, in all the models one usually considers only the female part of the population. Both continuous and discrete models have been proposed. More than the law of mechanics, which could have been written in discrete form, it seems, however, not always justified to use in this field continuous models. In fact the usual assumption that the variation per unit time of the number of individuals is small compared to the total number is no longer valid as in mechanics. Often the results of continuous and discrete models are qualitatively similar, but sometimes, especially in nonlinear models they are not and the continuous models are unable to explain observed phenomena (see Section 7.2). Let Yn be the number of females of a population at time t., The simplest model, called after Malthus, Malthusian model, is the following

= Yo,

Y"o

a> 0,

(7.1.1)

which simply states that the number of newborn individuals as well as the number of individuals that are deceased in the time interval tHn is proportional to the number of individuals at time t.: The parameter a is called intrinsic growth rate and is simply the difference between the birth and death rates. The solution of (7.1.1) is Yn = an-n,'Yo and has been found to be very effective to fit exponential data for not large populations. One of the weaknesses of the model is that for n ~ 00, Yn ~ 00, which is impossible due to limitation of the resources. The foregoing simple model does not take into account the age structure of the population, which is important because all of the essential life events depend on the age of individuals. A more refined model is the following (often called Leslie model). Let us divide the lifetime L into M parts and then population in groups (or classes) Yl(n), Y2(n), ... , YM(n) such that the group Yi(n) is made of individuals whose age a, satisfies L L (i -1)-:5 a, < i-.

M

M

Considering the unit time equal to the time spanned by a class, one has that Yi+l(n + 1) is made up of all the individuals in the class i at time n except of those who are deceased (or removed in some way). That is, i

= 1,2, ... , M

where /3i > 0 is the survival rate of the i lh age and M

y\(n

+ 1) = I

i=l

ll'jYi(n),

- 1,

7.1.

177

Linear Models for Popnlation Dynamics

which states the newborns at time n + 1 as sum of the contribution of the different groups each of which is considered to have an homogeneous behavior in the reproduction. Having the human population in mind, one would expect a; to be zero in- the extreme groups and the maximum in some group in the middle. In nature, however, there is a wide range of possibilities; for example a; = 0 for all groups except one, giving rise to different kinds of behavior. By introducing the vector yen) = (Yl(n), ... , YM(n))T and the matrix

A=

al

/32

aM

/31

0

0 (7.1.2)

0 0

0

/3M-1

0

MxM

the model can be written as yen

+ 1) =

(7.1.3)

Ay(n),

where Yo is supposed to be known. The solution is yen) = A"yo (we take no = 0 for simplicity). The behavior of the solution will then follow from the spectral properties of the matrix A. Consider first the case a, > 0 and /3i > 0 for all indices. In this case it is easy to verify that AM> 0 (see sec. A.6). By Perron-Frobenius theorem (see Theorem A.6-2), it follows that A has a real simple eigenvalue AI associated with a positive eigenvector U I• The remaining eigenvalues are inside the circle BA ( in the complex plane. By using the expression (A2.1-6) for the powers of A and considering that in the present case ml = 1, one has A" = A~ [ ZII

+

s mk-1A"-i(n) ] Ik=2 Ii~O ~ . Zk; , AI I

from which it follows that the double sum in brackets tends to zero as n ~ 00. Considering that Zll is the projection to eigenspace corresponding to AI> one has A"yo ~ cA "UI> where c is a constant. The population tends to grow as A~ and is close to U I, that means that the distribution between the age groups tends to remain fixed. In fact, y;(n)

I Yi(n)

U;

(7.1.4)

~--

I

Uj'

;=1

The vector UI is called the stable age vector and rate. Of course the overall population will grow for guish, for AI < 1.

AI AI

the natural growth

> 1 and will extin-

178

7. Models of Real World Phenomena

Special cases of interest are those where some lli can be zero. To make the study simpler, it is useful to change variables. Letting II = 1, h = n~:J Pi (k = 2, ... , M), a, = li ll i (i = 1,2, ... , M), D = diag(lt. 12 " , . , 1M ) , x(n) = D-Iy(n) and B ='D-IAD, one verifies that

o

o

o

o

1

o

which is in the companion matrix (see section 3.3). The model is now x(n

+ 1) = Bx(n),

(7.1.5)

and the characteristic equation of B is then (see 3.3.3): (7.1.6) The following theorem, due to Cauchy, is very useful in this case. Theorem 7.1.1. If ai is nonnegative and the indices of the positive aj have the greatest common divisor 1, then there is a unique simple positive root Al of the greatest modulus. For a proof, see Ostrowski [131]. Since the eigenvector U I of B is = (A~-\A~, •.. , 1)T, it follows that U1 is a positive vector and all the previous results are still valid. When the hypothesis on the indices of Theorem 7.1.1 are not satisfied, then some eigenvalues may have the same modulus of Al giving rise to the interesting phenomenon of population waves. In this case, there are oscillating solutions to the model. To see this in a simpler way, let us consider the scalar equation for the population of the first group. This can be seen from (7.1.5). The first equation of the system gives U\

x\(n

+ 1) =

M

I

aixi(n)

(7.1.7)

i=1

and from the following equations, choosing appropriately the index n, we obtain xi(n) = xl(n - i + 1). By substitutions one has M

x\(n

+ 1) = I ajxl(n - i + 1), i=1

(7.1.8)

7.2. The Logistic Equation

179

which is a scalar difference equation of order M. The characteristic polynomial coincides with (7.1.6). The solution of (7.1.8) in terms of the roots of (7.1.6) is given by (2.3.7) with Ai instead of z.. If there are two distinct roots with the same modulus, say At = p e in 8 , A2 = p e- in8 , one has

+ c2p e- in 8 + other terms. It is possible to find two new constants a and r/J such that xt(n) = c.p" e

C2

= a e-io/J

in 8

and then

Ct =

a e'",

+ r/J) + other terms,

n

x\(n) = 2 pa cos(n6

from which follows that xt(n) is an oscillating function. In the general case more than one period can coexist. In the extreme case a, = 0, i = 1,2, ... , M - 1, aM ,e 0 a number proportional to M of periods can coexist and for large M a phenomena similar to chaos (see next section) may appear. The population waves have been observed in insect populations.

7.2.

The Logistic Equation

The discrete nonlinear model, which we are going to present for the dynamics of populations and which takes into account the limitation of resources, has been used successfully in many areas such as meteorology, fluid dynamics, biology and so on. Its interest consists in the fact that in spite of its simplicity, it presents a very rich behavior of the solutions that permits to explain and predict a lot of experimental phenomena. Let N(n) be the number of individuals at time n and suppose that N (n + 1) is a function of N(n), that is, N(n

+ 1) = F(N(n)).

(7.2.1)

This hypothesis is acceptable if two generations do not overlap. For small N, (7.2.1) must recover the Malthusian model, that is,

F(N(n))

=

aN(n)

+ nonlinear terms.

The nonlinear terms must take into account the fact that when the resources are bounded there must be a competition among the individuals, which is proportional to the number of encounters among them. The number of encounters is proportional to N 2 ( n). The model becomes N(n

+ 1) =

aN(n) - bN2(n),

(7.2.2)

with a and b positive. The parameter a represents the growth rate and b is a parameter depending on the environment (resources). The quantity a] b is said to be the carrying capacity of the environment. For a > 1, this

7.

180

equation has a critical point

N=

a

Models of Real World Phenomena

~ 1 which, as shown below, is globally

asymptotically stable for Yo E (0, 1) and a in a suitable interval. This means that lim N (n) = Nand N is a limit size of the population. N depends on n_oo

a and b, which can vary with the time: b can diminish because the population learns to better use the existing resources or to discover new ones; a can grow because the population learns how to live longer or how to accelerate the birth ratio. Anyway, taking into account the variation of N (which happens with a longer time scale), one sees the evolution of the population can be described as a sequence of equilibrium points with increasing values of N. In case of two similar species, it can be shown that the species with longer N will survive. This is the Darwinian law of the survival of the "fittest." Almost all the previous results are similar to those obtained in the continuous case. A further analysis of the logistic equation shows solutions that are unexpected if one thinks the discrete equation merely as an approximation of the continuous one. To see this, let us change the variable and

simplify the equation. Let Yn = ~ N(n). The model becomes (7.2.3 ) The new variable Yn represents the population size expressed in unit of carrying capacity a/ b. The equation (7.2.3) has two critical points (the term equilibrium points is used more in this context), the origin and ji = (a - 1)/ a, which has physical meaning only for a > 1. For a < 1, by using the theorems of stability by first approximation (see Section 4.7), one sees at once that the origin is asymptotically stable. It can be shown (see problem 7.3), that it is asymptotically stable for Ao in the interval (0,1). For a > 1, the second critical point becomes positive and the origin becomes unstable. The stability of ji can be studied locally by the linearization methods using again one of the theorems of Section 4.7. In fact Yn+l - ji = f'(ji)(Yn - ji)

+f'~) (Yn -

ji)2

= (2 -

a)(Yn - ji) - a(Yn _ ji)2.

If the coefficient of the linear term is less than one in absolute value then ji is asymptotically stable.

As a consequence, one has that for 1 < a < 3 the point ji is asymptotically stable. For a = 2, the solution can be found explicitly (see Problem 1.2.7). For a > 3, a new phenomenon appears. There exists a couple of points Xl and X 2 such that (7.2.4)

7.2. The Logistic Equation

The couple follows that

Xl

and

181 X2

is called a cycle of period two. From (7.2.4) it (7.2.5)

that is,

Xl

and X2 are fixed points of the difference equation (7.2.6)

One can determine the fixed points by using the Equation (7.2.5), which is a fourth degree equation (it contains as roots the origin and y, which are circles of any period). The following theorem permits us to simplify the problem. Let f[x, y] be the usual divided difference, that is, f[ X,Y ]

= f (x ) - f (Y ) . X-Y

Theorem 7.2.1. The solutions ofperiod two for the difference equation Yn+1 f(Yn) exist iff there exist values ofx E (0,1) such thatf[s,f(x)] =-1.

Proof.

=

By definition f[ x, f( X )]

= f ( x ) - f (2)(X) X _ f(x) .'

(7.2.7)

If f(2)(X) = X then (7.2.7) is equal to -1. On the other hand, if (7.2.7) is • equal to -1, then it follows at once that X = f(2)(X). Applying the previous results one obtains in the present casef[x,f(x)] == a(l - x - ax + ax 2) = -1. This equation has real roots for a 2: 3. It can be shown, for example, considering the linear part of f(2)(X), that the circle is asymptotically stable for values of a in a certain range, while y becomes unstable. One says in this case that the solution y bifurcates in the circle of order two. As a result, it follows that if y is greater than ~ (in units of carrying capacity) then the system tends to oscillate between two values. We cannot go into details on what happens for a > 3 (see [106] for details), but we shall qualitatively sketch the picture. For higher values of a, a cycle of order four appears and then a cycle of order eight and so on all the cycles of even period Z" will appear. All this happens as a goes from 3 to 3.57, where the last point is an accumulation point of cycles of Z" periods. What is the solution like to the left of this point?

182

7.

Models of Real World Phenomena

For a near 3.57 one can find a very large number of points that are parts of some solution of even period (which can be very large and hardly distinguishable from the periodic solutions). In this region even if two similar populations evolve starting from two very close initial points, their history will be completely different. After a long time every subinterval of (0,1) will contain a point of the trajectory and if one maps the density of the number of occurrences of Yn in subintervals of (0, 1) the pictures are very similar to sample functions of stochastic processes. This behavior has been called chaos. After the value 3.57 of a, the solution of period three appears. There is a result of Li and Yorke that states if there is a cycle of period three, then there are solutions of any integer periods and for the same reasons discussed above, the term chaos is appropriate in this case as well. Almost all of the previous results can be extended to more general functions f leading to the conclusion that the qualitative results are widely independent from the particular function chosen to describe the discrete model.

7.3.

Distillation of a Binary Liquid

The distillation of a binary ideal mixture of two liquids is often realized by a set of N-plates (column of plates) at the top of which there is a condenser and at the bottom there is a heater. At die base of the column there is a feeder of new liquid to still. A stream of vapor, whose composition becomes richer from the more volatile component, proceeds from one plate to the next one until it reaches the condenser from which part of the liquid is removed and part returns to the last plate. On each plate, which is at different temperature, the vapor phase will be in equilibrium with a liquid phase. Because of this, a liquid stream will proceed from the top to the bottom. We suppose that the liquids are ideal (that is the Raoult's law applies) as well as the vapors (that is the Dalton's law applies). Let Yi (i = 1,2) be the mole fraction of the i th component in the vapor phase and x, the mole fraction of the same component in the liquid phase. Of course Yl + Yz = 1 and Xl + Xz = 1 (the sum of the mole fraction in each phase is 1 by definition). In this hypothesis the quantity, called relative volatility, Yl Xz

a=--

Yz

Xl

·can be considered constant in a moderate range of the temperature (see [173]). If a > lone says that the first component is more volatile. For

183

7.3. Distillation of a Binary Liquid

simplicity, we shall consider as reference only the more volatile component. Setting Yt = Y and Xl = X, one has a=

y(l- x) x(l - y)'

(7.3.1)

which will be considered valid every time the two phases are in equilibrium. Let us see what happens on the nth plate. Here the two components are in equilibrium in the two phases. Let x, be the mole fraction of the more volatile component in the liquid phase, y~ the mole fraction in vapor phase of the same component, and Yn the mole fraction of the same component leaving the plate n. If we assume that the plate efficiency is 100 percent, then y~ = Yn' Moreover, part of the liquid will fall down with mole rate d and the vapor will go up with mole rate V. Let D be the mole rate of the product, which is withdrawn from the condenser. Consider now the system starting from the nth plate (above the point where new liquid enters into the apparatus) and the condenser. We can write the balance equation (7.3.2) where XD is the mole fraction of the liquid withdrawn from the condenser. To this equation one must add the definition of relative volatility that will hold for the equilibrium of the two phases at each plate a=

Yn(l- x n )

xn(l - Yn)

(7.3.3)

.

From the last relation we obtain

«x,

Y =-----:..:..-n l+(a-1)xn

and, after substitution in (7.3.2) we get Xn XnXn-1 + a - I

+

-1) - Va d(a -1) Xn+l

DXD(a

DXD

+ dt a -1) =

0,

(7.3.4)

from which, by letting 1

a=--' a-I'

b = DXD(a -1) - Va. dl a - 1) ,

c=

DXD

d(a -1)

one obtains (7.3.5) This is a difference equation of Riccati type (see 1.5.8). Let us consider the boundary conditions. The initial condition depends on how the apparatus is fed from the bottom and we will leave this

184

7.

Models of Real World Phenomena

undetermined. The other boundary condition will depend on the condition that we impose on the composition of the fluid on some plate. In the literature there are two different types of such conditions. The first one (see [19]) requires that there is a plate i for which X i- 1 = Xi' We shall show that this condition either leads to a constant solution or to no solutions (according to the initial conditions). The second boundary condition (which can be deduced from Fig. 16 of [20]) requires that YN = XD, XN+1 = XD' Let us start with the first condition, that is Xi = Xi-I for some i E [2, N]. Then Xi must satisfy the equation (obtained from (7.3.5» x;

+ (a + bvx, + c = O.

(7.3.6)

Now the Riccati equation can be transformed to a linear one by setting (7.3.7) to obtain Zn+1

+ (b + a)zn + (c - ab)zn_1 =

O.

(7.3.8)

If Al and A2 are two solutions of the polynomial associated with the previous equation one has (if Al ,p A2)

and therefore (7.3.9) Put n = i and n = i-I. The only way to obtain x, = Xj-l is taking either = 0 or C2 = 0, that is, x, = Al - a or x, = A2 - a. One verifies at once that these two solutions satisfy (7.3.6). If the initial condition does not match one of the two constant solutions, then there does not exist any solution. Concerning the second boundary conditions, which is consistent with the condition V = d + D, necessary for other physical considerations, one has CI

VYN

Imposing

YN = XD

=

LXN+I

+ DXD'

we get

(7.3.10)

185

7.4. Models from Economics

One obtains an equation whose unknowns are

Cl C2

(the Riccati equation is

offirst order and its general solution must contain only an arbitrary constant) and N (the number of plates needed to complete the process). C

Let us put, for simplicity, K = 2. One obtains, after some simple algebraic C2 manipulations,

XD) -XD

A2 - a Al - a

Al

(7.3.11)

log-

A2

One can verify that in order to obtain X D = 1 (at this value only the more volatile component is present in the distillate), one needs N = 00. The value of K is determined from the condition of the feed plate (which can be assumed as the zeroth plate).

7.4.

Models from Economics

A model called the cobweb model concerns the interaction of supply and demand for a single good. It assumes that the demand is a linear function of the price at the same time, while the supply depends linearly on the price at the previous period of time. The last assumption is based on the fact that the production is not instantaneous, but requires a fixed period of time. Let the above period of time be unitary. We shall then have

d; = -apn + do, Sn

= bpn-l + So,

where dn , s; are respectively the demand and supply functions and a, b, do, So are positive constants. This model is based on the following assumptions: (l) The producer hopes that the price will be the same in the next period and produces according to this. (2) The market determines at each time the price such that (7.4.2)

From (7.4.1) and (7.4.2) one obtains Pn

b)

= ( -;

do - So Pn-I +--a-'

186

7.

Models of Real World Phenomena

The equilibrium price Pe is obtained by setting P« = Pn-l = Pe, which gives

do Pe

= a

So

+b

and the solution of (7.4.3) is given by (7.4.4) which is deduced from (1.5.4). b If - < 1, it follows that the equilibrium price Pe is asymptotically stable, a

_!!:. is negative, we see that

that is lim P« = p., Since

b

n_OO

P« will oscillate

a

approaching Pe. For b > 1 the equilibrium price is unstable and the process will never reach this value (unless Po = Pe). As a more realistic model, take s; as a linear function of a predicted price at time n. This means that the producer tries to predict the new price using information on the past evolution of the price. For example, he can assume as the new price P« = Pn-l + P(Pn-l - Pn-2) obtaining

Sn

b(Pn-l

=

+ P(Pn-l - Pn-2)) + So,

while the first equation remains unchanged. By equating the demand and supply, one obtains

-apn

+ do =

+bpn-l

+ bpbn- 1 -

bPPn-2 + So

from which

Pn

b (l a

+-

+ P)Pn-l -

b So - do - PPn-2 + - - = a a

o.

(7.4.6)

. .. do - So 1 num pnce IS now Pe = - - . Th e equiilib a-b The homogeneous equation is

Pn

b

+-

a

+ P)Pn-l -

(l

b - PPn-2 a

=

0

(7.4.7)

and the characteristic polynomial reduces to

z

2

b

+-

a

(l

+ p)z -

bp - = o. a

(7.4.8)

7.4.

187

Models from Economics

Let Zj and Z2 be the roots of this equation. The general solution of the homogeneous equation is

while the general solution of (7.4.6) is (7.4.9) where C j and C2 are determined by using the initial conditions. From (7.4.9) it follows that if both IZjl and /z21 are less than one, one has lim P« = Pe. If n ....OO

at least one of the two roots has modulus greater than one, then lim P« = . n-HXJ

00

(for generic initial conditions). The problem is now reduced to derive the conditions on the coefficients of the equation in order to have IZjl and IZ2/ both less than one. A necessary condition is The other condition is (see Theorem B1)

b

bp

a

a

bap l < 1, SInce . -bp = -ZjZ2'

I

a

-+1 _ bp

< 1,

a

.

which is satisfied for

a

a

I; < 3 and -I; <

a b p <~. r-

The positive value of p means that the producer expects the price will continue to have the same tendency, while the negative value means that the producer expects to have an inversion in the tendency. To an equation similar to (7.4.6), one arrives in the Samuelson's model of national income. Here the economy of a nation is modeled by considering four discrete functions: the national income Yn, the consumer expenditures en used to purchase goods, the investments In and the government expenditure Gi, The definition of Yn is (7.4.10)

188

7.

Models of Real World Phenomena

The other relations among the four variables are based on the following assumptions, which are self-explanatory even from the economics point of view 0< a < 1, P > O.

The parameter a is usually called marginal propensity to consume and P is the acceleration coefficients or "the relation." One usually assumes

On = a constant.

By substituting in (7.4.10) we get Yn - a(1 + P)Yn-. + apYn-2 - 0=0,

which is similar to (7.4.6). The constant solution (or equilibrium point) is now Ye

a

=--. I-a

If this point is asymptotically stable, then lim Yn = Y., which means that n-s co

in this hypothesis the national income, under the effect of the government expenditure a will tend toward the income Y., which is greater than G. The factor _1_ is said to be a multiplier. To see when this happens, just I-a

repeat what is said for the previous model remembering that in this case P cannot be negative.

7.5.

Models of Traffic in Channels

The models presented here concern the traffic in channels (for example telephone lines) and are from information and queueing theory. In the first simple model, the number of messages of given duration is derived. In the other two models the probability of requests for services (for example telephone calls) is derived as well as the probability of loss of the request in a system of limited channels. Consider a channel and suppose that two elementary informations 8. and 8 2 of duration 1. and 12 respectively (for example the point and line in the Morse alphabet) can be combined in order to obtain a message. Let 1 be a time interval greater than both 11 and 12 , One is interested in the number of messages N, of length 1. These messages can be divided in two classes: those ending with 8. and those ending with 8 2 , The number of

7.5. Models of Traffic in Channels

189

messages in the first class is N t - tt while the number of messages in the second class is N t - 12 • Then we have (7.5.1)

Suppose, for simplicity, that t l = 1 and t2 = 2. The previous formula becomes (7.5.2)

The initial conditions are N) = 1 and N 2 = 2. The equation (7.4.2) is the same that defines the Fibonacci sequence (see (2.3.16». The solution is

1+V5

I-V5

where PI = - 2 - ' P2 = --2-' Because Ip21 < 1, for large t, one has N,

=

cIP\. Shannon defines the capacity of a channel as C

.

log2 N,

= hm---= log2Pl' t t~CD

Difference equations arise very often in queuing theory. We shall give two examples. Consider a queue of individuals (or telephone calls in a channel). Let Pn(t) be the probability of n items arrived at time t and we have L~=o Pn(t) = 1. Let Aat be the probability that a single arrival during the small time interval at and suppose that the probability of more than one arrival in the same interval is negligible (Poisson process). Let JLM be the probability of completing the service in the interval at. We shall assume that the service is a Poisson process, that is, the probability of no arrivals in at is 1 - Aa.t and the probability of the service not completed in at (no departures) is 1 - Aat. The following model is due to Erlang (see [22]) and describes the situation where at the beginning there are no items in the single channel queue and the service is made on a first come, first served basis, Pn(t

+ at)

= Pn(t)(l- Aat)(l- JLat)

PO(t + at) = P o(t)(1 - Aat)

+ Pn_1(t)Aat + Pn+1(t)JLat, n

~

1,

+ PI(t)JLat.

The meaning of the equations is the following: the probability that at time t + at there are n items in line is equal to the sum of three terms: (1) The probability of already having n items at time t multipled by the probability of no arrivals during at and the probability of no departure in the same interval;

190

7.

Models of Real World Phenomena

(2) the probability of having n - 1 items at time t multiplied by the probability of a new arrival in Ilt; (3) the probability of having n + 1 items at time t multiplied by the probability of a departure in tu. Taking the limit as t

~

0 one has

dPn(t)

----;jf = -(A + !J-)Pn(t) + APn_l(t) + !J-Pn+l(t),

n ~ 1,

dPo(t)

-;j( = -APo(t) + !J-PI(t). It is important to know how the system behaves for large t. That is, to know if the limit P; = lim PnU) exists. !->CO

The probability of P; will describe the steady state of the problem. In order to get this information one observes that in the steady state the derivatives will be zero. It follows that the limiting probabilities P; will satisfy the difference equations (7.5.3) (7.5.4) To this one must add the condition L~=o P; = 1, which simply states that it is certain that in the system we must have either no items, or more items. Let p

=~. !J-

The solution of (7.5.3) is given in terms of the roots of

the polynomial equation Z2 -

which are

ZI

(I

= 1, Z2 = p. If p <

+ p)z + P = 0,

1, we get

Considering that from (7.5.4) one has PI = PPo , it follows that To derive C2 one uses the additional condition 1=

00

1

n~O

1- p

CI

= O.

L e; = C2 - - ,

which gives C2 = 1 - p and then P; = (I - p)pn, which is called geometric distribution.

7.5.

191

Models of Traffic in Channels

Important statistical parameters are: (1) the expected number in the system L =

n

d

00

00

I nPn = (1- p) n=O I np" = (1- p)pI pn = 1 ~ p; n=O dp n=O

(2) the variance

v= I

00

n=O

=I

00

(n-L)2pn

n 2Pn - L2,

n~O

2 one shows that I~=o n Pn = L + 2L2 and V = L (3) the expected number in the line 00

Lq

=I

n=O

(n - I)Pn

+ L 2;

= pL.

In the following generalization one considers the case of N identical channels with the same hypothesis on the distribution of arrivals and departures (Poisson distribution). Moreover, we shall assume that both the state with zero items Eo, and the state with N items EN are reflecting, that is, from these states transactions are possible only in one direction. From the state Eo a possible transaction will be to the state E I and from the state EN a possible transaction will be to the state EN-I' The resulting model is dPn(t) = -(A

n

+ nj.t)Pn(t) + APn_l(t) + (n + 1)j.tPn+ l(t),

dP o dt = -APo(t) dPN

----at =

1:$ n :$ N - 1,

+ j.tP1 (t ),

-Nj.tPN(t) + APN_I(t).

The steady state solution satisfies the difference equations

+ 1)Pn+ 1 PI = pPo,

(n

where, as before p

=~. j.t

from which one has Zo

(p

+ n)Pn + PPn- 1 =

0,

Let us look for solutions of the form P; = z"p",

= Po, ZI

=

Po, ZN =

Z;;I. The

equation in is then

192

Let

7.

Models of Real World Phenomena

+ (n + I)Zn+l' The equation becomes

On = -Zn

whose solution is

For n = N - 1, we must have ON = -ZN-l + NZN It follows then 00 = O. The equation for Zn is then -Zn

which has the solution z;

= -ZN-l + ZN-l = O.

+ (n + l)zn+l = 0,

= ..!..- Zo = ..!..- Po. Going back to n!

n!

P; we have P; =

~ pnpo. Imposing the condition I~=o r; = 1, one finds Po = l/I';:o ~;. The n. J. solution is then

which is called Erlang's loss distribution. For n = N, one obtains the probability that all the channels are busy, which also gives the probability that a call is lost. As N

~ 00 one obtains the Poisson distribution P; = p

n

e"".

n! The previous model also describes the parking lot problem. For this problem, N is the number of places or slots in the parking lot. The reflecting condition at the end just describes the policy that the parking lot closes when it is full. The probability that a car cannot be parked is PN • If the management follows the policy of allowing a queue waiting for a free place, then the model is modified as follows:

(n

+ l)Pn +! NPN + 1

-

-

(p

+ n [P; + PPn - 1 = 0,

(p

+ N)Pn + PPn - 1 = 0,

The solution is n:S

N,

n

is;

N,

n> N.

193

7.7. Notes

as before, and n>N.

Using the relation

7.6. 7.1.

I:=o P; = lone obtains

Po.

Problems In the case aj > 0, derive the existence of eigenvalue of greatest absolute value from Theorem 7.1.1 instead of Perron-Frobenius theorem.

7.2. Write the generating function for the solution of (7.1.8) and derive the behavior of the solution (Renewal Theorem). 7.3. Show that for a < 1 the origin is asymptotically stable for Yo E (0, 1) for the discrete logistic equation. (Hint: use the Lyapunov function Vn = y~.) 7.4.

Discuss the stability of y for the logistic equation using Vn = (Yn -

7.5.

Determine the solution of (7.1.5) for M a4 = 16.

7.6.

Discuss the modified Cobweb model with P = -1.

7.7.

Derive the national income model assuming that In

=4

and

y? a 1 = a2 = a3 = 0,

= P(Yn-l -

Yn-2)'

7.8. Suppose that the input for the traffic in N channels has Poisson distribution and depends on the numbers of free channels, that is, Pi is not constant. Derive the model and find the solution.

7.7. Notes Age dependent population models have many names associated with them such as Lotka, McKendrick, Bemardelli and Leslie. An extensive treatment of the subject may be found in Hoppensteadt [80], Svirezhev and Logofet [164]. In [164] we also have several references to the subject. Theorem 7.1.1, which is a generalization of a Cauchy theorem, may be found in Ostrowsky [121], see also Pollard [141]. The discrete logistic equation .has been discussed by many authors, for example, Lorenz [97], May [106, 107], Hoppensteadt [80], Hoppensteadt and Hyman [79]. The result of Li and Yorke [95] on the existence of periodic

194

7.

Models of Real World Phenomena

solutions is contained in a more general result due to Sharkovsky, see [155], [161], [79], [162] and [8] for more recent proofs and results. When a difference equation is derived from an approximation of a continuous differential equation, the question arises to what extent the behavior of the respective solutions are similar. This problem has been outlined and treated by Yamaguti et al [176,177,178] and Potts [145]. The distillation model has been adapted from [110], see also [173]. The discrete models in economics may be found in Gandolfo [50], Goldberg [60] and Luenberger [98]. Queueing theory is a large source of difference equations, see for example Saaty [149], where one may find very interesting traffic models more general than those presented in Section 7.4. For the parking lot problem, as well as other models concerning traffic flow, see Height [66].

Appendix A

A.I.

Function of Matrices

Let A be a ii x ii matrix with real or complex elements, and let us consider expression defined by k

peA) =

I

p.A',

(AU)

i=O

where k E N+ and Pi E C. Such expressions are said to be matrix polynomials. Associated with them are the polynomials p(z) = I.~~OPiZi. There are some differences of algebraic nature among polynomials defined in the complex field and those defined on a commutative ring (the set of powers of a matrix). For example, it can happen that An = AA n- 1 = but neither A or A n-l is the null matrix. More generally, it can happen that peA) = (A - z;I) with z, E C is the null matrix and A '" z.I, which cannot happen for complex polynomials. The Cayley theorem states that if Zi> Z2, ••• ,Zn are the eigenvalues of A, with multiplicity mi , letting

°

rr.,

peA) =

s

II (A -

z;I)m,

(Al.2)

i=1

one has peA) = 0. The polynomial p(z) is called the characteristic polynomial. Let ljJ(z) be the monic polynomial of minimal degree such that IjJ(A) = 0,

(AU)

and let n be its degree.

195

196

Appendix A

Theorem A.l.l. Every root of the minimal polynomial is also a root of the characteristic polynomialand vice versa. Proof. that

Dividing p(z) by l/J(z) we find two polynomials q(z) and r(z) such p(z) = q(z)l/J(z)

+ r(z)

(AlA)

with degree of r(z) less than n. For the corresponding matrix polynomials we have 0 = peA) = q(A)l/J(A) + rCA). Since l/J(A) = 0 and it is minimal, it follows that rCA) is identically zero. Thus (AlA) becomes p(z) = q(z)l/J(z) proving that the roots of l/J(z) are necessarily roots ofp(z). Suppose now that ,\ E C is a root of p(z) and therefore it is also an eigenvalue of A. For every j E N+, it follows that A'x = ,\jx, where x is the corresponding eigenvector. Then we have l/J(A)x = l/J(,\)x from which it follows l/J(,\) = 0, proving that ,\ is also a root of l/J(z). • Let h(z) be any other polynomial of degree greater than n. We can write h(z) = qi z)l/J(z) + r(z) where the degree of r(z) is less than n. Consequently h(A) = rCA).

(Al.5)

The last result shows that a polynomial of degree greater than n and a polynomial of degree less than n may represent the same matrix. Let h(z) and g(z) be two polynomials such that h(A) polynomial d(z)

= h(z) -

g(z)

= g(A). Then the (A.1.6)

annihilates A; that is, d(A) = O. It follows that it is possible to find q(z) such that d(z) = q(z)l/J(z). Let ml> m2,"" m, be the multiplicities of the roots of l/J(z). Then we have l/J(Zj) = l/J'(z;) = ... = l/J(m,-I)(Zj) = 0, i = 1,2, ... , s, from which d(zj) = d'(zj) = ... = d(m,-l)(z;) = 0, i = 1,2, ... , s. This result shows that h(z;)

= g(Zj),

h'(z;)

= g'(z;),

(A1.7)

h(m,-O(Zj) = g(m,-iJ(Zj),

for i = 1,2, ... , s. Two polynomials satisfying the above condition are said to assume the same values on the spectrum of A. Now suppose that the two polynomials assume the same values on the spectrum of A. It follows that d (z) has multiple roots z, of multiplicity m, and then it will be divided exactly by l/J(z); that is, d(z) = q(z)l/J(z). Then d(A) = g(A) - h(A) = 0 and the two polynomials represent the same matrix. The foregoing arguments prove the following result.

A.I.

Function of Matrices

197

Theorem A.1.2. Let g(z) and h(z) be two polynomials on iC and A E C. Then g(A) = h(A) iff they assume the same values on the spectrum of A. Definition A.I.I. Let f(z) be a complex valued function defined on the spectrum of A, and let g(z) be the polynomial assuming the same values on the spectrum of A. The matrix functionf(A) is defined by f(A) = g(A). The problem of definingf(A) is then solved once we find the polynomial g(z) such that g(i)(zd = fi)(Zk) for k = 1,2, ... , s, i = 0, 1, ... , mk-I' The last problem is solved by constructing the interpolating polynomial (Lagrange-Hermite polynomial) (A 1.8)

where cPki(Z) are polynomials of degree n - 1 such that ",('-1)( ) _ 'I' Zj -

"

ki

For example for m, = 1, i

{j, k = 1,2, ... , s,

"

UkjU'i,

i, r

= 1,2, ... , mk'

= 1,2, ... , n,

n (z - Zj)

cPkl(Z)

j>"k ="-----

n (z, -

j>"k

Zj)

It can be proved that the functions cPki(Z) are linearly independent. The matrix polynomial is then s

g(A)

=

mk

L: L: f(i-I)(Zk)cPki(A).

(A 1.9)

k=1 i=1

The matrices cPki(A) are said to be components of A. They are independent of the function f(z). As usual in this field we shall put (Al.10)

These matrices, being polynomials of A, commute with A. With this notation (Al.10) becomes f(A) == g(A)

s

mk

= L: L:

f(i-I)(Zk)Zki'

(Al.1t)

k=1 i=1

The set of matrices Zki are linearly independent. In fact if there exist constants Cki not all zero such that I~=I I~:I Ck;Zki = 0, the associated polynomial s(z) = I~=I I~:I CkicPki(Z) would annihilate A.

198

A.2.

Appendix A

Properties of Component Matrices and Sequences of Matrices

Let us list some properties of the component matrices Takingf(z) = 1, we get from (A.Lll)

Zki'

(A.2.l)

Also, taking f(z)

=

z, we see that s

A =

s

L ZkZkl + L Zk2' k=l

(A.2.2)

k~l

from which s

2

A =

s

L (ZkZkl + Z d j=l L (Zj2.il + 2.i2) k=l s

s

L L

=

(Zk ZjZkl2.il

k=lj~l

Starting directly from f( z) A

2

= Z2,

one has

s

= L

k~l

+ 2ZkZk l2.i2 + Zk22.i2)'

(z~Zk1

+ 2ZkZk 2 + 2Zk3)·

Comparing the two results, it follows that Zkl2.il = l)kjZkl, Zkl2.i2 = l)kj Zk2, Zk22.i2 = 2l)kjZk3'

Proceeding in a similar way, it can be proved in general that ZkpZ;r

=0

if k ;c i,

ZkpZkl

= Zkp

p;;::: 1,

Zkp Zk2

= pZk,p+l

p;;::: 2.

(A.2.3)

From the last relation it follows easily that Zkp

= ( 1 ) ZE 1 , P -1 !

P ;;::: 2.

It is worth noting that, from the second relation of (A.2.3), we get

(A.2.4)

199

A.2. Properties of Component Matrices

showing that the matrices Zkl are projections. Multiplying the expression A - z.I = L~=I (Zk - z;}ZkJ + L~=I Zk2' by Zit and considering (A.2.3), one gets, s

=I

(A - ZJ)Zil

k=1

(Zk - Z;)ZkIZil

+

s

I

k=1

Zk2Zil

= Zi2'

(A.2.6)

Because of (A.2.4) and (A.2.6), we obtain Z

1 Zp-I 1 (A I)P-IZ kp-(p-1)! k2 -(p-1)! -Zk kl'

(A.2.7)

From this result, it follows that (A. 1. 11) can be written as f(A) =

t I f(:-l)(z~)

k=1 i~1

(I

-1).

Let us consider now the functionf(z) f(A)

= (A -

(A - ZkI) i-I Zkl'

(A.2.8)

_1_, where A ."r:. Zk' The function

=

z-A

H)-I is expressed by s

mk

I I

(A - H)-I = -

k=1 i=1

(i -1)I(A - Zk)-iZki

(A.2.9)

from which, considering (A.2.4), we get (A - AI)-I

s

=- I

k=1

[(A -

zd- I Zkl + (A -

+ (A -

Zk)-3Z7.2 + ... + (A - Zk)-mkZ~~-I],

we then obtain because of (A - AI) = -

t

1=

k=1

[Zkl

+ (A -

+ (A -

Zk)-2Zk2

s

I

k=1

[(A - Zk)Zkl - Zd,

Zk)-IZkIZk2 + (A - Zk)-2ZkIZ7.2 + ...

ak)-mk+IZkIZ~~-1

- (A - Zk)-I ZklZk2 - (A - Zk)-2Z7.2 - (A - Zk)-3Zt2 - ...

- (A - Zd-mkZ~n s

I

[Zkl - (A - Zk)-mkZ~n

k~1

from which results

Z;; =

0,

showing that the matrices Zk2 are nilpotent.

(A.2.10)

200

Appendix A

This result allows us to extend the internal sum in formula (A.2.8) up to

mb the multiplicity of the characteristic polynomial. In fact, since m; ::'5 mk, Z~k+i = 0 for j = 0, 1, .. , , mk - mi, and hence (A.2.8) can be written as f(A) =

f

I

f:-I)(Zk) (A - ZkI)i-1 Zkl' k=li=1 (1-1)!

(A.2.1l)

Now consider a sequence of complex valued functions fl(z), f2(Z)"" defined on the spectrum of A. Definition A.2.1. The sequence fl J2, .. " converges on the spectrum of A if, for k = 1,2, ... ,s, limf(zk) = f(Zk), limf;(zd = f'(Zk), and so on i-+oo

until !imflmj-I)(zd

=

i-+oo

f(m j-I)(Zk)' The following theorem is almost evident

' .... 00

(see Lancaster), Theorem A.2.t. A sequence ofmatricesf(A) converges if the sequencef(z) converges on the spectrum of A. Corollary A.2.t. Let A be an s x s complex matrix having all the eigenvalues inside the complex unit disk. Then (I - A)-I = I:o Ai. Proof. Consider for i 2:: 0 the sequencef(z) = I;=o z'. This sequence (and the sequences obtained by differentiation) converges if Izl < 1 to (1- Z)-I. It follows that there exists the limit of f(A) and the limit is (I - A)-I, •

Corollary A.2.2.

Proof.

Let A be an s

x s complex matrix.

Then e A =

Consider for i > 0, the sequence f(z) = I;=o

converges for all z • to e/:

E

Definition A.2.2.

An eigenvalue Zk is said simple if mk

~.

J.

Ai

I';:o -:,. J.

This sequence

C. Likewise for fls)(z). It follows thatf(A) converges

= 1.

Definition A.2.3. An eigenvalue Zk is said semisimple if Zk2 = O. A semisimple eigenvalue is not simple (or degenerate) if mk = 1 and mk ¥:- 1.

In both cases the terms containing (A - ZkI)P-IZkl with p> 2 are not present in the expression of f(A).

201

A.2. Properties of Component Matrices

Example A.2.1. An important class of matrices are the so called companion matrices, defined by 0 0

1 0

0 1

0

A= 0 -an

0 1

0 -a l

For this class of matrices m, = mj (for an values of i), that is, the characteristic polynomial and the minimal polynomial coincide (see Lancaster) and this implies that there are no semisimple roots. Example A.2.2.

Let f( z) = e z t and A be an n x n matrix. Then s mk ti- I (A.2.l2) e At = " f.., " s: (. ) e zk teA - Zk I)i-1Zkl' k=1 i=1 I - 1 !

If the eigenvalues are all simple, the previous relation becomes if

e At - " f.., e k=1

If

Zj

vz.

(A.2.l3)

kl'

is a semisimple eigenvalue, (A.2.H) becomes (A.2.14)

Theorem A.2.2. Proof.

If Rez, < 0 for i

= 1,2, ... , ii,

From (A.2.12) it follows easily.

then lim IAt = O. (-+00

•

Theorem A.2.3. If for i = 1,2, ... , ii, Rez, s;; 0 and those for which Rez, = 0 are semisimple eigenvalues, then eAt tends to a bounded matrix as t ~ 00. Proof.

It follows easily from (A.2.l3).

Example A.2.3.

Letf(z)

= z"

and

•

202

Appendix A

If the eigenvalues

Zj

of A are all simple, then (A.2.I5) becomes ii

=I

An

If

Zj

(A.2.I6)

Z~Zk"

k~'

is semisimple, (A.2.I4) reduces to

Is mI k

An =

Theorem A.2.4. If

k=1 k¢j

~ z~-i(A - ZkI)iZkl + Z;~I' )

- , (

IZil < 1 for i = 1,2, ... , n,

then lim An n-+OO

The proof is easy to see from (A.2.I5).

Proof.

(A.2.I7)

I

i=O

= O.

•

Theorem A.2.S. Iffor i = 1,2, ... , ii, IZil ~ 1 and the eigenvalues for which IZil = 1 are semisimple, then An tends to a bounded matrix as n ~ 00. The proof is easy to see from (A.2.I7).

Proof.

•

It may happen, however, that for multiplicity of higher order and consequently higher dimension of the matrix, the terms in (A.2.I7) may become large. For example, let us take the matrix ,\

{3

0

A

0 {3

0 0

0

= AI + {3H

A=

0

0

0 A

(A.2.I8)

sXs

with 0 0

1 0

0 1

0 0 ... 0

H=

0 1 0

0

(A.2.I9)

sXs

and H = O. Then (A.2.I7) becomes S

An =

I

s-I

i=O

(n). I

A n-i{3iH i•

(A.2.20)

A.3. Integral Form of a Function of Matrix

203

Multiplying by E = (1,1, ... , l)T and taking n = s - 1, we have A"E

= itl

(;)A

"-if3

i

i

(A.2.2l)

H E.

The first component of this vector is (A"E)I

=

it(;)A

= (A + er,

"-if3i

(A.2.22)

from which it is seen that

IIA"EII2: IA + f31". (A.2.23) This implies that the component of A"E will grow even if IAI < 1, but IA + f31 > 1. Eventually they will tend to zero for n > s - 1, because from (A.2.20) the exponents of A become bigger and bigger while those of f3 remain bounded by s - 1. But in the cases (as in the matrix arising in the discretization of P.D.E.), where s grows itself, the previous example shows that the eigenvalues are not enough to describe the behavior of A". This will lead to the introduction of the concept of spectrum of a family of matrices. A.3.

Integral Form of a Function of Matrix

Let M(z) be an s x s matrix whose components mij(z) are functions of the variable z. In the following, we shall assume that the functions mij(z) are analytic functions in some specified domain of the complex plane. Definition A.3.1. The derivative of the matrix M(z) with respect to z, is the matrix M'(z) whose elements are mij(z). Definition A.3.2. The integral of the matrix M(z) is the matrix whose coefficients are the integrals of mij(z).

Let us consider now the matrix R(z, A) == (zI - A)-I, called

resolvent

matrix.

From

(A.3.1)

(A.2.9),

we

get

R(z, A) = (k = not containing other

I~=I I~~I (i -l)!(z - Zk)-iZki, which has singularities for

1,2, ... ,s). Suppose now that I' h is a circle around eigenvalues. Then (A.2.9) becomes R(z, A)

mh

=I

(j -l)!(z -

Zh)-jZhj

j=1

mh

=

I

j~l

+

S

mk

I I

k~lj=1

k,,
(j -l)!(z -

Zh)-jZhj

+ Sh(Z),

Zk

(j -l)!(z -

z

= Zk,

Zd-jZkj

204

Appendix A

where Sh(Z) is the matrix obtained by expanding the term (z - Zk)-I = 1 ( Z + Zh)-I -- 1- -. It is seen that the elements of Sh(Z) are analytic in Zh - Zk Zh - Zk r b- Integrating on r h, we have

r R(z, A) dz = J~I.I U -1)IZhj Jr.r (z - Zh)-j dz = 27TiZh

Jr.

t.

from which

1 . r R(z, A) dz. z., = -2 Jr.

(A.3.3)

7T1

Analogously, for t = 1,2, .... , mh - 1,

i

r.

I

(z - Zh) Rt z, A) dz

m.-I

= .I

J~l

(j -1)!Zhj

f

(Z-Zh)1 ( _ Ydz r, Z Zh

= (27Ti)tlZh,,+I' In general we shall obtain

f

1 Zhj = ( . _ 1)'2' (z - ZhY'-1 R(z, A) dz. J . 7T1 r,

(A.3.4)

(A.3.5)

Theorem A.3.t. Suppose that f is a function continuous on a closed and regular curve r, which contains in its interior the points Zt. Z2,' ... , z, and that f is analytic inside. Then

=2

f(A) Proof.

1 m

.

J.r f(z)R(z, A) dz.

(A.3.6)

From (A.2.9),

1 ~ r f(z)R(z,A)dz=IIU-1)!Zkj . r (z-zk)-jf(z)dz 2 7T1 Jr 2m Jr k

=

j

tt

. fU-1)(z) (J -1)!Zkj (j -1)! =f(A),

where the Cauchy formula has been applied.

AA.

•

Jordan Canonical Form

From (A.2.2), it results that s

A

=

I (ZkZkl + Zk2) k=l

(A.4.1)

A.4. Jordan Canonical Form

205

and from (A.2.1) it follows that the space R" can be decomposed into s subs paces M l , ••. , M, defined by j

= 1,2, ... , s.

(AA.2)

Similarly, from (A.2.3) it follows that for i ,e j Mj n

~ =

(A.4.3)

0.

Let mj = dim M; Then I:~l mi = n and it is seen that mj is the multiplicity of z, as root of the characteristic polynomial. We want to choose appropriately a base for the subspace ~.

° °

Let x(j) E ~,ZJ2X(j) ,e for :5 i < P and Zj2X(j) = 0. Then the vectors ZJ2X(j), i = 0, 1, ... ,p - 1 are linearly independent.

Lemma A.4.1.

Proof.

From p-l

L

i=O

Ci Z J2X(J ) =

0,

(A.4.4)

it follows, multiplying successively by Z)2-1, Z)2-2, .. . , that Cj = O. We shall call the set of vectors ZJ2X(j) for i = 1,2, ... ,p - 1 the chain starting at x'!'. These vectors are also called generalized eigenvectors associated to x(J). •

Let x(J) EM., ZJ2X(j),e 0 for 0:5 i < P and Z)2X(j) = 0. Suppose that there exists y(J~ E M, linearly independent from the previous defined set ofvectors {ZJ2Z(J)}. Then the vectors {ZJ2y(J)} are linearly independent from the previous set.

Lemma A.4.2.

Proof.

Similar to the previous proof.

•

Going back to our problem we shall distinguish the following cases: (1) If m, = 1 = mdor i = 1,2, ... , n, it follows from (A.4.1),

Xi

E

M i , that (A.4.5)

Defining the matrix X

= (Xl' ... ,Xn ) ,

(

~l

0

we obtain ...

AX=X.

:

o

f)

0 0

Zn

(A.4.6)

206

Appendix A

and considering that the matrix X is nonsingular we define the diagonal matrix B similar to A by (A.4.7)

B = X-lAX.

(2) m; = 1, m; < mi' In this case we can choose mi linearly independent vectors in M i , such that (it is Zi2 = 0):

mi

j = 1,2, ... ,

and the matrix A can be transformed in diagonal form, as in the previous case. (3) At least an mj is different from 1, mj = mj and ZJ2 ¢ 0 for t.rn, - 1. Consider the choice of vectors x\j) = ZJ2X~j), i = 0, 1, ... , mj - 1, where x~) is a vector in M.i. From Lemma A.4.1, these vectors are linearly independent and can be chosen as a base for M.i. From (AA.I) we have Ax\j) =

ZiX\j)

+ Zk2X\j) =

z;x\j)

AX~;_I = ZjX~;_I' . X -- ( X o(I) , D e fini nlng the matnx have from (A.4.8)

AX

=X

+ x\~\ ,

= 0, 1, ... , mj

i

(I) ... , Xmrl>"

(5)

-

2,

(5») we

., Xo , •.• , Xm,-l ,

0

ZI Zz

Since X is nonsingular, a matrix J, similar to A can be defined as J = X-lAX = diag(Jj, J2 ,

••• ,

Js ) .

The structure of B is a block diagonal, each block has the form

i: =

(~k. .:. : '.

o

0

0 ).

(A.4.9)

1 Ak

(4) At least an mj is different from 1 and ZJ2 ¢ 0 for 0 $ t < mj' Choosing xo(j) E M·J' we define the set of vectors x(j) = ZiJ 2XO (j) for i = 0, 1, ... , i -s t. These f vectors are linearly independent (see Lemma l

207

A.5. Norms of Matrices and Related Topics

1), but they are not enough to form a base of M]. Choose another vector xV) independent of the previous ones and define x;~\ = Z;2XY) and so on until a set of mj linearly independent vectors has been found. Then we proceed as in the previous case.

The matrix J is block diagonal as in the previous case, but the block corresponding to the subspaces M, can be decomposed in different subblocks with each one corresponding to one chain of vectors. Each subblock is of the type (A.4.9). The matrix J is said to be the Jordan canonical form of the matrix A. Each chain associated to M, contains an eigenvector associated to Zj (the last vector of the chain). The number of eigenvectors associated to Zj is the geometric multiplicity of the chain. The dimension of M, is the algebraic multiplicity of Zj.

A.5.

Norms of Matrices and Related Topics

The definitions of norms for vectors and matrices can be found on almost every book on matrix theory or numerical analysis. We recall that the most used norms in a finite dimensional space R S are: (I)

Ilvll

S

L Iv;l,

=

l

i~1

S

1/2

)

(2)

Ilvlb =

(3)

Ilvllco = max IVil.

(

;~I

v7

,

l~r$s

By means of each vector norm, one defines consistent matrix norm as follows

I Ax II

IIAII = ~~~N· The corresponding matrix norms are

f

la;jl,

(1')

IIAII

(2')

IIAlb = (p(A

(3')

IIAllco = max j=l L layl,

I

= max

i=1

ls}::=:;s

HA»I/2,

S

]SI$S

(A.5.I)

208

Appendix A

where A is any s x s complex matrix, A H is the transpose conjugate of A, and p(A) is the spectral radius of A. Given a nonsingular matrix T, one can define other norms starting from the previous ones, namely, IIvllT = I Tv I and the related consistent matrix norms IIAIIT = IIT-IATII. If T is a unitary matrix and the norm chosen is 11·lb, it follows that

IIAII = IIAIIT'

(A.5.2)

For all the previous defined norms the consistency relations

IIAxII :5 IIAllllxll,

(A.5.3)

IIABII :5I1AII IIBII, where A, B, x are arbitrary, hold.

(A.5.4)

and

Theorem A.5.l.

For every consistent matrix norm one has p(A)

:5

IIAII.

(A.5.5)

Proof. Let A be an eigenvalue of A and v the associated eigenvector. Then we have

IAlllvl1 :5IIAvll :5IIAllllvll, from which (A5.3) follows. Corollary A.5.l.

•

If IIAII < 1, then the series

(A.5.6)

converges to

(1 -

A)-I, and

11(1 - A)-III < 1/(1-IIAII).

Proof. From Theorem A.5.1 it follows that p(A) < 1 and the results follow from Theorem A1.6 (see also exercise ALl). • Another simple consequence is the following (known as Banach Lemma). Corollary A.5.2. Let A, B be s x s complex matrices such that IIA-I II :5 a, IIA - BII :5 13, with 0'13 < 1. Then B- 1 exists and liB-III < 0'/(1- 0'13).

Proof. Let be C = A-I(A - B). By hypothesis IICiI < 0'13 < 1. Then (1C)-I exists and 11(1 - C)-III :5 1/(1- 0'13). But B- 1 = (1 - c)-lA-I and • then liB-III :5 0'/(1 - a(3). Theorem A.5.2. Given a matrix A and e > 0, there exists a norm IIAII such that IIAIl :5 p(A) + e.

209

A.6. Nonnegative Matrices

Proof.

Consider the matrix A' =

.!.e A

and let X be the matrix that trans-

forms A' into the Jordan form (see section A.4). Then 1

-X e

-I

1 AX=Je==-D+H, e

where D is a diagonal matrix having on the diagonal the eigenvalues of A and H is defined by (A.2.19)'. For anyone of the norms (1'), (2'), (3'). IIHII = 1, and IIDII:s: p(A). It follows that IIX-IAXII == IIAllx :s: IIDII + ellHl1 :s: p(A) + e, which completes the proof. •

A.6.

Nonnegative Matrices

In many applications one has to consider special classes of matrices. We define some of the needed notions. Definition A.6.1.

An s x s matrix A is said to be

(1) positive (A> 0) if aij > 0 for all indices, (2) nonnegative (A ~ 0) if aij ~ 0 for all indices. A similar definition holds for vectors x

E

R S considered as matrices s x 1.

Definition A.6.2. An s x s nonnegative matrix A is said to be reducible if there exists a permutation matrix P such that PAp

T

= (~

~)

(A.6.l)

where B is an r x r matrix (r < s) and D an (s - r) x (s - r) matrix. Since P" = p- I , it follows that the eigenvalues of B form a subset of the eigenvalues of A. Definition A.6.3. A nonnegative matrix A, which is not reducible is said to be irreducible. Of course if A > 0 it is irreducible. Proposition A.6.1. Proof.

If A is reducible then all its powers are reducible.

It is enough to consider that

PA 2p T and proceed by induction.

=

PApTpAP •

=

B2 ( G

210

Appendix A

Definition A.6.4. An irreducible matrix is said to be primitive if there exists an m > 0 such that Am> o. Definition A.6.S. primitive.

An irreducible matrix A is said to be cyclic if it is not

It is possible to show that the cyclic matrices can be transformed, by means of a permutation matrix P, to the form

0

PAp T

=

Or

01

0

0

O2

0

...

0 (A.6.2) Or-I

0

In case when A is irreducible we have the following result, due to PerronFrobenius. Theorem A.6.2. If A > 0 (or A ~ 0 and primitive) then there exist Ao E R+ and xo ~ 0 such that

(a) Axo = Aoxo, (b) if A ~ Ao is an eigenvalue of A then IA 1 < Ao, (c) Ao is simple. If A ~ 0 but not primitive, then (a) is still valid but

IAI ~ Ao in (b).

Theorem A.6.3. Let A ~ 0 be irreducible and Ao its spectral radius. Then the matrix (Al- A)-I exists and is positive if IAI > Ao .

Proof. Suppose IAI > Ao. The matrix A* = A-IA has eigenvalues inside the unit circle. Hence it follows (see Corollary A2.l) that (1 - A*)-I is given by (1 - A*)-I =

L 00

A*i

i=O

from which it follows (Al- A)-I

= A-1(1 -

A*)-I

= A-I L 00

(A-IA)i.

i=O

The second part of the proof is left as an exercise.

•

(A.6.3)

Appendix B-The Schur Criterium

We have seen in Chapters 2 and 3 that the asymptotic stability problem for autonomous linear difference equations is reduced to the problem of establishing when a polynomial has all the roots inside the unit disk in the complex plane. This problem can be solved by using the Routh method (see for example [152] and [12]). In this Appendix we shall present the Schur criterium, the one most-used for such a problem. Let p(z) = L~=OPiZi be a polynomial of degree k. The coefficients Pi can be complex numbers. Let q(z) = L~=OPiZk-i, (Pi are the conjugates of p;), be the reciprocal complex polynomial: q(z) = Zkp(Z-I). Let S be the set of all the Schur polynomials (see Definition (2.6.4)). Consider the polynomial of degree k - 1: p(l)(z)

= fikp(z) -

Poq(z).

z

. easy t 0 see t h at p (1)() I t IS z Theorem B.t.

Proof.

p(z) E S

Suppose p(z)

E

k (- ) i-I = "':"'i=1 hPi - POPk-i Z •

iff (a)

IPol < IPkl and p(l)(z) E S.

S and let

ZI, Z2, •.• , Zk

IPol = IPk

III

I

z, < IPkl

and condition (a) is verified. On the unit circle

Iq(z)1 =

be its roots. Then

Izl = lone has

IitO iI = IitoPiZ- iI = litPizil = pZ

k-

Ip(z)l·

211

212

Appendix B-The Schur Criterium

Since condition (a) is verified, we get IPkP(z)l> IPop(z)1 = IPop(z)1

= IPoq(z)1 = I-Poq(z)l·

Applying Rouche's theorem (see [76]) it follows that the polynomial PkP(Z) and Pkq(Z) - Poq(z) = zp(l)(z) have the same number of roots in Izi < 1, that means that p(l)(z) E S. Suppose now that p(l)(z) E Sand IPol < Ipkl. It follows that on Izi = 1, IPkP(z)l> I-Poq(z)1 and again by Rouche's theorem the polynomial PkP(Z) has n roots inside the unit disk, that is p(z) E S. The previous theorem permits to define an algorithm to check recursively if a polynomial is a Schur polynomial. The algorithm is very easily implemented on a computer. Next theorem, which is similar to the previous one, gives the possibility of finding the number of roots inside the unit disk. Consider the polynomial of degree k - 1 Tp(z) == Pop(z) - Pkq(z) =

k-t

I

;=0

i

(POPi - Pkfik_Jz •

The polynomial Tp(z) is called the Schur transform of p(z). The transformation can be iterated by defining T'p = T(T'-tp), for s = 2, 3, ... , k: Let 'Ys = T'p(O), s = 1,2, ... , k: • Theorem B.2. Let 'Ys ;t' 0, s = 1,2, ... , k, and let SI, S2, ••. , s.; an increasing sequence of indices for which 'Ys < O. Then the number ofroots inside the unit disk is given by h(p) = Lj':t (~IY-1(k + 1 - Sj)' Proof.

See Henrici [76].

•

Analogous results can be given for Von-Neumann's polynomials. Let N be such a set. Theorem B.3.

A polynomial p(z) is a Von-Neumann polynomial

(1) IPol < IPkl or

and

(I') p(l)(z) == 0 and

Proof.

See [113].

•

(2) p(l)(z)

E

(2') p'(z)

S.

E

N

if either

Appendix CChebyshev Polynomials

C.l.

Definitions

The solutions of the second order linear difference equation where

Z E

Yn+Z - 2ZYn+l + Yn = 0, C, corresponding to the initial conditions Yo

= 1,

Yl

(Cil )

= z,

(C2)

and Y-l = 0, Yo = 1, (C.3) are polynomials as functions of Z and are called Chebyshev polynomials of first and second kind respectively. They are denoted by Tn(z) and Un(z). We list the first five of them below. To(z)

=1

U_ 1 = 0

T1(z) = Z Tz(z) T3(z)

= 2z z = 4z 3 -

T4(z) = 8z

4

-

Since the Casorati determinant

Uo(z) = 1 1

3z

8z z + 1

I

K(O) = To(z) T1(z)

U1(z) Uz(z) U3(z)

= 2z = 4z z -

= 8z

U_1(z) Uo(z)

3

-

1

4z.

I

is equal to 1, the general solution of (Cil ) can be written as Yn(z)

= Cj

Tn(z)

+ czUn-1(z).

(C.4)

213

214

Appendix C-Chebyshev Polynomials

Let WI and (C.1), that is,

W2

be the roots of the characteristic polynomial associate to 2 W -

2zw

+ 1 = O.

It is easy to express Tn(z) and Un(z) in terms of obtains, considering that W2 = w~\

Tn(z)

=

w n + w- n I 2 I = cosh n log(z

Un(z)

=

1 (Z2 -1)1/ 2

W~+I -

(C.S) w~

and

w~.

In fact one

+ (Z2 _1)1/2),

(C.6)

w~n-I

2 (C.7)

( 2 Z

For

Z E [

-

1

sinh(n + 1) log(z + (Z2 -

1)1/

e, it follows that WI = e" = cosh nlog e'" = cosh nU} = cos ne,

-1, 1], by setting Tn(z)

)1/2

1

2).

Z

= cos

_ sin(n + 1)e

Un ( Z ) -

•

sin

e

and

,

which are the classical Chebyshev polynomials.

C.2.

Properties of Tn(z) and Un(z)

One easily proves that the roots of Tn(z) and Un_l(z) are 2k + 1 1T n 2

k=O,I, ... ,n-l,

Zk=COS---,

(C.8)

and Zk =

k1T

cos-, n

k

= 1,2, ... , n -

1,

(C.9)

respectively. Other properties of T; (z) and U; (z) are the following, which can be verified easily. (1) (2) (3) (4) (S)

(6) (7) (8)

Tn(z) + (T~(z) - 1)1/2, Tn(z) = Ln(z), (symmetry) 1jn(z) = 1j(Tn(z)), (semigroup property) ~n-I(Z) = ~_I (Tn(z)), Uc; = -Un_l(z), Tn- 1(z) - zTn(z) = (1- Z2)Un_l(z), Un-I(z) - zUn(z) = -Tn+l(z), Un+iz) + Un_j(z) = 21j(z) Un(z).

w~ =

C.2.

Properties of T.(z) and U.(z)

215

From the last property one can derive many others. For example, (9) U.+j_1(z) + U._j_1(z) = 27j(z)U._1(z), (10) U.+j(z) + U._ j_ 2(z) = 27j+l(Z)Un_1(z), (11) 2T.(z) U.(z) = U2.(z) + 1, (12) 2T.+1(z)Un- t(z) = U2.(z) -1, (13) 2T.(z) = U.(z) - U.-iz).

Among the properties of Chebyshev polynomials the following has a fundamental importance in approximation theory. (14) Let P; be the set of all n degree polynomials having as leading coefficient 1. Then for any p(z) E P«, max

-l~zSl

12 1 - · Tn (z)l :::; max

-lsz~l

Ip(z)l.

The proof of this property is not difficult and can be found in several books on numerical analysis. (15) T.(z) and U.(z), as function of z, satisfy the differential equations T~(z)

= nUn-1(z);

= _(Z2 _l)-l[ZU.(Z) - (n + l)T.+1(z)], (1- z2)T~(z) - zT~(z) + n 2T.(z) = 0, Z2) U~(z) - 3zU~(z) + n(n + 2) U.(z) = O.

(C.l0)

U~(z)

(1 -

(C.lI) (CoI2)

The first two are consequences of the definitions (C.6) and (C.?) and the other can be derived from them. Finally T.(z) and U.(z) satisfy the orthogonal properties. for n = 0, for n > 0,

j7T . where Zj = cos - , ] = 1'0' 0' n - 1. n

Solutions to the Problems

Chapter 1 1.1.

From (1.2.12) Ys =

YI

Y2 Y3 Y4

we get Ys 1.3.

E = I;=o e)ll 4

jYI and from the scheme

YI

110

III

-2 -2

o

11 2

11 3

11 4

2 2

o

o

2 4

0 4

= 10.

The first result of problem 1.2 can also be written 11

ei(ax+bl

= 2 e i(ax+b+a/2) ( e ia/2 ~ e -ia/2) = 2i sin ~ e i(ax+b+a/2).

By taking the real and imaginary parts one gets the result. 1.4.

Using the Sterling transform one has / 11-1/

=

·(2)

.(4)

2

4

=/0 + 3/2 ) + /3)

and

L + f3) + L from which one obtains:

I/ = 11n

j=O

1 /

Ij=n+1 j=O

n 2(n + I? = -'---'--

4

217

218

1.5.

Solutions to the Problems

From the result of problem 1.3 we have

a-1cos(ax+b+a/2) Setting a

=

sin(ax + b) . / . 2sma 2

= q, b = -q/2 and x = n, we have: A-I

/.1

cos

qn

=

sin(qn-q/2) .

2 Sill q/2

and

f

L..-

;=1

.

cos ql =

A-I /.1

.1 i=n+1

cos ql

;=1

=

sin[(n +

1)q - q/2] - sin q/2 . . 2 Sill q/2

1.7.

and x(x) = f(x + 1), f(x + 1) n)(x-n) = f(x - n + 1) we have x(n) = ----'-----'-f(x + n + 1)

1.10.

Using the Sterling transformation, p(x) can be written in the form p(x) = I~=o ax'", Applying a,a2 , ••• , ak and putting x = 0 one aip(O) gets a, = -.-,-.

From

x(n+x-n) = x(n)(x - n)(x-n)

(x-

1.

1.11.

One has:

.)=Q-f (_IY( n +]q .)+(_1)q-n 1

Qf(-I Y( q j=O n +]

j=O

= q-n-I L (-1)1"[(q-l) + (q-l)] j=O n +j n +j - 1 + (_1)Q-n

=

q-f- (-ly(qn -+]~) + q1:1

j-O

1

(-ly(q -

1)

q- 1

j=O

x (_1)q-n q-n "( q-l) = q-n-I L (-1)1"(q-l) . + L (-1)1 . j-O n +] j=O n +] - 1 q-n-I

= j~O +

"(q - 1)

(-1)1 n + j

(qn-l -1).

+

q-n

j~1

"(

q- 1)

(-1)1 n + j - l

Chapter 1

219

By setting j - 1 = i in the second term, it is easily seen that the two sums cancel. 1.12.

One has:

= q-fl (-l)j-1(

S(q, I, n)

j~1

(j) + (_l)q-n-l(q -I n)

q .) n +] I

= q-fl (_l)j-l(q - ~)(j) n +]

j=/

+

)

I

q-fl (-I)j-1( n q~" (j) +] -1 I j=l

+

qf (_l)j-l(

j~l+l

= q~f-l (-l)j-t(q -~) J~/

n

(j)

q- 1 ) +( q- 1 ) n +j - 1 I n+I- 1

+]

[(j)I _(j +I 1)]

( q-l )= (q-l) n+l-l .

+ n+l-l

en - e2 •

1.23.

The solutions are Yn

1.24.

the equation becomes Zn+1 = z.; + 1. m, Letting n = 2 Ym = C(2 m) and gm-l = f(2 m), one has Ym m) 2Ym-1 + f(2 =::: 2Ym-1 + gm-h Yt =::: 2. The solution is Ym 2m - s - l 2 m - l Yl + ",m-l 4..s~1 gs .

1.25.

Letting Yn

=

= Z;;-l

1.26.

Letting Zn

1.27A.

From log Yn+1 2" Yn =::: Yo .

1.27B.

. b ecomes Zn+1 = Zn, 2 f rom 1 - z; t h e equation By su bstituti stituting Yn = -2-

=:::

A 1/2 cot Yn one has Yn+1

=::: =:::

= 2 log Yn

which one obtains Yn 1.27C.

Let z;

= 1-

1.28.

Yn+l(l

+ y~) + Yn

=:::

=:::

2Yn.

one obtains log Yn

= 2n log Yo and

![1 - (l - 2Yo)2"].

Yn and obtain the equation of problem 1.26. =:::

k.

then

220

Solutions to the Problems

1.30. Un

From the result of problem 1.29 one has

~ 13 eXP(H I/2M J=O nf ( _1')1/2) n ]

s=O 2 It is known that I~=I k- / < 2n l / 2 1

Un

+ 13 nf eXP(h l / 2M nf ( _1 )1/2)'

~ 13( (exp(h 1/2 M(2n 1/2 -

-

r=s+1 n 1. Then one has

r

1)) + ~~~ exp(h 1/2 M(2(n - s - 1)1/2 -1))

1.31.

Let Vn = C + I~=n+l k,Vs ' One has Yn ~ Vn and Vn = Vn+ 1 + kn+1 Vn+l' that is Vn = n:=n+l (l + k.) V, ~ ~ exp(I:=n+1 ks ) and for t ~ 00, Yn -s Voo exp(I~=n+l k s ) = C exp(I~=n+l k s ) '

1.32.

For a < 1 one obtains Y~+l < y~ + bn(Yn + Yn+l) and then Yn+1 < Yn + b.; from which Yn ~ Yo + I;~ b.. For a > 1 one has Y~+l aYn2 --< bn(Yn+ Yn+1 ) an d t h en Yn+1 - a 1/2Yn

bn 1/2 Yn + Yn+l < bn, a Yn + Yn+1 from which one obtains Yn < (a 1/2)nyo + I;:~ (a 1/2r-j - 1b.. ~

Chapter 2 2.8c. 2.l3c.

Write the equation (2.3.1) in the form LlYn+l = Yn and apply Ll- I to both sides. In (2.5.10) one imposes q(x) = p'(x) from which it follows YoPo = + POYI = 2p2, .... Let ei+1 = Si+l - 8i+1> r, = ai+1 - ai+1> l3i = 1O-'(5i + ai+I)' One has ei+l = e, + ri - l3 i, from which Ienl ~ k; + 10-' II~:~ l3i l. From this one deduces that one must keep II 13;1 as small as possible and this can be achieved summing first the smaller a, in absolute value. If lail < a, then Is;1 ~ ia. Finally one has leNI -s k N + lO-'a I~~I i ~ . that t h e error grows lik N 2. k + 10- ta N(N - 1) showing 1 e

PI> YOPI

2.14.

N

2.19

2

From (2.5.18) we have

221

Chapter 3

where q~2) = q\2) = 0 and qn = 1/ n. The roots of denominator are 1 and 2. In the unit circle it can be written:

1 ( -2 Z2

-

3 -2 Z

+2

)-1

with 'Yi bounded and lim r,~ 'Yi 1=0

00 I

'YiZ i

i=O

h were Max

lss~n

-

Ci -

00 I

00 Ii=O 'YiZi

=

= 00

q~2)zn+2 =

(why?). Then one has

00 I

CiZi+2,

Yn

= Cn-2 = 4..j=o j + 1 'Yn-j-2 $

i=O

n=O

"\'i (2) 4..n=2 qn 'Yi-n'

Then

1

,,\,n-2

r,;=1 Y): The last inequality follows from the result of problem

1.13a. 2.20.

Taking f = 0, the error equation becomes p(E)en = p(1) whose solution is the sum of the general solution of the homogeneous equation and a particular solution. The general solution is given by (2.3.14) from which it follows that p(z) must be a Von Neumann polynomial.

2.21.

Taking f = 0 as in the previous problem and eo = e1 = ... = ek-l = 0, we have (see 2.4.11) en = p(l)/ p(l) and this cannot be zero for n ~ co (nh $ T) unless p(1) = O. Similarly for f == 1: p(E)en = h(p'(1) - 0"(1».

2.24.

Use Theorems B.l and B.2 of Appendix B. For case a) one finds D = [-i, i].

Chapter 3 3.7.

Exchanging

with

care

sum in the expression one has gi r,~~opH(n + ki,j) + gn = g.; (Remember that some values of H are zero).

r,~=OPi(n) r,;:~o H(n + k -

3.11.

Izsl ~ 1, S =

3.12.

It depends on the roots of

H(n

the

ti:

t.D«;

1,2. Z2

+ PIZ + P2' It will be

+ 1) + PIH(n) + P2 H(n -1) = 0 H(l)

+ PIH (O) + P2 H( -1)

= 1.

for n

~

0

222

Solutions to the Problems

According to the values of the roots, several cases are possible. For example: (a) IZII < 1, IZ21 > 1

H(n)

I)-I

Z; {(ZI + PI + P2 Z(ZI + PI + P2Z21)-IZ~

=

(b) IZII < 1, IZ2/ < 1, H(n) H(n) = 0 for n ~ O.

= (ZI -

for n ~ 0 for n ~ 0

Z2)-I(Z~ -

zD for n ~ 0 and


3.13.

H (n

3.14.

Let y~ the nth element of the i th column of the inverse matrix C 1.}. By multiplying C N and its inverse one shows that the columns of C-;J are solutions of the boundary value problems:

sm

sm

,'Y~i~1 + ay~i) + f3y~L ay~i)

= 8~,

+ f3Ai) =

rY<;)-1 + ay<;)

n = 2, 3, ... , N - 1

8:

= 8~

for i = 1,2, ... ,N. Let ZI and Z2 be the roots of The solutions are:

y~i)

=

/-1

H N+2

r + az + f3z2 = O.

)H(N-i+2)H(n+1)

for n es i

- /-1 ) H(N _ n + 2)H(i + 1)(zl z2)n-

ifor

H N+2

where H(n)

=

n-I

Z2

n-I

- ZI

Z2 - ZI

. One

~ i,

sees that for IZIZ21 ~ 1 and IZ21 > 1,

the elements remain bounded for N 3.15.

n

~ 00.

Suppose that A(n) is a companion matrix, that is ofthe form (3.3.3). AT (n) is not of the same form and then the components of X n, given by (3.4.9) are not successive values of the solution of a scalar equation. Now consider the matrix - Pk( n ) V(n) =

o

(

:

o

-Pk-I(n) -Pk(n + 1)

., . .,.

-PI(n)

-pin + 1) -Pk(n

+ k)

)

223

Chapter 3

verifies that x~ = X~+1 V(n + 1), where x~ = k - I ) and t; is the last component of Xn. For this new one has Xn=X~_IV-I(n)=x~A(n)V-I(n)= vector xn X~+I V(n + 1)A(n)V- I(n) == x~+ID(n). The matrix D(n) == V(n +.1)A(n)V- 1(n) is given by One

easily

u; tn+I , · .. , tn+

D(n)

=

(

;:~: : ~~ ~ ~ ::: 0) .

: Pk(n

+ k)

1 0

0

,

which is the companion form for the adjoint equation defined in Section 2.1. 3.20.

From the definition, one has G(j,j)

=

-cI>(j, O)Q-l L~o L;cI>(n;,j + 1)T(j + 1, n;)

and G(j + l,j)

= cI>(j + I,j + 1) - AcI>(j, O)Q-I X

3.21.

L~o L;cI>(n;,j + 1)T(j + 1, n;) = 1+ AG(j,j).

One has N

2:

N

Lp(n;,j)

=

;'TO

2:

L;cI>(n;,j

;=0

N

- 2:

+ 1)T(j + 1, n;)

LjcI>(nj,O)Q-I

;=0 N

X N

=

2:

2: LscI>(n"j+1)T(j+l,ns) s=O

LscI>(n"j + 1)T(j + 1, ni)

i~O

N

- 2: 3.22.

From Yn+1

=

LscI>(n"j + 1)T(j + 1, n.)

=

O.

s~O

cI>(n

+ 1, O)Q-I W +

nN-l

2:

G(n

+ 1, svb,

5=0

= AcI>(n, O)Q-I W+

"N- 1

2:

s=O

AG(n, s)bs + b;

+ AG(n, n'ib;

s>'n = A[cI>(n, O)Q-I W

+

"N- 1

2:

s=O

G(n, s)bsl + b;

= AYn + b.:

224

Solutions to the Problems

For what concerns the boundary conditions one has: N

I

i=O

N

LJ'n,

=I

Lj(nj, O)Q-I W +

i~O

N

= w + i~O

n

-I

+

n

-I

-Jo

i=O s=O

L j (n j, s + I)T(s

%0

%0

nN-I

(

- (n;, O)Q-I = W

N

I I t.ot»; sib,

Jo

+ 1, nj)

Lj(nj, s + I) T(s

+ 1, nj)

)bs

(N

;~o Lj(n;, s + I) T(s + 1, n;)

Lj(nj, s + 1) T(s

+ 1, nj »)b, = w.

Chapter 4 4.3.

The system is symmetric with respect to the origin: gi( -x, -y) = -gi(X, y) for i = 1,2. This allows us to consider only the upper half plane. According to the signs of gl and g2, let us consider the following sets: A

= {(x, y) Iy 2: 2x}

B

= {(x,y)ly < 2x

c

= {( x, y) x (y - x)

I

2

where gl(X, y) 2: 0, g2(X, y) > 0 and x 2(y - x) + y5 2: O}

+ Y 5 < O}

where gl(X, y) 2: 0, gix,y) < 0 where gl(X, y) < 0, g2(X, y) < O.

It is clear that for (Xb Yk) E A, LiXk =:: 0, LiYk =:: 0, that is both the sequences Xb Yk are not decreasing. If they remain in A, they will · . gix,y) h b never cross t h e 1me y = 2x and the ratio ( ) as to e greater gl x,y ,

2

;( (y - ~X) 5> 2, which is impossxy-x+y ible because the y in the denominator has degree larger than the y in the numerator. The sequences must cross the line y = 2x and enter in B. In a similar way it can be shown that they cannot remain in than 2 for all k. This means that

225

Chapter 4

B. They will enter in C, where both ~Xk and ~Yk are negative and the sequences are decreasing. Now if Yk > 0 it follows that

Yk+1

= Yk +

=

4.5.

Y~(Yk - 2Xk) 2

'k

+

6

'k

=

Yk«Xk - Yk)2 + y~ +

,i+ r%

Yk('~ + '% + y~ - 2XkYk) 2

,n >0

'k

+

6

'k

and similarly for xi, This shows that the sequences must remain in C and must converge to a point where both ~Xk and ~Yk are zero, which is the origin. Starting from (-8,0) for small positive 8, it is easy to check that in the following iterations the points (Xk, Yk) have increasing distance from the origin until they reach the regions B or C, showing that the origin is unstable. The solution is log(no + 2) y(n, no, Yo) = log(n + 2) Yo· The series L:=no Iy(n, no, Yo)1 does not converge, showing that the origin is not II-stable.

4.9.

In this case D is the set of all positive numbers. If Yo E D, then Yn E D for all n. Consider V(y) = Y J(l + y 2). It is V(y) 2: 0 and

~V() Yn

=

Yn(1-Yn?(Yn-l) (l + y~)(l + y~)

=-

() W Yn < O.

The set E is {O, I} and W(x) ~ 0 for x ~ 00. Then, according to the theorem Yn is either unbounded or tends to E. In fact the solution is Yn = y~_2)n and it tends to zero if Yo < 1 and n even and it is unbounded for n odd. If Yo = 1 then Yn = 1 for all n. 4.10. The eigenvalues of A are Al =! with multiplicity s - 1 and A2 = (1 - s)J2. They can be obtained easily by considering that A is a circulant matrix and the eigenvalues are the values assumed by the polynomial p(z) = -!(z + Z2 + ... + ZS-I) on the Slh roots of the unity. There is global asymptotic stability for s = 2, only stability for s = 3 and instability for s > 3. 4.11. In order to have Vn+ 1 2: 0, it must be Vn - w( Vn) 2: O. If u and v are such that U 2: n, U - w(u) 2: 0, V - w(v) 2: 0, and U - v 2: w(u) - w(v) one has u - w(u) 2: V - w(v). The solution satisfies Un = Uo - L;:~ w(Uj)' Since w is increasing and Un must remain positive, it follows that Uj ~ O.

226

Solutions to the Problems

4.12. Let x E O(Yo), there exists a sequence n, ~ 00 such that y(n j , no, Yo) ~ x. But y(ni+l,no,Yo)=f(y(nj,no,Yo» and limy(nj+j,no,Yo)= "j""'OO

f(x), showing thatf(x) E O(Yo) and that O(Yo) is invariant. Now let Yk be a sequence in O(Yo) converging to y. We shall prove that y E O(Yo). For each index k, there is a sequence m~ ~ 00 such that y(m~, no, Yo) ~ Jk. Suppose for simplicity that dist(Yk, y(m~, no, Yo» < k- 1 and m~ ~ k for i ~ k: Consider the sequence mk = m~. Then dist(y, y(mb no, Yo» $ dist(y, yd + which implies dist(y, y(mb no, Yo» ~ 0 and then y E O(Yo).

u:',

Chapter 5 5.2.

From the mean value theorem one has

F(x) - F(y) - F'(x)(y - x) =

IIF(x) 5.4.

Let ao

f

[F'(x

F(y) - F'(x)(y -

= ~f3'Y.

One has

I/Xn+l -

x, II

$

x)11

+ s(y - x) $

'Y

fs

F'(x)] ds(y - x);

dslly -

x11

2

•

aollxn - xn_111 2

and I/X k+m -

xkll $ $

$

$

k+m-l

Ilxj+! - xJ $ I a~J-lllxk - xk_dl 21 j=k j=1 Ilxk - Xk_111 2 f: a~1-11Ixk - Xk_111 2L2 j=l m [x, - Xk_111 2I a~1-1ut~12 j=l Ilxk - Xk_111 2I a~1-11J21-2( a 2k- 1?1- 1 j=l

I

m

227

Chapter 5

5.5.

. atn u; Un-I C onsiid er t h e equation Un = --Un-I' One has A = - - , that is ~tn-I /.Jotn ~tn-I U

~ is constant and must maintain its initial value UI/ t., Then

txt;

Un = ~tn(UI/tl)' It follows then that ~Xn:5 Un' Moreover decreasing that is for m , ,

Ilxn+p -

x, I

= t +

n

- t tn + 1 - tn

I ~xm I :5 I ~xn I

'~tn

~tn

:5 I;':~ Ilxn+j+1 - xn+J :5 "p-I at .11~Xn+jll:5 ,,~-I ~t n+J

L.J~I

n p

n

:2:

~t. n+)

L.J=I

lI~xnll

--A-

tu;

.

1S

and

.11~xn I

n-r]

~t

n

II~xn I

from which it follows that

lim Ilxn+ p

r-«

Being II ~xn I

:5 Un

xnll:5

-

one has

II x *- X 5.6.

t* - tn Ilaxnll. tn + 1 - tn

n

I 11«t*-tn)u • tl

In this case the comparison equation is ~tn y _ u; - (~tn-If Un-I>

whose solution is Un

5.7.

Let Yn

=1 -

= ~tn (~:) yH.

As before one has

. Zn. The equation becomes Yn+1

1 - 2Zo) ="21( Yn + ~

and

apply the result of problem 1.26.

5.8.

From the given solution one has Zo = 1 - (l - 2Z0 ) 1/ 2 coth k from which k = log 0- 1/ 2 and then

z; = 1 -

1+ 0

2"

- - 2 " (l

1-

e

1/2

- 2zo)

228

Solutions to the Problems

from which [x" - x

1

n

II ~ -(z* P1

Z ) = n

2 -(1 - 2z

P1

0

(}2" )1/2 _ _ • 1 - (}2"

5.12. The equation (5.5.2) is the homogeneous equation related to (5.5.1) when one takes

x;

Yn-I Yn

=--.

Imposing that (5.5.4) satisfies the nonhomogeneous equation one gets (5.5.3). In fact one has Yn+1 - 20Yn + xnYn + Zn = gn from which Yn =

Yn+1 20 - X n

z; -

+ 20 -

gn Xn

== Xn+IYn+1 + Zn+I'

Chapter 6 6.1.

Consider the term I;:~ A n-j-I Wi and the decomposition (A2.2). It follows that A = ZlI + I~=2 (AkZ kl + Zk2) == 8 + 8], where d is the number of distinct eigenvalues of A. Using the properties of the component matrices Zkj it is seen that 8 j = 8 for all j and jim S{ = O. )->00

Moreover Aj = sj + 8{. The sum I;:~ An-j-I Wi becomes 8 I;:~ Wi + I;:~ 8~-j-1 Wi. The quantity 8Wi is called essential local error. If the errors are such that SWi = 0 then one can proceed as usual. 6.5.

Applying the theorem B3 one has p(l)(z) = -4 Req and p'(z) 2(z - q). It follows that p(z) E N iff Req = 0 and Iql < 1.

6.8.

Rewrite the equation (6.1.15) such that the linear autonomous matrix A is the companion matrix of O'(z), obtaining En + 1 = (A + Bn)En + W n, where B; =
=

b. = (Xj - {3j + h{3j({3kCn+k - cn+j) } 2 - h{3kCn+k .

Find a bound for bj independent on n and consider that by hypothesis IIAII < 1.

6.9.

Consider the nonhomogeneous equation CNX N - Ax N = O(Nl, where IlxNII = 1 for all Nand Ilo(Nlll = 8 2• Componentwise one considers the equation PX;:_I + «(X - A)x;: + YX;:+1 = 0;: whose solutions are

229

Chapter 7

chosen such that the initial condition (a - A)xi' + ,,/xf = 0 is satisfied. In the hypothesis made the solution will diverge and (6.7.1) will not be satisfied. 6.12.

The matrix AN is the matrix representing the centered second order discretization, that is

AN =

-2 1 0

1 -2

0 1

.,

0

.

0 0 0

1 -2 NxN 0 1 2A D 2 _ (IlX N 2N 0 Il)A N)' which is symmetric. The spectrum of (llx 2)- 1D~N is S = {-2 + 2 cos 8, 0 ~ () < 7T}. The spectrum of D 2N is then SI/2 (see proposition 6.7.3), which is an interval of the imaginary axis. The midpoint rule can be used because its region of absolute stability lies on the imaginary axis and Ilt can be chosen appropriately in order that the region contain SI/2.

0

Chapter 7 7.2.

From 2.5.10 one has X(z) =

q(Z) "M aiz

i ' The problem reduces to 1 - "-i=1 the study of the roots of the denominator. They are the reciprocal of the characteristic roots and they are outside the circle B(O, A~I). The series X(z) = L:=o xl(n)zn is then convergent in this circle. To see if ZI = A~I is less or equal to 1, one considers that 1 - L~I a.z' is monotone decreasing on the real line from 1 to -00 and the ZI will be less or greater than 1 according to the sign of 1 - L a.. If this quantity is positive, then Z I > 1, otherwise Z I < 1. In the first case one has

co

1

n~o xl(n) =.1- I

aj

and xl(n) ~ O. In the second case xl(n) ~ 00. If 1 Theorem 2.5.2 the solution xI(n) is unbounded.

L a, = 0 then by

230

7.6.

Solutions to the Problems

=

f(x) - f(2 J(X)

. Ifj<2)(x) x-y tion assumes the value -1, and vice versa.

By definitionf[x,f(x)]

= x then the frac-

7.9. The model is PI = PoPo (n

+ l)P + 1 -

(Pn + n)Pn + Pn-1Pn- 1 = 0

n

NPN - PN-1P N- 1 = O.

To obtain the solution, observe that (n

+ l)Pn + 1 -

p.P; = nPn - Pn-1Pn- l

,

That is nPn - Pn-1Pn- 1 = K, where K is a constant. From the first equation one sees that K = O. One obtains n-I

P = n

and then from

I.:I Pi = 1 one

TI

i=O

n!

Pi

P 0

obtains Po.

References

[I] Agarwall, R. P. and Thandapani, E., On some new discrete inequalities, App!. Math. and Computation, 7 (1980), pp. 205, 224. [2] Agarwall, R. P. and Thandapani, E., Some inequalities ofGronwa/l type, Analele Stjietifice Univ. Iasi., XXVII (1981), pp. 139, 144. [3] Agarwall, R. P., On multipoint boundary value problems for discrete equations, Jour. of Math. Analysis and App!., 96 (1983), pp. 520-534. [4] Agarwall, R. P., Initial value method for discrete boundary value problems, Jour. of Math. Analysis and App!., 100 (1984), pp. 513-529. [5] Agarwall, R. P., Difference calculus with applications to difference equations, In W. Walter, General Inequalities 4. [6] Albrecht, P., Numerical treatment of ODE: the theory of A-methods, Num. Math., 47 (1985), pp. 59-87. [7] Alekseev, V. M., An estimate for the perturbation of the solution of ordinary differential equation, (Russian) Vestu, Mosk. Univ. Ser I, Math. Med., 2 (1961), pp. 28-36. [8] Arneodo, A., Ferrero, P. and Tresser, C., Sharkovskii orderfor appearance ofsuperstable cycles: An elementary proof, Comm. Pure and App!. Math. XXXVII (1984), pp. 13-17. [9] Atkinson, F. V., Discrete and continuous boundary value problems, Academic Press (1964). [10] Bakhvalov, N. S., Numerical Methods, MIR, Moscow, (1977). [11] Barna, B., Uber die diuerenzpunbe des Newtonches verbfahrens zur bestinemmung von wurzelu algebraischen, Publications Mathematical Debreceu, 4 (1956), pp. 384-397. [12] Barnett, S. and Silijak, D. D., Routh algorithm, a centennial survey, SIAM Review, 19 (1977), pp.472-489. [13] Bermand, A. and Plemmons, R., Nonnegative matrices in the Mathematical Sciences, Academic Press, New York, (1979). [14] Bernussou, J., Point Mapping Stability, Pergamon, (1979). [15] Beyn, W. J., Discrete Green's functions and strong stability properties ofthe finite difference method, Appl. Analysis, (1982), pp.73-98. [16] Brand, L., Differential and Difference Equations, J. Wiley, (1966). [17] Burrage, K. and Butcher, J. c., Stability criteria of implicit Runge-Kutta methods, SIAM JNA, 15 (1979), pp.46-57.

231

232

References

[18] Burrage, K. and Butcher, J.e., Nonlinear stability for a general class ofdifferential equation methods, BIT, 20 (1980), pp. 185-203. [19] Bus, J. e. P., Numerical solution of systems of nonlinear equations, Mathematical Centre Tracts, Amsterdam, (1980). [20] Butcher, J.C., A stability property for implicit Runge-Kutta methods, BIT, 15 (l975), pp.358-361. [21] Cash, J. R., Finite Reccurrence Relations, Academic Press, (1976). [22] Cash, J. R., An extension of Olver's method for the numerical solution oflinear recurrence relations, Math of Comp., 32 (l978), pp. 497-510. [23] Cash, J. R., Stable Recursions, Academic Press, London, (1979). [24] Cash, J. R., A note on the solution of linear recurrence relations, Num. Math., 34 (1980), pp.371-386. [25] Clenshaw, C. W., A note on the summation of Chebyshev series, MTAC, 9 (1955), pp.118-120. [26] Coffman, C. V., Asymptotic behavior of solutions of ordinary differential equations, Trans. Am. Math. Soc., 110 (1964). pp. 22-51. [27] Collatz, L., Einige anwendungenfunktionanalytischer methoden in der praktischen analysis, Z. Angen. Math. Phys., 41 (1953) pp.327-357. [28] Collatz, L., Functional Analysis and Numerical Mathematics, Academic Press, (1966). [29] Collet, P. and Eckmann J. P., Iterated Maps on the Interval and Dynamical Systems, Birkhauser, London, (1980). [30] Corduneanu, C., Principles of Differential and Integral Equations, Chelsea, New York, (1977). [31] Corduneanu, C., Almost periodic discrete processes, Libertas Math., 2 (1982), pp. 159-169. [32] Dahlquist, G. and Bjork, A., Numerical Methods, Prentice-Hall, (1974). [33] Dahlquist, G., A special stability problem for linear multistep methods, BIT, 3 (1963), pp.27-43. [34] Dahlquist, G., Error analysis for a class ofmethods for stiffnonlinear initial value problems, Num. Anal., Dundee, Springer Lect. Notes in Math., 506 (1975), pp.60-74. [35] Dahlquist, G., On stability and error analysis for stiff nonlinear problems, Part I, Report Trita-NA-7508, (1975). [36] Dahlquist, G., G-stability is equivalent to A-stability, BIT, 18 (1978), pp.384-401. [37] Dahlquist, G., On the local and lobal errors ofone-leg methods, Report, TRITA-NA-8110, (1981). [38] Dahlquist, G., Some comments on stability and error analysis for stiffnonlinear differential systems, preprint, NADA Stockholm, (1983). [39] Dahlquist, G., Liniger W., and N evanlinna 0., Stability of two step methods for variable integration steps, SIAM JNA, 20 (1983), pp. 1071-1085. [40] Deuflhard, P. and Heindl G., Affine invariant convergence theorems for Newton's method and extensions to related methods, SIAM JNA, 16 (1979), pp. 1-10. [41] Deuflhard, P., On algorithm for the summation of certain special functions, Computing, 17 (1976), pp. 37-48. [42] Deuflhard, P., A summation technique for minimal solutions of linear homogeneous difference equations, Computing, 18 (1977), pp. 1-13. [43] Diamond, P., Finite stability domains for difference equations, Jour. Austral. Soc., 22A (1976), pp. 177-181. [44] Diamond, P., Discrete Liapunov function with V> 0, Jour. Austral. Soc., 20B (1978), pp.280-284. . [45] Di Lena, G. and Trigiante D., On the stability and convergence of lines method, Rend. di Mat., 3 (1982), pp. 113-126.

References

233

[46] Driver, R. D., Note on a paper of Halanay on stability offinite difference equations, Arch. Rat. Mech., 18 (1965), pp. 241-243. [47] Fielder, M. and Ptak, V., On matrices with nonpositive off-diagonal elements and positive principal minors, Czechoslovakia Math. lour., 12 (1962), pp.382-400. [48] Fielder, M. and Ptak, V., Some generalizations of positive definiteness and monotonicity, Num. Math., 9 (1966), pp.163-172. [49] Fort, T., Finite Differences and Difference Equations in the Real Domain, Oxford, (1948). [50] Gandolfo, G., Mathematical Methods and Models in Economics Dynamics, NorthHolland, Amsterdam, (1971). [51] Gantmacher, F. R., The Theory of Matrices, Vol. 1-2, Chelsea, (1959). [52] Gautschi, W., Computational aspects of three terms recurrence relations, SIAM Rev., 9 (1967), pp.24-82. [53] Gautschi, W., Minimal solutions of three term reccurrence relations and orthogonal poly. nomials, Math. of Computation, 36 (1981), pp. 547-554. (54] Gear, G. W., Numerical Initial Value Problems in Ordinary Differential Equations, Prentice-Hail, (1971). [55] Gear, G. W. and Tu K. W., The effect of variable mesh size on the stability of multistep methods, SIAM lNA, 1 (1974), pp. 1025-1043. [56] Gekerel, E., Discretization methods for stable initial value problems, Lecture Notes in Math., Springer, (1984). [57] Gelfand, A. 0., Calcul des Difference Finies, Dunod, Paris, (1963). [58] Gelfand, A. P., Calculus ofFinite Differences, Hindusten Publishing Corp., Delhi, (1971). [59] Godunov, S. K. and Ryabenki, V. S., Theory of Difference Schemes, North-Holland, (1984). [60] Goldberg, S., Introduction to Difference Equations, 1. Wiley, New York, (1958). [61] Gordon, S. P., Stability and summability of solutions of difference equations, Math. Syst, Theory,S (1971), pp. 56-75. [62] Grobner, W., Die Lie-Reiben und Ihre Anwendungen, Berlin, DVW, (1960). [63] Grujic, L. 1. T. and Siljak, D. D., Exponential stability oj large scale discrete systems, 1. Control, 19 (1976), pp.481-491. [64] Guckenheimer, 1., Oster, G., and Ipaktchi, A., The dynamics of density dependent population models, 1. Math. Biology, 4 (1977), pp. 101-147. [65] Hahn, W., Stability of Motion, Springer, Berlin, (1967). [66] Haight, F. A., Mathematidal Theories oj Traffic Flow, Academic Press, (1963). [67] Hairer, E. and Wanner, G., Algebraically stable and implementable Runge-Kutta methods of higher order, SIAM lNA, 18 (1981), pp. 1098-1108. [68] Halanay, A. and Wexler, D., Teoria Calitative a Sistemlor cu Impulsuri, Bucharest, (1968). [69] Halanay, A., Solution periodiques et presque-periodiques des systems d'equationes aux difference finies, Arch. Rat. Mech., 12 (1963), pp. 134-149. [70] Halanay, A., Quelques questions de la theorie de la stabilite pour les systemes aux differences finies, Arch. Rat. Mech., 12 (1963), pp. 150-154. [71] Hang-Chin, Lau and Pou-Yah, Wu, Error bounds for Newton type process on Banach spaces, Num. Math., 39 (1982), pp, 175-193. [72] Hartman, P. and Wintner, A., On the spectre of Toeplitz matrices, Am. 1. of Math., 72 (1950), pp.359-366. [73] Hartman, P., Difference equations: disconjuancy, principal solutions, Green's Junctions, complete monotonicity, Trans. Am. Math. Soc., 246 (1978), pp. 1-30. [74] Hebrici, P., Discrete Variable Methods for Ordinary Differential Equations, 1. Wiley, (1962). [75] Henrici, P., Error Propagation for Difference Methods,l. Wiley, (1963).

234

References

[76] Henrici, P., Applied and Computational Complex Analysis, Vol. 1, J. Wiley, New York, (1974). [77] Hildebrand, F. 8., Finite Difference Equations and Simulations, Prentice-Hall, (1968). [78] Hildebrand, F. 8., Methods of Applied Mathematics, Prentice-Hall, (1952). [79] Hoppensteadt, F. C. and Hyman, J. M., Periodicsolutions ofa logistic difference equation, SIAM JAM 32 (1977), pp. 73-81. [80] Hoppensteadt, F. c., Mathematical Methods of Population Biology, Courant Inst. of Math. Science, (1976). [81] Hurt, J., Some stability theorems for ordinary difference equations, SIAM JNA, 4 (1967) pp. 582-596. [82] Jones, G. S., Fundamental Inequalities for discrete and discontinuous functional equations, J. Soc. Ind. Appl. Math., 12 (1964), pp.43-57. [83] Jordan, c., Calculus of Fnite Difference, Chelsea, New York, (1950). [84] Kalman, E. and Bertram, J. E., Control System analysis and design via the'second method' of Lyapunov, Part 11, Discrete time Syst., Trans. ASME, Ser. D.J. Basic Enr., 82 (1960), pp. 394-400. [85] Kannan, R. and Ray, M. B., Monotone iterative methods for nonlinear equations involving a non-invertible linear part, Num. Math., 45 (1984), pp. 219-225. [86] Kato, T., Perturbation Theory for Linear Operators, Springer, (1966). [87] Khavanin, M. and Lakshmikantham, V., The method of mixed monotony and first order differential systems, Nonlinear Analysis, 10 (1986), 873-877. [88] LaSalle, J. P., The stability ofdynamical systems, Regional Conference Series in Applied Mathematics, SIAM (1979). [89] Ladde, G. S., Lakshmikantham, V., and Vatsala, A. S., Monotone Iterative Techniques for Nonlinear Differential Equations, Pitman Publishers Co., (1985). [90] Lakshmikantham, V. and Leela, S., Differential and Integral Inequalities, Academic Press, New York, Vol. I & II, (1969). [91] Lakshmikantham, V.and Vatsala, A. S., Method ofmixed monotonyfor nonlinear equations with a singular linear part, Appl. Math. and Computations, 23 (1987),235-241. [92] Lambert, J. A., Computational Methods in Ordinary Differential Equations, J. Wiley, (1973). [93] Lancaster, P., Theory of Matrices, Academic Press, (1969). [94] Levy, H. and Lessmann, F., Finite Difference Equations, McMillan, New York, (1961). [95] Li, T. Y. and Yorke, J.A., Period three implies chaos, Am. Math. Monthly, 82 (1975), pp. 985-992. [96] Liniger, W., Numerical solution of ordinary and partial differential equations, Notes of a cource taught at the Suisse Federal Institute of Technology, Lausanne, (1971-1973). [97] Lorenz, E. N., The problem ofdeducing the climatefrom the governing equations, TELLUS, 16 (1964), pp. 1-11. [98] Luenberger, D. G., Introduction to Dinamic Systems, J. Wiley, (1979). [99] Luke, Y. L., The Special Functions and Their Approximations, Academic Press, Vol. I, (1969). [100] Maslovskaya, L. V., The stability of difference equations, Diff. Equations, 2 (1966), pp.608-611. [101] Mate, A. and Nevai, P., Sublinear perturbations of the differential equation y = 0 and the analogous difference equation, J. Diff. Eq., 53 (1984), pp. 234-257. [102] Mattheij, R. M. and Vandersluis, A., Error estimates for Miller's algorithm, Num. Math., 26 (1976), pp.61-78. [103] Mattheij, R. M., Characterizations of dominant and dominated solutions of linear recursions, Num. Math., 35 (1980), pp.421-442.

References

235

[104] Mattheij, R. M., Stability of block LU-decompositions of the matrices arising from BVP, SIAM J. Alg. Dis. Math., 5 (1984) pp. 314-331. [105] Mattheij, R. M., Accurate estimates for the fundamental solutions of discrete boundary value problems, J. Math. Anal. and Appl., 101 (1984), pp.444-464 [106] May, R. M., Simple mathematical models with very complicated dynamics, Nature, 261 (1976), pp.459-467. [107] May, R. M., Biological populations with nonoverlapping generations. Stable points, Stable Cucles and Chaos, Science, 186 (1974), pp. 546-647. [108] McKee, S., Gronwall Inequalities, Zamm., 62 (1982), pp.429-431. [109] Meil, G. J., Majorizing sequences and error bounds for iterative methods, Math. of Computations,34 (1960), pp. 185-202. [110] Mickley, H. S., Sherwood, T. K., and Reed, C. E., Applied Mathemaitcs in Chemical Engineering, McGraw-Hill, N. Y., (1967). [Ill] Miel, G., An updated version ofthe Kantarovich theoremfor Newton's method, Computing, 27 (1981), pp.237-244. [112] Miller, J.C. P., Bessel Functions, Part II, Math. Tables, Vol X, British Association for the Advacement of Sciences, Cambridge Univ. Press., (1952) [113] Miller, J. J. H., On the location of zeros ofcertain classes ofpolynomials with applications to numerical analysis, J. Inst. Math. Applic., 8 (1971), pp.397-406. [114] Miller, K. S., An Introduction to the Calculus of Finite Differences and Difference Equations, Hold and Company, New York, (1960). [115] Miller, K. S., Linear Difference Equations, Benjamin, New York, (1968). [116] Milne-Thomson, L. M., The Calculus of Finite Differences, McMillan & Co., London, (1933). [117] Nevanlinna, o. and Odeh, F., Multiplier techniques for linear multistep methods, Num. Funct. Anal. and Optimiz., 3 (4) (1981), pp. 377-423. [118] Nevanlinna, o. and Liniger, W., Contractive methods for stiff differential equations, BIT, 19 (1979), pp. 53-72. [119] Nevanlinna 0., On the behaviour of the lobal errors at infinity in the numerical interaction of stable IVp' Num. Math., 28 (1977), pp.445-454. [120] O'Shea, R, P., The extention of Zubov method to sampled data control systems described by difference equations, IEEE, Trans Auto. Conf., 9 (1964), pp. 62-69. [121] Odeh, F. and Liniger, W., Non Linear fixed h stability of linear multistep formulas, J. Math. Anal. Appl., 61 (1977), pp. 691-712. [122] Olver, F. W. and Sookne, D. J., Note on backward recurrence algorithms, Math. of Computation, 26 (1972), pp.941-947. [123] Olver, F. W., Numerical solutions of second order linear difference equations, Jour. of Research, NBS, 7t8 (1967), pp. 111-129. [124] Olver, F. W., Bounds for the solution of second-order linear difference equations, Journal of Research, NBS, 718 (1967), pp. 161-166. [125] Ortega, J. M., Stability ofdifference equations and convergence ofiterative processes, SIAM JNA, 10 (1973), pp.268-282 [126] Ortega, J.M. and Rheinboldt, W. C., Monotone iterations for nonlinear equations with application to Gauss-Seidel methods, SIAM JNA, 4 (1967), pp. 171-190. [127] Ortega, J. M. and Rheinboldt, W. C., Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, (1970). [128] Ortega, J. M. and Voigt, G., Solution ofpartial differential equations on vector and parallel computers, SIAM Rev., 27 (1985), pp. 149-240. [129] Ortega, J. M., The Newton-Kantarovich theorem, Am. Math. Monthly, 75 (1968), pp. 658660. [130] Ortega, J. M., Numerical Analysis, Academic Press, New York, (1972).

236

References

[131] Ostrowski, A., Les points d'otraction et de repulsion pour l'iteration dans /'esace a n dimension, C. R. Acad. Sciences, Paris, 244 (1957), pp. 288-289. [132] Ostrowski, A., Solution of Equations and Systems of Equations, Academic Press, New York, (1960). [133] Pachpatte, B. 8., Finite difference inequalities and an extension of Lyabunov method, Michigan Math. J., 18 (1971), pp. 385-391. [134] Pachpatte, B. G., On some fundamental inequalities and its applications in the theory of difference equations, Ganita, 27 (1976), pp. 1-11. [135] Pachpatte, 8. G., On some discrete inequalities of Bellman-Bihari type, Indian J. Pure and Applied Math., 6 (1975), pp.1479-1487. [136] Pachpatte, B. G., On the discrete generalization ofGronwall's inequality, J. Indian Math. Soc., 37 (1973), pp. 147-156. [137] Pachpatte, B. G., Finite difference inequalities and their applications, Proc, Nat. Acad. Sci., India, 43 (1973), pp.348-356. [138] Pasquini, L. and Trigiante, D., A globally convergent method for simultaneously finding polynomial roots, Math. of Computation, 44 (1985), 135-150. [139] Patula, W., Growth, oscillations and comparison theorems for second order difference equations, SIAM J. Math. An., 10 (1979) pp.1272-1279. [140] Piazza, G. and Trigiante, D., Propagazione degli errori nella integrazione numerica di equazioni differenziali ordinarie, Pubbl. lAC III, Roma 120 (1977). [141] Pollard, J. H., Mathematical Methods for the Growth of Human Populations, Cambridge Univ. Press, (1973) [142] Popenda, J, Finite difference inequalities, Fasciculi Math., 13 (1981), pp: 79-87. [143] Popenda, J., On the boundness of the solutions of difference equations, Fasciculi Math., 14 (1985), pp. 101-108. [144] Potra, F. A., Sharp error bounds for a class of Newton-like methods, Libertas Matematica, 5 (1985), pp. 71-84. [145] Potts, R. B., Nonlinear difference equations, Nonlinear Anal. TMA, 6 (1982), pp. 659-665. [146] Redheffer, R. and Walter, W., A comparison theorem for difference inequalities, Jour. of Dill. Eq., 44 (1982), pp. 111-117. [147] Rheinboldt, W. C., Unified convergence theory for a class of iterative processes, SIAM JNA,5 (1968), pp.42-63. [148] Saaty, T. L., Modern Nonlinear Equations, Dover, New York, (1981). [149] Saaty, T. L., Elements of Queuing Theory with Applications, Dover, New York, (1961). [150] Samarski, A. and Niloaiev, E., Methodes de Resolution des Equation des Mailles, MIR, Moscow, (1981). [151] Sand, J., On one leg and linear multistep formulas with variable stepsize, Report, TRITANA-8112, (1981). [152] Schelin, C. W., Counting zeros of real polynomials within the unit disk, SIAM JNA, 5 (1983), pp. 1023-1031. [153] Schoenberg, I. J., Monosplines and Quadrature Formulae, In T.N.E. Greville, Theory and Applications of Spines Functions, Academic Press, (1969). [154] Scraton, R. E., A modification of Miller's recurrencealgorithm, BIT, 12 (1972), pp. 242251. [155] Sharkovskii, A. N., Coexistence of cycles of continuous map of line into itself, Ukrainian Math. J., 16 (1964), pp. 61-71. [156] Shintani, H., Note on Miller's recurrence algorithm, Jour. of Science of the Hiroshima University, Ser. A. 1,29 (1965), pp. 121-133. [157] Skeel, R., Analysis offixed step-size methods, SIAM JNA, 13 (1976), pp. 664-685. [158] Smale, S., The fundamental theorem of algebra and complexity theory, Bull. Am. Math. Soc.,4 (1981), pp. 1-36.

References

237

[159] Smith, R. A., Sufficient conditions for stability of a class of difference equations. Duke Math. Jour., 33 (1966), pp.725-734. [160] Spiegl, M. R., Finite Difference and Difference Equations, Schaum Series, New York, (1971). [161] Stephan, P., A" theorem of Sharkovskii on the existence of periodic orbits of continuous endomorphisms of the real line, Comm. Math. Phys., 54 (1977), pp. 237-248. [162] Straffin, P. D., Periodicpoints ofcontinuous functions, Math. Mag., 51 (1978), pp. 99-105. [163] Sugiyama, S., Difference inequalities and their applications to stability problems, Lec-· tures Notes in Math., Springer, 243 (1971), pp. 1-15. [164] Svirezhev, Y. M. and Logofet, D.O., Stability of Biological Community, MIR, Moscow, (1983). [165] Toeplitz, 0., Zur Theorie der Quadratischen Fozunlu von unendlichrillen Yerenderlichen: Gitinger Nachtrichten (1910), pp. 351-376. [166] Traub, J. F., Iterative Methods for the Solution of Equations" Prentice-Hall, (1964). [167] Trigiante, D. and Sivasundaram, S., A new algorithm for unstable three term recurrence relations, Appl. Math. and Compo 22 (1987), pp. 277-289. [168] Urabe, M., Nonlinear Autonomous Oscillations, Academic Press, (1976). [169] Van der Cruyssen, P., A reformulation of Olver's algorithm for the numerical solution of second order difference equations, Num. Math., 32(1979), pp. 159-166. [170] Varga, R., Matrix Iterative Analysis, Prentice-Hall, (1962). [171] Wanner, G. and Reitberger, H., On the perturbation formulas of Grobner and Alekseev, Bul. Inst. Pol. Iasi, XIX (1973), pp. 15-25. [172] Weinitschke H., Uber eine kass von iterations oerfahren, Num. Math, 6 (1964), pp, 395404. [173] Weissberger, A. (Editor), Technique of Organic Chemistry, Vol IV. Distillation, Interscience, N.Y., (1951). [174] Willet, D. and Wong, J. S. W., On the discrete analogues of some generalizations of Gronwall's inequality, Monatsh. Math., 69 (1965), pp. 362-367. [175] Wimp, J., Computation with Recurrence Relations, Pitman, (1984). [176] Yamaguti, M. and Ushiki, S., Discretization and chaos, C.R. Acad. Sc. Paris, 290 (1980), pp.637-640. [177] Yamaguti, M. and Hushiki, S., Chaos in numerical analysis of ordinary differential equations, Phisica, 3D (1981), pp.618-626. [178] Yamaguti, M. and Matano, H., Euler's finite difference scheme and chaos, Proc. Japan Acad., 55A (1979), pp. 78-80. [179] Yamamoto, T., Error bounds for Newton's iterated, derivedfrom the Kantarovich theorem, Num. Math., 48 (1986), pp. 91-98. [180] Zahar, R. V. M., Mathematical analysis of Miller's algorithm, Num. Math., 27 (1977), pp.427-447.

Subject Index

A

C

A-stability, 55, 162 Abel formula, 23 Absolute stability, 49, 55, 161, 167 region, 61, 162 Actractivity, 88 Adjoint equation, 31, 77, 223 Affine transformation, 135 Age class, 176 vector, 177 Antidifference, 3, 4, 6 Asymptotic stability, 48, 88,92,93, 108, 123, 128 Autonomous difference equation, 110, 112 Autonomous systems, 92

Canonical base, 30 Capacity of a channel, 189 Carrying capacity, 179 Casorati matrix, 31, 65, 70 Cauchy sequence, 99, 133, 137 Cauchy-Ostrowsky theorem, 178 Central differences, 167 Chaos, 179, 182 Characteristic polynomial, 35, 69, 72,143, 158 Chebyshev polynomials, 39, 47, 57, 142, 173, 213-215 Circular matrix, 225 Clairaut equation, 12 Class K, 105 Clenshaw's algorithm, 139, 142 Cobweb, 185, 193 Companion matrix, 69, 92, 201, 222 equation, 41 Comparison principle, 14, 95, 130 Component matrix, 92, 197, 198 Consistency, 50, 60, 61 Contractivity, 132 Convergence, 50, 60 Converse theorems, 115-119 Courant condition, 168 Cournot, 124 Critical points, 88 Cyclic matrix, 210

B

Backward differences, 167 Balance equation, 183 Banach lemma, 134, 208 Barna, B., 128 Bernoulli method, 59 numbers, 7, 9, 23 polynomials, 7, 9 Biomathematics, 114, 175-182 Boundary value problems, 56, 79, 173 Bounded solutions, 77

239

240

Subject Index

D

Dahlquist, G., 155, 163 Dahlquist polynomial, 55 Dalton's law, 182 Darwinian law, 180 Decrescent functions, 105, 107 Definition set, 2 Demand, 185 Derivative of a matrix, 203 Difference equations definition, 10 normal form, 11 Difference operator, 2 negative powers, 3 Discrete Bihari inequality, 18 Discrete Gronwall inequality, 15 Distillation, 182 Domain of stability, 111-115 E

Economics, 185 Eigenvalues, 60, 94 generalized, 79 semisimple, 92, 94 simple, 92 Equilibrium price, 186 Erlang's loss distribution, 191 Error bounds, 135, 151, 152 Euler function, 22 Euler method, 172 explicit, 55, 167 implicit, 55, 163 Euler-Mel.aurin formula, 24 Expected number, 191 Expenditure, 187 Exponential stability, 88, 90, 103, 123 F

Factorial powers, 4, 5 negative, 5 Fast Fourier Transform, 24 Fibonacci sequence, 38, 58, 189 Finite interval, 161 Finite sum, 4 First approximation, 102 First integral, 14, 135 Fixed points, 88 Floating point, 59 Floquet solutions, 95

Formal series, 43, 68 Frobenius matrix, 69 Functions of matrices, 195, 203 integral form, 203 Fundamental matrix, 65, 66, 91, 96 G

Gautschi, W., 153 Gear, G. W., 160 General solution, 31, 36, 66, 67 Generalized inverse, 82 Generating functions, 42 Geometric distribution, 190 Global attractivity, 88 asymptotic stability, 88, 112, 128 convergence, 128 error, 156 results, 128 Golden section, 58 Green's function, 84 one sided, 71 Green's matrix, 82 H

High order equations, 69 Homogeneous equation, 63, 66, 77 Homer rule, 24 Hyperbolic equation, 173

Infinite interval, 161 Infinite matrix, 84, 222 Integral of a matrix, 203 Interest rate, 58 Invariant set, 111, 131 Investments, 187 Irreducible matrix, 150, 209 Iterative function, 128 matrix, 128 methods, 127-153 non stationary, 129 J

Jordan canonical form, 204 K

Kantorovich, 1. V., 133

241

Subject Index L

La Salle invariance principle, 108, 130 Laurent series, 44 Leslie model, 176 Li and Yorke, 182 Limit set, 110 Limiting probability, 190 Linear difference equation, 27-61, 63-85, 96, constant coefficients, 34 stability, 47, 48 space of solutions, 28, 64 Linear independence, 29,64 Linear multistep methods, 50, 156 Local error, 160 Local results, 128, 129 Logistic equation, 179, 180, 193 Lp stability, 88, 108 Lyapunov function, 87, 104-124, 164 Lyapunov matrix equation, 113 Lypshitz derivative, 151 M

M-matrix, 150 Malthus, T. R., 175, 176 Market, 185 Matrix norm, 91, 207 logarithmic, 98 Matrix polynomial, 195 Method of lines, 166-172 Midpoint method, 61 Miller's algorithm, 139, 141 Minimal polynomial, 195 Minimal solution, 139, 140 Mixed monotone, 145 Models, 175-193 Mole fraction, 182, 183 Mole rate, 183 Monotone iterative methods, 144-153 Monotone sequences, 145, 149 Morse alphabet, 188 Multiple root, 35 Multiplicators, 78 N

National income, 187, 193 Negative definite functions, 105, 107 Newton formula, 23 integration formula, 24 iterative function, 134 method, 24, 58, 128, 130, 135, 152

Newton-Kantarovich, 133, 136, 151 Nonhomogeneous equation, 63, 77 Nonlinear case, 163 Nonnegative matrix, 93, 209 vector, 93 Normal matrices, 169, 173 Null space, 82 Number of messages, 188 Numerical methods, 155-173

o Oligopoly, 124 Olver's algorithm, 139, 142 One-leg methods, 163, 172 Operators A and E, 1, 39 Order of convergence, 59, 135 p

Parking lot problem, 192 Period two cycle, 181 Periodic coefficients, 93 equations, 94 function, 3 solutions, 75, 77, 94 Perron, 0., 75, 104 Perron-Frobenius theorem, 93, 177, 193,210 Perturbation, 102, 119, 138, 161 Perturbed equation, 139 Poincare theorem, 73 Poisson distribution, 192 process, 189, 191 Polynomials p and (T, 50-53, 60, 156 Population dynamics, 175 Population waves, 178, 179 Positive definite functions, 105, 106 Positive definite matrix, 112 Positive matrix, 209 Positive solution, 93 Practical stability, 119, 122, 138, 144, 159 Principal parts, 44

Q Quasi extremal solutions, 145 Quasi lower solutions, 144 Quasi upper solutions, 144 Queuing theory, 188, 189 R

R-Orthogonality, 80 Range, 82

242 Raoult's law, 182 Reducible matrices, 209 Reflecting states, 191 Region of convergence, 132 Relative volatility, 182 Renewal theorem, 193 Resolvent, 69, 203 Riccati equation, 13, 183, 184 Roundoff errors, 155, 159 Runge- Kutta, 173

s Samuelson, 187 Schur criterium, 211 polynomial, 49, 170,212 transform, 212 Secant method, 130 Second differences, 111, 114 Semilocal results, 128, 132 Sequence of matrices, 200 Shift operator, 2 Similarity tranformation, 70 Simpson rule, 61 Smale, S., 128 Spectral radius, 93, 158 Spectrum of a family, 168, 171, 173,203 Spectrum of a matrix, 196, 200 Stability, 88, 106, 123 Step matrix, 81 Stepsize, 160, 162 Sterling numbers, 5 Stiff, 161 Sturm-Liouville problem, 79, 80

Subject Index

Supply, 85 Sweep method, 142, 144

T Taylor formula, 4, 6 Test functions, 51 Toeplitz matrices, 173 Total stability, 119, 159 Traffic in channels, 188 Transpose equation, 32, 142 Trapezoidal method, 55, 172 Truncation error, 50 U

Uniform lp stability, 88 asymptotic stability, 88,91, 107, 157 stability, 88, 91, 107 Unstable problems, 139-144

v Vandermonde matrix, 35, 70, 73 Vapor stream, 182 Variance, 191 Variation of constant method, 32, 38, 67, 99, 11 Volatile component, 182 Von Neumann polynomial, 49, 52, 212

z z- transform, 46 Zubov theorem, 113 O-stability, 52, 158