THEORY OF DIFFERENCE EQUATIONS NUMERICAL METHODS AND APPLICATIONS Second Edition
V. Lakshmikantham Florida Institute of Technology Melbourne, Florida
Donate Trigiante University of Florence Florence, Italy
n MARCEL
MARCEL DEKKER, INC.
D E K K E R
Copyright © 2002 Marcel Dekker, Inc.
NEW YORK • BASEL
The first edition was published by Academic Press as Theory of Difference Equations: Numerical Methods and Applications, 1988. ISBN: 0-8247-0803-2 This book is printed on acid-free paper. Headquarters Marcel Dekker, Inc. 270 Madison Avenue, New York, NY 10016 tel: 212-696-9000; fax: 212-685-4540 Eastern Hemisphere Distribution Marcel Dekker AG Hutgasse 4, Postfach 812, CH-4001 Basel, Switzerland tel: 41-61-261-8482; fax: 41-61-261-8896 World Wide Web http://www.dekker.com The publisher offers discounts on this book when ordered in bulk quantities. For more information, write to Special Sales/Professional Marketing at the headquarters address above. Copyright © 2002 by Marcel Dekker, Inc. All Rights Reserved. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage and retrieval system, without permission in writing from the publisher. Current printing (last digit): 10 9 8 7 6 5 4 3 2 1 PRINTED IN THE UNITED STATES OF AMERICA
Copyright © 2002 Marcel Dekker, Inc.
Preface Difference equations appear as natural descriptions of observed evolution phenomena because most measurements of time evolving variables are discrete, and as such these equations are in their own right important mathematical models. More importantly, difference equations also appear in the study of discretization methods for differential equations. Several results in the theory of difference equations have been obtained as more or less natural discrete analogues of corresponding results of differential equations. This is especially true in the case of the Liapunov theory of stability. Nonetheless, the theory of difference equations is a lot richer than the corresponding theory of differential equations. For example, a simple difference equation resulting from a first-order differential equation may have a phenomena often called the appearance of ghost solutions or the existence of chaotic orbits that can only happen for higher order differential equations. Consequently, the theory of difference equations is interesting in itself, and it is easy to see that it will assume greater importance in the near future. Furthermore, the application of the theory of difference equations is rapidly increasing to various fields such as numerical analysis, control theory, finite mathematics, and computer science. Thus, there is every reason for studying the theory of difference equations as a well-deserved discipline. The present book offers a systematic treatment of the theory of difference equations and its applications with special emphasis on numerical analysis. For example, we devote special attention to iterative processes and numerical methods for differential equations. The investigation of these subjects from the point of view of difference equations allows us to systematize and clarify the ideas involved and, as a result, pave the way for further developments of this fruitful union. Moreover, the deep connections with the Pascal matrix, which is a basic notion in combinatorics, are presented in Chapter 1. With respect to the previous edition of the book ([105]), we have added two new chapters (5 and 9) and revised and added additional sections in the remaining material. The newly added Chapter 5 contains the relations among difference equations and linear algebra. Such relations are usually not included in the existing books, leading to the erroneous conclusions that the two fields have no intersections. Finally, in Chapter 9 we have added some classical difference equations of relevant historical interest, such as the in Copyright © 2002 Marcel Dekker, Inc.
iv
PREFACE
Gaussian arithmetic-geometric mean. The book is divided into nine chapters and four appendices. The first chapter introduces difference calculus, deals with preliminary results on difference equations, develops the theory of difference inequalities and introduce the Pascal matrix along with many of its countless properties chosen in order to give a large overview of its application to many problems. In the second chapter, we present the essential techniques employed in the treatment of linear difference equations with special reference to equations with constant coefficients. Chapter 3 deals with the basic theory of systems of linear difference equations. Chapter 4 is devoted to the Liapunov theory of stability including converse theorems and total and practical stability. In Chapters 5, the relations between difference equations and banded matrices are presented. This gives us the opportunity to present both the theory of linear difference equations from another point of view and to give an overview of classical problems such as orthogonal polynomials, the euclidean algorithm, roots of polynomials, and the problem of well-conditioning. Chapters 6 and 7 deal with some applications of the theory of difference equations relevant in numerical analysis. In Chapter 8, we present applications of difference equations to many fields such as economics, chemistry, population dynamics, and queueing theory. Finally, in Chapter 9 we present some historically important uses of difference equations, i.e. the arithmetic-geometric mean and its generalizations, the Weierstrass iteration, and some applications of difference equations in number theory. The necessary linear algebra used in the book, as well as the relevant notions concerning the Schur criteria and the Chebyshev polynomials, are given in Appendices. Finally, several carefully selected problems at the end of each chapter complement the material of the book. Some of the important features of the book include: (i) development of the theory of difference inequalities and the various comparison results; (ii) unified treatment of stability theory through Liapunov functions and the comparison method; (iii) emphasis on the important role of the theory of difference equations in numerical analysis and some basic notions of combinatorics (the Pascal matrix and its properties); (iv) demonstration of the versatility of difference equations by various models in the real world; (v) timely recognition of the importance of the theory of difference equations and of presenting a unified treatment. The book can be used as a textbook at the graduate level and as a reference book.
Copyright © 2002 Marcel Dekker, Inc.
PREFACE
v
We wish to express our immense thanks to S. Leela, F. Mazzia, F. lavernaro, P. Amodio, and L. Aceto for their helpful comments and suggestions. There are many changes with respect to the first edition. All chapters have been revised. Moreover, Chapter 1 has been enlarged and partially rewritten. Chapter 3 has been enlarged, and Chapters 5 and 9 are new. Many new problems have been added to almost all the chapters. After the publication of the first edition of this book, a rapid increase in the activity on this subject has occurred. In fact, a new completely dedicated Journal (Journal of Difference Equation and Application, Gordon and Breach) has been created. Moreover, many new books, some of a more general type (Agarwal [8]), others exploiting particular aspects of the subject (e.g., Kocic and Ladas [102], Ahlbrandt and Peterson [12], Kelley and Peterson [99], Elaydi [62], Jagermann [96]), have been written. This shows the vitality of the subject and its increasing importance in modern applications.
V. Lakshmikantham and D. Trigiante
Copyright © 2002 Marcel Dekker, Inc.
Contents Preface 1 Discrete Calculus 1.0 Introduction 1.1 Discrete Calculus 1.2 Summation and Negative Powers of A 1.2.1 Equations reducible to simple form 1.3 Factorial Powers and Stirling Numbers 1.4 Bernoulli Numbers and Polynomials 1.5 Matrix Form 1.5.1 Pascal matrix and combinatorics 1.5.2 Pascal matrix and Bernoulli polynomials 1.5.3 Pascal matrix and Bernstein polynomials 1.5.4 Pascal matrix and Stirling numbers 1.6 Comparison Principle 1.7 Problems and Remarks 1.8 Notes 2 Linear Difference Equations 2.0 Introduction 2.1 Preliminarie 2.2 Fundamental Theory 2.2.1 Adjoint and transposed equations 2.3 The Method of Variation of Constants 2.4 Linear Equations with Constant Coefficients 2.5 Use of Operators A and E 2.6 Method of Generating Functions 2.7 Stability of Solutions 2.8 Absolute Stability 2.9 Boundary Value Problems 2.10 Problems and Remarks 2.11 Notes
Copyright © 2002 Marcel Dekker, Inc.
viii 3 Linear Systems of Difference Equations 3.0 Introduction 3.1 Basic Theory 3.2 Method of Variation of Constants 3.3 Autonomous Systems 3.4 Systems Representing High-Order Equations 3.4.1 One-sided Green's functions 3.5 Poincare Theorem 3.6 Periodic Solutions 3.7 Boundary Value Problems 3.8 Problems 3.9 Notes 4 Stability Theory 4.0 Introduction 4.1 Stability Notions 4.2 The Linear Case 4.3 Autonomous Linear Systems 4.4 Linear Equations with Periodic Coefficients 4.5 Use of the Comparison Principle 4.6 Variation of Constants 4.7 Stability by First Approximation 4.8 Liapunov Functions 4.9 Domain of Asymptotic Stability 4.10 Converse Theorems 4.11 Total and Practical Stability 4.12 Problems 4.13 Notes 5 Difference Equations as Banded Matrices 5.0 Introduction 5.1 Initial Value Problems 5.2 Boundary Values Problems 5.2.1 Invertibility of tridiagonal matrices 5.2.2 Sufficient conditions for well-conditioning 5.3 Cyclic Reduction 5.3.1 The case of Toeplitz tridiagonal matrices 5.4 Problems and Remarks 5.5 Notes 6 Applications to Numerical Analysis 6.0 Introduction 6.1 Iterative Methods 6.2 Local Results
Copyright © 2002 Marcel Dekker, Inc.
CONTENTS 6.3
6.4 6.5 6.6 6.7 6.8 6.9
Semilocal Results 6.3.1 Newton-Kantorovich-like theorems 6.3.2 Effect of perturbations Miller's, Giver's, and Clenshaw's Algorithms Boundary Value Problems Monotone Iterative Methods Monotone Approximations Problems Notes
7 Numerical Methods for Differential Equations 7.0 Introduction 7.1 Linear Multistep Methods 7.2 Finite Interval 7.3 Infinite Interval 7.4 Nonlinear Case 7.5 Other Techniques 7.6 The Method of Lines 7.7 Spectrum of a Family of Matrices 7.8 Problems 7.9 Notes 8 Models of Real World Phenomena 8.0 Introduction 8.1 Linear Models for Population Dynamics 8.2 The Logistic Equation 8.3 Distillation of a Binary Liquid 8.4 Models from Economics 8.5 Models of Traffic in Channels 8.6 Problems 8.7 Notes 9 Historically Important Equations 9.0 Introduction 9.1 Combinations of Means 9.1.1 Arithmetic-harmonic mean 9.2 Arithmetic-Geometric (Borchard) 9.2.1 Arithmetic-geometric mean II 9.3 The Weierstrass Method 9.4 Difference Equations and Prime Numbers 9.5 Problems 9.6 Notes Appendices
Copyright © 2002 Marcel Dekker, Inc.
x A Function of Matrices A.I Introduction A.2 Properties of Component Matrices A.3 Particular Matrices A.4 Sequence of Matrices A.5 Jordan Canonical Form A.6 Norms of Matrices and Related Topics A.7 Nonnegative Matrices B The Schur Criteria B.I The Schur'Criteria C The Chebyshev Polynomials C.I Definitions C.2 Properties of Tn(z) and Un(z) D Solutions to the Problems D.I Chapter 1 D.2 Chapter 2 D.3 Chapter 3 D.4 Chapter 4 D.5 Chapter 5 D.6 Chapter 6 D.7 Chapter 7 D.8 Chapter 8 D.9 Chapter 9 Bibliography
Copyright © 2002 Marcel Dekker, Inc.
Chapter 1
Discrete Calculus 1.0
Introduction
This chapter is essentially introductory in nature. Its main aim is to introduce certain well-known basic concepts in difference calculus and to present some important results that are not as well-known. Sections 1.1 to 1.4 contain needed difference calculus and some notions related to it, most of which are found in standard books on difference equations. Section 1.5 deals with a more modern approach consisting of the systematic use of the vector and matrix notation. This not only permits us to rewrite many results obtained in the previous sections in a shorter and more elegant form, but also to get new surprising ones. The central role in such an approach is played by the Pascal matrix. Applications to Combinatorics and to Computer graphics (Bernstein polynomials) are also presented. In Section 1.6 we develop the theory of difference inequalities and prove a variety of comparison theorems that play a prominent role in the development of the book. Several problems are given in Section 1.7 which, together with the material of Sections 1.1 to 1.5, cover the necessary theory of difference calculus.
1.1
Discrete Calculus
Let
where no is an integer number. The generic element in such a set will be denoted by n. We shall consider functions defined on N^Q and assuming values on 1R (or, when explicitly mentioned, on C). They are also called sequences and denoted by f ( n ] or by fn (sequence notation). However, any discrete set of points on which a one-to-one correspondence with N+0 can be established, may be used as definition set. For example, particular
Copyright © 2002 Marcel Dekker, Inc.
2
CHAPTER 1. DISCRETE CALCULUS
circumstances may require the use of the following discrete sets:
where XQ € IR. The generic element in the above sets will usually be denoted by x. They are used when it is desirable to exhibit the explicit dependence of the function on the initial point. The advantage in using J^ h as the definition set is that, for a discrete function which is the approximation of a function defined on El, the dependence of the approximation on the parameter h, usually called stepsize, is explicit. In this chapter, we shall often use J.+ as the definition set whenever we need the dependence on x 6 El (or x e C) in order to consider, for example, derivatives with respect to x. There will be no difficulty in translating the results in terms of other notations. As a rule, we shall only use the sequence notation in the problems at the end of chapters. Definition 1.1.1 Let y : J+0 ~^ IR- Then A is the difference defined by Ay(x) = y ( x + l ) - y ( x )
operator, (1.1)
and E is the shift operator, defined by Ey(x)=y(x+l).
(1.2)
It is easy to verify that the two operators A and E are linear, and that they commute. That is, for any two functions y ( x ) , z(x) and any two scalars a, (3, we have
&(ay(x) + j3z(x)) E(cty(x) + 0z(x}}
= aAy(x) + /3Az(x), = aEy(x) + (3Ez(x),
and &Ey(x] = EAy(x). The second difference on y(x] can be defined as
In general, for every k G A^ + , A fc ,y(x) = A(A f c ~ 1 y(a:))
and
Eky(x] = y(x + k ) ,
with A°y(x) = E°y(x) — I y ( x ) , I being the identity operator such that Iy(x] = y ( x } . In the case when the definition set is N£QI one has Ay n = yn+i — yn and Eyn = yn+\ • It is easy to see that the formal relation between A and E is A = E — I and thus, powers of A can be expressed in terms of powers of E and vice versa. In fact, k
Z-A ?=o
Copyright © 2002 Marcel Dekker, Inc.
. I/
i fc
\\
\ i I V /
1.1. DISCRETE
CALCULUS
and A',
(1.4)
where (^) are the binomial coefficients. Additional properties of the operator A are reported in the Problem section. Concerning the binomial coefficients, we recall that (°) = 1 and (.) = 0 for j ± 0. The above relations (1.3) and (1.4) are usually used to express the value of the generic term of a discrete function by means of its the variations at the previous points. We report as examples a few such relations (many others are presented in the problem section). It is worth mentioning that these kinds of relations have been considered very important in the past, and many of them are associated with the names of famous mathematicians. The reason is that they permit us to simplify many hand computations and to save time when hand calculating. But hand calculations are over, fortunately. Theorem 1.1.1 Let un be defined on NQ . Then
(1.5) Elu0.
(1.6)
i=0
Proof.
Just apply (1.3) and (1.4) to UQ.
Theorem 1.1.2 (Discrete Taylor formula) Let k, n £ be defined on NQ~ . Then
n-k v—\ n — s — 1 E k-l 3=0
k-i
E
• n 1=0
Proof.
n
From (1.5) it follows that
fc-i
E
A
n—k
+ i=0
Copyright © 2002 Marcel Dekker, Inc.
fr-l-1
A
-
n
k < n and un
(1.7)
4
CHAPTER 1. DISCRETE
CALCULUS
By using the identity (see Problem 1.14)
one obtains (1.6). D A generalization of Theorem 1.1.2 is as follows. Theorem 1.1.3 Let j, k, n e N^, j < k — 1, k < n and un be defined on JV0+. Then k-l /
A
- '«O + \
n~k+j
J "
1.2
/
~
8
..
Summation and Negative Powers of A
Definition 1.2.1 Let uj : J^Q —> IR. The function LJ(X] is said to be periodic of period fc if u(x + k) — LJ(X). For example, u(x) — e z2vrx is a periodic function of period 1. The constant functions are particular periodic functions. It is easy to see that ACJ(.X) = 0 for any periodic function of period 1. When the function u(x] needs to be a polynomial, it must be a constant since the only polynomial taking infinite times the same value is the one of degree zero. Consider the equation Ay(x)
= g(x),
(1.9)
where g : J^o —> IR is a known function. The function y ( x ) . denned on the same set of points, is unknown. We shall denote by y(x) = A~lg(x) a particular solution of the above equation. It is not unique because y(x] = y(x] +uj(x), where uj(x) is an arbitrary periodic function of period 1. is also a solution of (1.9). The operator A""1 is called the antidifference operator and it is linear. Moreover, the operators A and A"1 do not commute since, when writing the above considerations in operator form, we have, AA- 1 - / and A'1 A - / + u(x).
Although the last expression is not formally correct, since u(x) is not an operator, nevertheless it is useful because it expresses in a compact form the fact that the operation A~~ T is defined up to an arbitrary periodic one function. If /. g : J+Q —>• IR are two functions such that A/(x) = A#(:r), then it is clear that f ( x ) = CJ(X)+LJ(X). In particular, if f ( x ) and g(x) are polynomials, A/ = A# implies f ( x ) = g(x] + c, where c is a constant.
Copyright © 2002 Marcel Dekker, Inc.
1.2. SUMMATION AND NEGATIVE POWERS OF A
5
We shall now state the relation between the finite sum £]i=o f ( x + i) and the antidifference A~lf(x). Theorem 1.2.1 Let AF(z) = f ( x ) . Then n
V^ f(x + i} — F(x + n + 1) - FCx1) = F(x + z)l i= ^ +1 7
J
\ JL>
n
t' /
-»•
V **^
T
' fr 1 ^ -*- /
V
/
~"~"
V
'^
/17
0
*
(1 10") \
/
i=0
Proof. that
Since by hypothesis we have f ( x ) — AF(x), it is easy to see
i=0
i—O
Note that (1.10) can also be written as
i=0
If we leave the sum to remain indefinite and consider x as the discrete variable, we can express the relation (1.10) in the form
in analogy with the notation for indefinite integrals. In the case when the definition set is N+ , the foregoing formulas reduce to v—^
>
—
yi = A
_i
yi
• i -i i=n+\
^-^
and > y{ = A ^—'
_,
j/ + u;,
respectively. If to the solutions of 1.9 is imposed to satisfy a condition, for example to assume an assigned value y$ at XQ initial condition, then u is equal to yo and the solution becomes x-l
y(x} = yo+ Y^9(s}. S=XQ
Copyright © 2002 Marcel Dekker, Inc.
(1.12)
CHAPTER 1. DISCRETE CALCULUS
1.2.1
Equations reducible to simple form
It may happen that more difficult linear difference equations or even nonlinear ones may be reduced, by more or less judicious transformations, to the simple linear form considered above. Consider, for example, the equation z(x + 1) -p(x)z(x)
= q(x),
Z(XQ) = ZQ.
(1-13)
By setting P(x) = l f i ~ * Q p ( t ) , P ( x o ) = 1 and dividing (1.13) by P(x + l ) , we have z(x+l) z(x) q(x) If we write y(x] = pr\ and g(x] = p ( + i \ > equation (1.13) now takes the form Ay(x) = g ( x ) . The solution of (1.13) is then given by
__
=
I
s=x0 r^b "^ l)
+z
(1-14)
s ^(s) n ?(*) ° n ?(*)•
X—1
X— 1
.S = XQ
t= 5+ l
X—1
t — XQ
We present a few examples below taken from the applications. Example 1 The following equation is often encountered in the study of propagation of errors of iterative processes. The reduction to the easy linear form is almost trivial (see Problem 1.28): yn+i = Q>ynThe solution is 1
"
a yn — -( yo) 2 • a
Example 2 (1.15) This equation is solved by yn — T2«(z), where Tj(z) are the Chebyshev polynomials of the first kind described in Appendix C. and z is a complex value to be determined later. In order to check the assertion, we need the so-called semigroup property of Chebyshev polynomials, i.e. Tjm(z] — T3(Tm(z)}. In fact, considering that T^z] — 2z2 — 1, one has = r 2 « 2 (z) = T2(T2n(z}) = T2(yn) = 2y2n - 1.
Copyright © 2002 Marcel Dekker, Inc.
1.2. SUMMATION AND NEGATIVE POWERS OF A
7
Furthermore, by considering that yo = T\(z], we have z — yo, and then yn = T2"(yo). It is well known that when \z\ < 1, |Tj(z)| < 1, for all j. The oscillatory nature of yn is then easily recognizable from the above expression. More than that, small variations of the initial condition yo may drastically change the solution (see Problem 1.5). We shall see in the following example that Equation (1.15) is related to the so-called chaotic behavior. Example 3 2/n+i = cyn(l - yn). This equation is the easiest equation whose solutions may have chaotic behavior (see Chapter 8). For a generic value of c, it is not possible to write the solution in closed form. It is however possible in the cases c = 2 and c = 4. In fact, the substitution yn = ^f^ transforms the above equation to zn+i = l + -(4- 1). The case c = 2 leads to Example 1. The case c = 4 has been considered in Example 2. It seems worthwhile to focus on the different behavior of the two solutions. In the case c = 2 the solution is yn — ^ (l — (1 — 2yo) 2 ")- It is then evident that for 0 < yo < 1/2, one has limy n = 1/2. In the second case the behavior is oscillating. As matter of fact, the value c = 2 is outside the chaos window, while the value c = 4 is just inside. The following equation arises often in the theory of algorithms, especially when dealing with the class of divide and conquer algorithms. Example 4 yn = ky^ + f ( r i ) . Depending on the particular applications, /(n) may assume different forms. Particular important cases are
2. k = 2, f ( n ) = n Iog2 n; 3. fc = 7, f ( n ) = n2.
In the first case yn represents the maximum possible cost of a binary search. The second case arises in many different applications, for example in the odd-even merge sort. The third case arises in evaluating the complexity of certain algorithm of matrix multiplications. In all applications, the initial
Copyright © 2002 Marcel Dekker, Inc.
8
CHAPTER 1. DISCRETE CALCULUS
value y\ is known. The solution is obtained by setting n — 2 m , zm — y^™ and gm = f(1m}. The resulting equation is z
m ^ KZm — i + 9m,
whose solution is m —1
j=0
i.e. / (log2 yn = k og2 n \ y\ + y^
(
h
/
The above mentioned cases correspond respectively to the solutions: 1- yn = y\ +log 2 n; / 2.
llr, —
The following example is taken from Amer. Math. Monthly (1999) (Problem N. 10578)
Example 5 (n + l)(n - 2)y n+1 - n(n2 -n-
l}yn + (n - l)3yn-i = 0 ,
n> 2
with j/2 — Z/3 = 1- The change of variable xn — nyn transforms the equation to •En+l ~~ '-En / -\\'^-"n ~~ xn — 1 ,-, (n - 1 ~— = 0, n —1 n —2 which, in turn, by setting zn — X "^'_^ XT ' , gives zn- (n- l)2 n -i = 0, whose solution is zn = (n — 1)!. The solution of the original equation is then ( n - 1)1 + 1 Un -
n
•
It is interesting to note that a theorem of number theory (the Wilson theorem) may be used to state that yn is an integer if and only if n is prime. In Table 1.2 we list differences arid antidifferences of the most common functions, omitting the periodic function uj(x}.
Copyright © 2002 Marcel Dekker, Inc.
1.3. FACTORIAL POWERS AND STIRLING NUMBERS
1.3
9
Factorial Powers and Stirling Numbers
The factorial powers defined below have, in the discrete calculus, the same role that the functions xn have in differential and integral calculus. Definition 1.3.1 Let x E IR. The n-th factorial power of x is defined by
It is easy to verify from the above definition that ^^(n) _
nx(n-\]
and
i
AA —1,^(71—1) *< > = - *_(7"i) ( > +i „ .
(1-16)
/ i 1 T\ (i. 17)
According to the observation made in the last section, the periodic function is actually a constant, since both sides of (1.16) are polynomials. We also have x(m+n"> = x^m\x - m)( n) . When m = 0, this yields x (0+n) = z (0 M n) , which shows that x^ — 1. Moreover, for m — —n, we get 1 = x^°^ — x^~ n ^(x + n}(n\ which allows us to define the negative factorial power x^~ n ^ by
from which we also derive that 0^~ n ^ = ^. Moreover, from the definition, we have (-x) (n) = (-l) n (z + n - l)^. (1.18) Relations (1.16) and (1.19) suggest that it will be convenient to express other functions in terms of factorial powers whenever possible. For example, in the case of polynomials we have the following result. Theorem 1.3.1 Let n G NQ~ . The powers xn and the factorial powers are related by n
xn = '£tS?x®
(1.19)
i=0
where 5" are the Stirling numbers (of the second kind) that satisfy the relation
with S% = S? = 1, 5£ = 0, for n ^ 0.
Copyright © 2002 Marcel Dekker, Inc.
CHAPTER 1. DISCRETE CALCULUS
10
Table 1.1: Stirling Numbers of the Second Kind
n\i 1 2 3 4 5 6
1 2 1 1 1 1 3 1 7 1 15 1 31
3
4
5
6
1 6 25 90
1 10 65
1 15
1
Proof. Clearly (1.19) holds for n — I . Suppose it is true for some n, multiplying both sides of (1.19) by x, we get n
n+l r=l n \
N
i=2 n+1
i=l
showing that (1.19) holds for n + 1. induction. D
Hence the proof is completed by
Stirling numbers Sf for i.n = 1 , 2 , . . . ,6 are given in Table 1.1. Using the relation (1.19), one can immediately derive the differences and the antidifferences of a polynomial. Theorem 1.3.2 The first difference of a polynomial of degree k is a polynomial of degree k — I and in general the s-th difference is a polynomial of degree k — s. Proof. It is not restrictive to consider xk instead of a polynomial of degree k. From (1.19) we have
Copyright © 2002 Marcel Dekker, Inc.
1.4. BERNOULLI NUMBERS AND POLYNOMIALS which is of degree k — s.
11
D
By using Equation (1.16), it is easy to check that for x e ij
j- v 1
j
:_ -«••
The latter result is often called Stirling identity, often used in constructing the binomial coefficients table (Pascal table), i.e.
x\ (x — 1\ (x - l\ ( ;)-(j-G-i)-
(L2o)
Example 6 From (1.11) and (1.20) one readily obtains
b
£ 1.4
Bernoulli Numbers and Polynomials
From (1.19) one has that for every n e NQ~, x £ IR
where o;n is constant with respect to x. Let ujn = ^rjCn+i and let us set JL ^(i+i) Bn+1(x) = (n + l)A~1xn = (n + 1) ]T 5J1— -r- + Cn+1 i=l
(1.22)
l+
with BQ(X} = 1. The polynomials Bn(x) satisfy the relation ABn(x) = nxn~l.
(1.23)
They arc not uniquely defined because the constants Cn are arbitrary. Usually it is convenient to avoid the Stirling numbers in the determination of Bn(x). This can be accomplished as follows. Theorem 1.4.1 Let n G -/V^ , BQ(X) = 1 and Bn(x) be polynomials satisfying (1.23). Then the two functions
i=0
and Gn(x) = nxn-1 differ by a constant.
Copyright © 2002 Marcel Dekker, Inc.
(1.25)
12
CHAPTER 1. DISCRETE Proof.
CALCULUS
From (1.23) one has n-l /
n x
( ) ~~ 2^ {i i=Q \
and
AG n (:c) = n[(x + I)71"1 - xn~l Hence, it follows that AF n (x) — AG n (x) and since F n (x), Gn(x) are polynomials, we have F71 (x} (1.26) \ / — G 71(x} V / +' d 71 ? \ * / where dn are constants (with respect to x). D When the constants dn have been fixed, (1-24) allows us to construct the polynomials Bn(x). The constants dn are fixed by imposing one more condition to be satisfied by Bn(x). The most commonly used condition is dBn(x) ~rf7~~ n 5 n - l ( < r ) '
(
27)
or
I11 / B n (x)dx = 0, f o r n = l , 2 , . . . . Jo One, in fact, has the following result.
(1.28)
Theorem 1.4.2 If for every n e N+, the polynomials Bn(x] satisfy (1.23) with BQ(X) — 1 and either (1.27) or (1.28) is satisfied, then (1-29) Proof. Let us start with (1-27). Differentiating (1.26) and using (1.27), we have nFn-i(x} = F^(x) — G'n(x] = nGn-i(x}. This implies that 0 = n(Fn_i(x) — Gn-\(x)} = ndn^i from which it follows dn-\ = 0. Let us now suppose that (1.28) holds. From (1.24) we obtain J() Fn(x)dx = /Q1 B()(x}dx = 1 and f^ Gn(x)dx = 1. Because of (1.26), we now get 1 — 1 + d n , which implies dn = 0. n As we have already observed. (1.29) define the polynomials Bn(x) uniquely. They are called Bernoulli polynomials. The first five of such polynomials are as follows:
Copyright © 2002 Marcel Dekker, Inc.
B0(x)
= 1,
Bi(x]
- z-i,
B2(x)
— x 2 — x H—1 , 6
1.5. MATRIX FORM
13
_ -
3
3 1 - -x2 + -x,
~4 rr
o 3+ , x2 -2ar
3Q .
The values of Bn(0) are called Bernoulli numbers and are denoted by Bn. As an easy consequence of (1.29), we see that the Bernoulli numbers satisfy the relation n-l / \
(1.30) which can be considered as the expansion of (1 -f B)n — Bn, where the powers Bl are replaced by Bi. This property is often used to define Bernoulli numbers. It can be shown that the Bernoulli numbers of odd index, except for J9i, are zero (see Problem 1.19). The values of the first ten numbers are 5
o = l,51 = --,52 = - 1 B 4 = -35,B6 = -,B8 = - - , B 1 o = g g .
From (1.23), applying A"1 to both sides, we get
n
A simple application of (1.31) is the following: suppose that x takes integer values. Then, from (1.11) and (1.29), we see that x
Bn(x]
= -[Bn(m
x=0n
from which we get the sum of the (n - l)-th powers of integer numbers. When n — 3, for example, we have m
E x=0 1.5
o
-•
J-
o
Matrix Form
Discrete calculus has many applications and constitutes the background of many old and new disciplines such as numerical analysis, combinatorics, umbral calculus, theory of algorithms, wavelets, etc. The use of vectors and matrices permits us not only to state results in a shorter and more compact form, but also to generalize such results. In this section we shall develop such an elegant approach.
Copyright © 2002 Marcel Dekker, Inc.
14
CHAPTER 1. DISCRETE
CALCULUS
Table 1.2: Differences and Antidifferences /(x) c
A / ( x ) A ~ 0
a
:
)
ex
(r 1 \rx \C — 1 )C
C:r / i ~r^l • > £ / * •
xcxT
/ -1 \ T -4- 1 (c - l)x(r 4- cTx+i
C^' C \ ^rrj (I x - ^TI ) , c // -1I
/XN
/
Vri-l^
cos(ax + b]
—2 sin f sin (ax + bx + f )
/
/ (
rx C
ln^
•
1
, i \
sm(ax + o)
o - n
X
I
X
, i , o \
2sm ^ cos (ax + b + |)
cosfax+fe-^'
—
log(x + c)
1.5.1
Pascal matrix and combinatorics
In most applications the central role is played by the so-called Pascal matrix and its countless properties. Even if such a matrix is the oldest known matrix, the systematic study of its properties is very recent. Most results already obtained and used in the previous sections may be deduced in a more elegant and easy way by using the matrix notation and, of course, the Pascal matrix. We shall give here a short presentation of the main properties of such a matrix along with a few examples of its applications. Let n > 0. The entries of the Pascal matrix of dimension n are defined as follows:
(1.32)
Copyright © 2002 Marcel Dekker, Inc.
15
1.5. MATRIX FORM
It is a lower triangular matrix whose entries on each row are the binomial coefficients. It is strictly related to the creation matrix H defined by
\
( 0 1 H =
(1.33)
n-l
0)
To see the relation between the two matrices, let us state a few properties of the matrix H. Let e^, i = 0, 1, . . . n — I be the unit vectors in IRn. Whenever the index of such vectors will result greater than n — 1, the corresponding vector will be assumed to be the null vector. From the definition of H one has Het = i + l e i i and Theorem 1.5.1 The Pascal matrix is given by P — eH . Proof.
Since Hn — 0 (i.e. it is equal to the zero matrix), we have n-l
i=0 Such a matrix has the same entries of P. In fact,
^•_^C7+^ r s=0
5=0
s=0
where Sij is the Kronecker symbol1 which is one if both indices are equal and zero otherwise. The above expression is then equal to zero if i < j. Otherwise one has
3 and this completes the proof.
D
T
The matrix PP is called the symmetric Pascal matrix. Its entries are
(PPT),, = 1
(1.34)
The Kronecker symbol is a sort of alien in this setting, where there exists the corresponding symbol T° ).
Copyright © 2002 Marcel Dekker, Inc.
16
CHAPTER 1. DISCRETE
CALCULUS
This relation contains the famous Vandermonde convolution formula, i.e. (see Riordan [157] and Problem 1.32))
The power Px, for all x G IR, is easily defined as Px — exp(xH). With arguments similar to those used above, one easily checks that the nonzero entries of Px are (1-35) It follows that all the powers of P, included the negative ones, are easily provided. Combining such powers appropriately, many famous combinatorial identities can be established. Here we report some examples.
2z
where Q are the Catalan numbers. The first identity is obtained from P-P = P 2 , by using (1.35). The second is similarly obtained by multiplying P and its inverse obtained again by (1.35) by taking x = — 1. Other identities are not so easy, but their proof in matrix form is always easier, or at least more elegant. To give an example of more involved results, we give here a proof of the identity used in the previous section. To do this, we need to introduce the shifted Pascal matrices. Let s be an integer. The shifted Pascal matrix Ps is defined by
(1.36)
The entries of the inverses of such matrices have also a simple expression, i.e.
(see Problem 1.33).
Copyright © 2002 Marcel Dekker, Inc.
1.5. MATRIX FORM
17
By considering the shift matrix / 0 1
K= \
, 1
(1.37)
0 // n x n
one may easily check the following result.
Theorem 1.5.2 For all values of s one has (i) Ps=e(H+sK); (n) Ps_! =(I-K)P3; (in) PS = PS^(I + K)Proof. The proof of the first relation is similar to the one provided in Theorem 1.5.1. The remaining two are easy consequences of the Stirling identity (1.20) on the combinatorial coefficients. We leave as an exercise the remainder of the proof. D From them, the following relations (valid for all integers s) easily follow:
P = (I-K)SPS, Ps = (I - K)P8(I + K), PSK-KPS KPS
= KPSK, = P^K,
Let e = (1,1,..., 1)T. From (ii) one has Ps-\P~le = GQ. Suppose now we want to prove the first identity used in Section 1.1, i.e. q—n
j=0
n-l
The left hand side can be written as
t
\ q —n + n
where m — q — n. In matrix form, it is equivalent to
Copyright © 2002 Marcel Dekker, Inc.
m +n
18
CHAPTER 1. DISCRETE CALCULUS
Many other properties of P can be established by considering the following differential equation in IRn:
~ = Hy, ax
(1-38)
whose solution is y ( x ) = Pxy(0). By varying the initial condition, one obtains many known functions all satisfying the above equation (see [4]). For example, the vector £(x) = ( x ^ x 1 , . . . ,xn~l)T is one of such functions. It corresponds to the choice y(0) = CQ. One has P£(x) = exp((x -f l)H)eo — £(x + 1) and, in general,
This very simple relation may be used to establish many combinatorial identities. For example, by taking x — 1 and j — 1 we have Pe = £(2), which is a short form of writing fc */ ;IS* \\
k
\=2k
Analogously, from £(x — j ] — P
J
^(x) one obtains, for example, £(0) =
A less trivial example is the following (see Am. Math. Monthly (1997) Problem 10632). Example 7 Evaluate the expression
The expression can be rewritten as
I" C(x - l)xmdx
Jo
Copyright © 2002 Marcel Dekker, Inc.
1.5. MATRIX FORM
19
It is an easy matter to verify that F ' ( y ) = 0, i.e. that F ( y ) is independent of y. The original expression is then equal to m'r?'
/"I F(0) - / xn(l - x)mdx =
(JTl ~~r 71 ~r 1J-
JO
If instead of the real numbers x in the definition of £, we use either the operator E or A, the relations (1.5) and (1.6) among the operators A and E are essentially obtained. In fact, one has £(E) = Pf (A).
(1.39)
When applied to a first term of a sequence U Q , W I , . . . , the s-th row of the above relation provides the values of u s _i in terms of the differences A ? WO, i — 0 , . . . , s — 1. It is worth noting that ax
px = HPX,
(1.40)
since Px is a fundamental matrix for (1.38).
1.5.2
Pascal matrix and Bernoulli polynomials
Let us consider now the Bernoulli polynomials Bi(x) and the vector
By (1.27), this vector also satisfies (1.38), and then b(x) = Pxb(Q). It follows that the values of the polynomials are easily obtained once the entries (Bernoulli numbers) of 6(0) are known. Moreover, for all integers j, we have b(x + j ] = Pjb(x). The property (1.28) is imposed by considering the matrix n
~l
rl Li —
/
±
(IX
—
/
f]i -
7TT .
&S (* + !)!
Jo
The mentioned property becomes now £6(0) - e 0 ,
and then 6(0) = L~I€Q, i.e. the Bernoulli numbers are the entries of the first column of the matrix L~1 . Moreover, since both P and L are polynomials of the same matrix H, they commute. This permits the assertion that 6(x) = F X 6(0) = PxL-leQ = L~lPxeQ =
Copyright © 2002 Marcel Dekker, Inc.
20
CH AFTER 1. DISCRETE
CALCULUS
which shows that the matrix L is the transformation matrix between the Taylor expansion and the Bernoulli expansion. In other words, if a function f(x] has a Taylor expansion f ( x ) = fT£,(%) + high order terms, where /7 is the vector containing the coefficients of the expansion, the expansion in terms of Bernoulli polynomials will be: /(:/;) — f7 Lb(x) + high order terms. From the definition of L it easily follows that LH = HL = P-I.
(1.41)
This relation permits us to obtain many known relations. We give a few examples below. Example 8 By taking the difference between two successive vectors 6(.x), we have b(x + l)-b(x) = (P-I)b(x) = (P-I}PxL-le0 = (PThis is nothing but the property (1.23) in vector form.
Example 9 By multiplying on the right by b(x). we get (P — I}b(x] HLb(x) = H£(x) which is equivalent to (1.29).
Example 10 By multiplying on the right by 6(0). we get (P - /)6(0) — HLb(0) = H£(0) = He.Q which is equivalent to (1.30). 1.5.3
Pascal matrix and Bernstein polynomials
To quote one more application of the Pascal matrix, we discuss the Bernstein polynomials which are the foundations of modern computer graphics (see Farin [64]). Such polynomials B^(x) are defined as if
0
otherwise,
i,j — 0 , . . . . n — 1.
We define the Bernstein matrix Be(x] by setting (Be(x}}ij — B-j(x). By using the Pascal matrix, the matrix Be assumes the very simple form
Copyright © 2002 Marcel Dekker, Inc.
1.5. MATRIX FORM
21
~l 2
n
(1.42)
1
where Dx = diag(l,x,x :... ,x ~ }. The easy check is left as exercise (see also [4]). Prom (1.42), it is very simple to derive the properties of the Bernstein polynomial. For example it can be seen as a similarity transformation to the diagonal form of Be. The eigenvalues of Be are then the diagonal entries of Dx. The first eigenvalue is 1, to which corresponds the eigenvector e — i.e the first column of P which is e = ( 1 , 1 , . . . , 1)T. It then follows that (1.43)
Bfe = e.
In vector form the above expression states a well known property, namely that the Bernstein polynomials form a partition of unity. Almost trivial is the proof of the so called subdivision property stating that Be(ct) = Be(c}Be(t). By the way, we note the interesting and useful relations among the newly defined matrix Dx and the matrices H and P: Px = DXPD-X,
_X = xH.
Moreover, since
/1 rl
1
P -i
>-i1 = P Be(x) dx = P / Dxdx P" o Jo by considering that
fl P / D-r-dx = / DxdxPi, Jo 10 where PI is the shifted Pascal matrix (see (1.36)), and by using a result in Theorem 1.5.2, we obtain
Be(x}dx
= '0
/1
\
/I
n /
, 1 \
1
0
... 0 \
2
i \
(
l
1
° 1
2
2
..
'•• 1
Copyright © 2002 Marcel Dekker, Inc.
0 \
0
I
'•. o
1 i/
CHAPTER 1. DISCRETE
22
CALCULUS
This is a compact form of writing a very important property of Bernstein polynomials, i.e. (1.44) Finally, we note the lesser known result \fx £ IR\{0}
(1.45)
and, more in general terms, for all integers j (1.46)
1.5.4
Pascal matrix and Stirling numbers
The Stirling numbers may be also patterned in a matrix form, i.e.
/ s° 0
S=
0
0
0
\
1
S ^l
0
V o
-1 /
The nonvanishing entries of S are the Stirling numbers of the second kind defined in Section 1.3. Considering the above defined vector £(x) and the analogous vector, defined by means of factorial powers,
the Stirling transformation considered in Section 1.3 is written in matrix form as £(z) = Sri(x}. The reverse relation is, of course, defined by means of the inverse matrix S~l whose entries are called Stirling numbers of the first kind. By considering the Vanclermoride matrix, defined by its columns
and the analogous matrix Vx = (r/(x), 7](x + 1), . . . , r,(x + n ~ 1)) . the above expression leads to Wr = SVT.
Copyright © 2002 Marcel Dekker, Inc.
1.5.
MATRIX FORM
23
The matrix Vb is upper triangular, as can be easily checked from the definition of the factorial powers. We have then found an LU type factorization of the Vandermonde matrix WQ. The result may by refined by introducing the diagonal factorial matrix Df = diag(l, 1!, 2!, . . . , n!). In fact one has Vb = DfPT. Moreover, since Wx satisfies the differential equation ^-Wx = HWX, ax
(1.47)
as can be immediately checked by considering that each column of Wx satisfies (1.38), one then has Wx = PXW0 = PxSDfPT, which is a factorization of type LDU. Such factorization is used in numerical analysis to solve efficiently the Vandermonde systems of large dimension (see Golub[78]). Moreover, we get also Vx = S-PXSV0,
(1.48)
which is obtained by considering that ax
X
ax
X
and that es~~lHSx = S~1PXS. A deeper relation between the matrices 5 and P can be stated as follows. The relation (1.16) in vector form becomes ATJ(X) = HTJ(X). Since the columns of Vx are just made by the successive vectors ^(x), we also have AFX - HVX,
from which we obtain V\ = VQ + HVo and then ViV^"1 = I + H. By considering (1.48) for x = 1, we obtain I +H
(1.49)
or P — I = SHS~l. Moreover, since DJ1HD/ — K, we also have (SDf)-1P(SDf)
= I + K.
(1.50)
The above discussion proves the following theorem. Theorem 1.5.3 The matrix SDf transforms the Pascal matrix P to the Jordan bidiagonal form.
Copyright © 2002 Marcel Dekker, Inc.
24
1.6
CHAPTER 1. DISCRETE CALCULUS
Comparison Principle
One of the most efficient methods of obtaining information on the behavior of solutions of difference equations, even when they cannot be solved explicitly, is the comparison principle. In general, the comparison principle is concerned with estimating a function satisfying a difference inequality by the solution of the corresponding difference equation. In this section, we shall present various forms of this principle. Theorem 1.6.1 Let n e N+0,r > 0 andg(n,r) be a nondecreasing function with respect to r for any fixed n. Suppose that for n > HQ, the inequalities
yn+\
< g(n,yn),
(1.51)
un+i
> g(n,un)
(1.52)
hold. Then
yno < uno
(1.53)
implies that yn
< un,
n > n0.
(1-54)
Proof. Suppose that (1.54) is not true. Then, because of (1.53) there exists a k e N+0 such that yk+i > Wfc+i- It follows, using (1.51), (1.52), and the monotone character of g, that
g(k.Uk) < uk+i < yk+i < g ( k , y k ) < g(k,uk) which is a contradiction. Hence the proof. D Usually in applications, (1.52) is an equation and the corresponding result is called the comparison principle. Corollary 1.6.1 Let n G N£Q, kn > 0 and yn+\ < knyn + pn. Then, for n > HQ, we have n —l
yn < yno n
s~n0
n— l
n—l
ks +
S
Ps
s=no
II
kr
r=s+l
-
( li55 )
Proof. Because kn > 0 the hypotheses of Theorem 1.6.1 are verified. Hence yn < unj where un is the solution of the linear difference equation un+i = knun+pn,
uno = yno.
(1.56)
By (1.12), we see that the right-hand side of (1.55) is the solution of (1.56). D
Copyright © 2002 Marcel Dekker, Inc.
1.6. COMPARISON PRINCIPLE
25
Theorem 1.6.2 Let g(n, s, y] be defined on N+0 x N£Q x IR and nondecreasing with respect to y. Suppose that for n G n-l
, yno < pno implies yn < un,n > no, where un is the solution of the equation n-l
s=no
Proof. If the claim is not true, then there exists a k e N+0 such that > Wfc+i and ys < us for s < k. But, + 1, s, y5) - g(k + 1, s, us)] < 0 s=no
which is a contradiction.
D
Corollary 1.6.2 (Discrete Gronwall inequality). Let n £ N+Q,kn > 0 and n
yn+i
Then, for n > no, n— 1
yn
<
n— 1
I/no If t s=no
1+
n— 1
<
^) + Z P» H ( X + ^) s=no r=s+l n—1
n— 1
y n o exp(^ fcs) J] p s exp( Y^ 5— no
Proof.
n— 1
s=no
k
r}-
r=s+l
The comparison equation is n-l
Un = Un0 + 5Z f^ sW5 +Pa], 5=no
W no = y no .
This is equivalent to Aiin = fcnwn + pn, the solution of which is n— 1
n— 1
n— 1
Wn = Uno T[ ( + ks) + ^ Ps II ( X + k^~ s—no s=no r=s+l l
The proof is completed by observing that 1 + ks < exp(/cs) The proof of the following corollary in very similar.
Copyright © 2002 Marcel Dekker, Inc.
D.
26
CHAPTER 1. DISCRETE CALCULUS
Corollary 1.6.3 Let n e N+Q,kn > 0 and n yn+l < Pn+i + Yl k^ys: s=n0
P
Vno <
n0-
Then, n— 1
3/n
n— 1 l +
n— 1
M + E
<
PnQ U( s=no
<
Pno exp I ^ k3 \ + J^ Q* exP
\
/n-l
\s=no
n-l
/
/ n-1
.s=no
\
E kr I ' YT=S+!
/
where qn = AP n . Another form of the foregoing result is as follows. Corollary 1.6.4 Let n e N+Q,kn > 0 and 71
J/n+1 < ^n+1 + E ^ s ^ s ' •s=no
^"o < ^no-
Then, n— 1
2/n
<
n— 1
Pn + E P fl fc a s=n 0
J] (l
+k
r}
n-l
Pn+
Proof.
Let n-l
Then we have
yn
(1.58)
Applying the operator A to both sides of (1.57), we get t-\ vn — i\nyn ^ ^nrn > ^nvn AT/
If
II
<
I?
P
-\r-
k
\/
from which by Corollary 1.6.1 it follows that n— I
n —l
s—rio
r=s+l
111 view of (1.58), we obtain the desired estimate.
Copyright © 2002 Marcel Dekker, Inc.
D
1.6. COMPARISON PRINCIPLE
27
Corollary 1.6.5 Let k(n, s,x) : N+Q x N+Q x IR+ —> IR+ be monotonic nondecreasing in x and g(n, u] : N+Q x IR+ —» IR+ be monotonic nondecreasing in u. Suppose that
( n~l \ yn < 9 I n, J^ fc(n,s,ys) J . \
s=no
(1.59)
/
TTien
where un is the solution of n-l w
n = J] k(n,s,g(s,us)). s=no
Proof. r
n = Es-no
The relation (1.59) can be written as yn < ^(n,r n ), where fc
( n ' 6 1 '^)-
But this
yields r™ -
^=nok(nisi9(s,rs)}- Now
applying Theorem 1.6.2, we obtain rn < un, which in turn shows
Theorem 1.6.3 Suppose that gi(n,u) and gi(n,u) are two functions defined on 7V+o x IR+ and nondecreasing with respect to u. Let 92(n,un) < un+i < gi(n,un). Then, Pn < un < r n ,
where Pn and rn are the solutions of the difference - 9i(n,rn).
Proof.
equations
rno > uno,
Applying Theorem 1.6.1 twice, we obtain the needed estimate.
D
Theorem 1.6.4 (Discrete Bihari inequality). Suppose that hn is a nonnegative function defined on -/V+0, M > 0. and W is a positive strictly increasing function defined on IR + . If for n > no,
yn< Vn where
n-l
Vn=y0 + M
Copyright © 2002 Marcel Dekker, Inc.
28
CHAPTER!.
DISCRETE CALCULUS
then for n l ( ~ 1 n G N, = in e 7V+|Af ^ ha < G(oo) - G(x 0 ) ,
I
s=no
J
n-l
yn < G-1 ( G(y 0 ) + M X! s=n 0
where G is the solution of
Proof.
We have AFn = MhnW(yn) < MhnW(Vn}. It follows that G(Vn+l)
from which, in view of Theorem 1.6.1, we get n-l
G(Vn) < G(y0) + M ^ ha. s=no
Hence, for n G JVi, Fn < G-^GCyo) + M E?=n0 ^)-
°
Theorem 1.6.5 Le£ '(/j, j = 0, 1, . . . , be a positive sequence satisfying
(
n-l
n-2
\
2/n'S^'Z]^ ' j=0 j=0 /
where g ( y , z, w) is a nondecreasing function with respect to its arguments. If 7/0 < U
then, for all n > 0 'i/n < '"n-
Proof. The proof is by induction. The claim is true for n = 0. Suppose it is true for n — k. Then, we have n-l -^
n-2 _—^ _—^
\\
2/n, 2^ yj- Z^ y.l j=0 /
Copyright © 2002 Marcel Dekker, Inc.
II
n-2 -^
n-l _—^
_—^
<9Un,2^ <9\Un,2^ V j=0
-^
W W
U J' Z^ 3 j=0
I
=U
n+l-
D
1.7. PROBLEMS AND REMARKS
29
Theorem 1.6.6 Let yi i = 0, 1, . . . be a positive sequence satisfying inequality
the
yn+i < g(yrnyn-\, • • • ,yn-k}, where g is nondecreasing with respect to its argument. Then,
where un is the solution of Un+l = g^n-.Un-lT-iUn-k},
Uj>y^
Proof. Suppose that the conclusion is not true. Then there exists an index m > k such that ym+i > Um+i and yj < Uj,j < m. It follows that 5 ( 2 / m , 2 / m - l , - • • ,2/m-fc) > 2/m+l > W m + l = 0 ( u m , U m _ i , . . . , U m _ f c ) ,
which is a contradiction.
1.7
D
Problems and Remarks
1.1 Let yi — —2,2/2 = — 2, j/3 = 0,1/4 — 4. Compute j/s supposing that A 4
(~\
1.2 Show that
i(ax+b)
1.3 Usin the result of Problem 1.2. show that
A cos (ax — b)
=
A sm(ax + 6) =
—2 sin - sin ( ax + 6 H— 1 , z \ z/ a ( a\ 2 sin - cos I ax + b H— I . Z
\
2/
1.4 Show that
+i
1.5 Consider the result obtained in Example 2. Show that a small perturbation in the initial condition may change considerably the solution. (Hint: use the result established in Appendix C: T-'2n (z) = 1nT'2,l_l(z}}.
Copyright © 2002 Marcel Dekker, Inc.
30
CHAPTER 1. DISCRETE CALCULUS
1.6 Show that £"=i j3 = \n2(n + I) 2 . 1.7 Prove that ^ sin(n + \}q - sin^g > cosg?, = *•— —. ~{ 2sinig 1.8 The factorial powers can be defined for values of x that are not integers. Lotting m = 1 and n = x - 1, in the relation X ' (m+m) = x^m\x - m) (n) , we get x^ — x(x— l)^" 1 ). Setting y ( x ) — x^x\ the previous expression can be written as y(x] = xy(x-l). The solution of this equation is y(x) — F(x + l), where F(x) is the Euler function, which is denned for all real values of x except the negative integers. This allows us to define x^ for x — 0. In fact, we have 0 (0) = T(l) = 1. 1.9 The function F(x) is also defined as roo
F(x) = / Jo
e-^-^dt.
Show that for positive integer, F(x) — (x — 1)!. 1.10 Using the previous result, show that F(x + l) F(x + 1 - n)
1.11 Show that AlogF(x) = logx and A"1 logx = logF(x) +u;(x). 1.12 Let
where F'(x) is the derivate of F(x). Show that A'0(x) = -. x The function //'(x) has many interesting properties. Among them we recall lim x _ o c (0(x) - log(x)) = 0. 1.13 Show that, if p(x) is a polynomial of degree k. it can be written in the form
which is the Newton form.
Copyright © 2002 Marcel Dekker, Inc.
1.7. PROBLEMS AND REMARKS
31
1.14 Show that for q,ri G N£ , <]-n
/
q
\
/
~
i\
(Hint: use the Stirling identity.) 1.15 Show that for g, /, n € ./V^, one has -\
/
~
/
i
Note that S(q, 0, n) = (^1*) as established in the previous exercise. 1.16 Verify that, if yn and zn are two sequences, one has (a)
(b)
A— = ——
—-,
(c) JV-1
N-l
XI l/nA2 n = 2/ n ^n|n=o ~ S n=() n=0
the last formula is called summation by parts. By setting Azn = 6n we have, (d) A^-l
TV
7V-1
n
Y^ y^bn = yN^bi n=0
z=0
known as the Abel's formula. 1.17 Let f(x) be an analytical function in an interval [a, 6]. Ehf(x] = f ( x + h), with x, x + h e [a, 6], prove that
where A h /(x) = f(x + h} - f ( x ) and Df(x] =
Copyright © 2002 Marcel Dekker, Inc.
Putting
32
CHAPTER 1. DISCRETE CALCULUS
1.18 Expand in power series the function f ( x ) = ^-j- and verify that f(x)
=
Z!^=o ^f x ™> where Bn are the Bernoulli numbers.
1.19 Prove that -^ + f = 2 shit */ 2 show that B2k+i — 0, k — 1, 2, . . . .
and using the result of
Problem l-l%i
1.20 Using the results of problem 1.2, show that A"1 = £^0 ^Dn~l. 1.21 (Euler-McLaurin formula). From the previous result, show that D-lf(x)
oc
/" = \ f(x)dx = A7
B
»-
n=l
This formula can be used either to approximate an integral or to approximate a sum using an integral. 1.22 Using the result of problem 1.17 prove that
D-1 = A-1 + i / - - A + - A 2 - . . . . 2 12 24 from which one obtains the Newton integration formula f ( x ) d x = A-V(x) + \f(x} - ^ A/(x) + i A 2 /(^) . . . . 1.23 Show that
This formula can be used to approximate numerically the derivate of a function. 1.24 (Horner Rule). Find the solution of the difference equation Vn = zyn-i +bn,
yQ = bQ.
1.25 Find the solution of yn+\ + yn — n + 1. 1.26 Solve the following nonlinear equation yn+i(l + yn) — yn. 1.27 The Newton method to compute the square root of a positive number A reduces to the difference equation zn+i = ^ (zn + jM. Transform this equation to a linear one and find the solution. 1.28 Solve the following nonlinear equations (A)
Copyright © 2002 Marcel Dekker, Inc.
yn+i = yl-
(B)
yn+i = aypn,
p >1
1.7. PROBLEMS AND REMARKS
33
1.29 Solve the following nonlinear equation: 2/n+l
i -yn
1.30 Find the first integral of
(1 + ynyn+\) yn+2 = yn———2 • 1 + 2/n+i 1.31 Show that, for all integers s, one has
(I - K}-1 = P~\I + K}PS. 1.32 Derive the Vandermonde convolution from (1.34). 1.33 Show that the entries of the inverse of a shifted Pascal matrix are those indicated. (Hint: consider that a Pascal matrix is lower triangular and that a shifted matrix is block diagonal.) 1.34 Show that WQ W\ and V^~ V\ are Frobenius-type matrices. Give an explanation about last columns, which should be the same in both cases. 1.35 Show that where the matrices H and K are defined in Section 1.6, and R(K] is a suitable matrix to be found. 1.36 Suppose that c > 0, kn > 0, kn < e for n > no and S=TIQ
Show that yn < ^ exp ^ E^n0 ^ 1.37 Let h > 0, M > 0, 6 > 0, and
un < h^'M 2^
^T^UJ + 6,
Find a bound for un. 1.38 Suppose that c > 0, kn > 0 for n > no and 00
s=n+l
Show that yn < cexp(^^2.n+1 ks). 1.39 Study the following inequality 2/n+l ^
for a > 1 arid a < 1.
Copyright © 2002 Marcel Dekker, Inc.
0
34
1.8
CHAPTER 1. DISCRETE CALCULUS
Notes
Most of the contents of Section 1.1 to 1.4 may be found in many classical books on difference equations, such as Jordan [97], Milne-Thomson [130], Fort [65], Brand [21], and Miller [128]. More results concerning the discrete Taylor formulas similar to those presented in Section 1.1 can be found in Agarwal [7]. Condition 1.28 can be found in Schoenberg [161]. Many properties of the Pascal matrices have been studied recently. The results presented are taken from Aceto and Trigiante [4], where other references can be found. Theorem 1.6.1 and many other comparison results on difference inequalities are taken from Sugiyama [168]. Theorem 1.6.2 was established by Maslovskaya [20]. Other theorems in the same section are due to Pachpatte [143, 144, 145], and Agarwal and Thandapani [9], Agarwal and Thandapani [9, 10]. For allied results on difference inequalities, see Popenda [152, 153], Mate and Nevai [116], Patula [149], and Agarwal [10].
Copyright © 2002 Marcel Dekker, Inc.
Chapter 2
Linear Difference Equations 2.0
Introduction
This chapter investigates the essential techniques employed in the treatment of linear difference equations. We begin Section 2.2 with the fundamental theory of linear difference equations, and then we develop the method of variation of constants. We then specialize our discussions on linear difference equations with constant coefficients, since this is an important class in itself. We exhibit, in Sections 2.3 to 2.5, methods of obtaining solutions of such equations by the use of difference operators as well as generating functions that offer elegant methods of solving such difference equations. In Section 2.6, we present the theory of stability for difference equations with constant coefficients and discuss in Section 2.7 linear multistep methods for the numerical treatment of differential equations as an application of the stability theory. Section 2.8 considers the scalar boundary value problem that is needed later on. Section 2.9 completes the treatment by filling in the gaps with interesting problems.
2.1
Preliminaries
In the previous chapter, we have seen that the knowledge of A~lg(x) allows us to solve the equation Ay = g ( x ) , which is a very simple difference equation. In general, a difference equation of order k will be a functional relation of the form F(x, y ( x ) , A y ( x ) , . . . , A f e y(z), g(x)) = 0 where y,g : J+Q —» IR. More often, instead of the operator A, one uses the operator E. The difference equation is then written in the form G(x. y ( x ) , E y ( x } , . . . , E k y ( x } , g ( x } } = 0. 35 Copyright © 2002 Marcel Dekker, Inc.
36
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
If the function F (or G) is linear with respect to y ( x ) . A y ( x ) , . . . . A f c y(x) (or, respectively, y ( x ) , Ey(x),..., Eky(x)}, then the difference equation is said to be linear. The theory of linear difference equations will be presented later in this chapter. Except for some specific cases, we shall consider difference equations that can be written in the normal form Eky(x) = $(z, y(z), Ey(x],... , Ek~\(x\g(x)\
(2.1)
where $ : J.+ x D x . . . x D —> D, D C IR is a uniquely defined function. With the equation (2.1). we associate k initial conditions y(x0) = ci,y(x0 + I) = c2,...,y(x0 + k- 1) = ck.
a € D.
(2.2)
An existence and uniqueness result for the Problems (2.1)-(2.2) is the following. Theorem 2.1.1 The difference (2.2) has a unique solution.
equation (2.1) with the initial conditions
Proof. From (2.1) and (2.2), we get y(xo + k). By changing x0 into XQ 4-1 and using the last (k — l) values from (2.2), we obtain y(xQ + l+k}. Repetition of this procedure yields the unique solution y(xo + n) for all values of n. D The possibility of obtaining the values of solutions of difference equations recursively is very important and does not have a counterpart in other kinds of equations. Having at our disposal machines that can do a large amount of calculations in a second, we can get, in a short time, a great number of values of the solutions of a difference equation. For this reason, continuous problems are transformed in approximate discrete problems. This way of obtaining solutions, however, although very efficient for some purposes, is insufficient for others. For example, it does not give information on asymptotic behavior of solutions, unless one is willing to accept costs that may become exceedingly high. Hence, it is of great importance to have solutions in a closed analytical form, or at least to deduce information on the qualitative behavior of the solutions in some other way. Sometimes it is possible to reduce nonlinear equations to linear ones, or to lower order equations. We have already encountered examples of such cases in Chapter 1. Here we provide two more of them. Example 11 Analogous to the differential equation of Riccati type, we have the difference equation (2.3)
Copyright © 2002 Marcel Dekker, Inc.
2.1. PRELIMINARIES
37
where p\(x] , P2(x) , Pz(x) are arbitrary functions defined on J.+ . Equation (2.3) can be reduced to a linear one by setting
2/M = —Z\X) T~} — PI(X)The resulting equation is z(x + 2)
+ \p2(x) - pi(x + l ) ] z ( x + 1) +
\ps(x) - pi(x)p2(x)]z(x)
= 0.
(2.4)
In the following example the nonlinear equation is not reduced to a linear equation, but it is transformed to a lower order one. It arises in the study of iterative processes (see Chapter 7). Example 12 Let us consider the difference equation 5!
(2.5)
•
By setting yn = zn — zn-\, z-\ = 0, equation (2.5) reduces to 2\zn ~ zn-l)
,0 „•,
Zn+i = Zn + +--- , z Jn
which is of second order. Now consider the first order difference equation
where a is a constant. It is easy to check that the solution of (2.7) satisfies (2.5). In fact, by multiplying (2.7) by zn — 1, we obtain 2 Zn+l
zn+izn - z-n+i +
_
~
Zn2
2
from which it follows that \z^+l — zn+\ + ^a = ^(z n +i — zn)2. Thus we have 172 _
2 n
I
i i 11
n ' 2 Zn
II //
2\
n
1
__
^n~ Zn
\2 •
By imposing that the two initial values ZQ are the same, we must" assume a = 2zo. Equation (2.7) is said to be the first integral of (2.5). The solution of (2.7) can be written explicitly (see Problem 1.29).
Copyright © 2002 Marcel Dekker, Inc.
(2.6)
38
CHAPTER 2. LINEAR DIFFERENCE
2.2
EQUATIONS
Fundamental Theory
Since it is not essential to specify the properties of functions with respect to x e IR, we shall use in this section N+Q as the definition set. We shall denote a sequence by {yn}^ which is the set of all values of the function y Definition 2.2.1 Let po(n] = l,pi(n), . . . ,pk(n),gn be k -4- 2 functions defined on N+o . An equation of the form yn+k + Pi(n)yn+k-i + ••• +Pk(n}yn = 9n is called a linear difference
(2.8)
equation of order k, provided that p k ( n ) ^ 0.
The following k initial conditions C
2, • • • , 2/n 0 +A:-l ~ ck,
(2-9)
where c7 are real or complex constants, are usually associated with (2.8) in order to obtain a unique solution. For convenience, we shall state the following theorem, which is a special case of Theorem 2.1.1. Theorem 2.2.1 The equation (2.8) with the initial conditions (2.9) has a unique solution. We shall denote by t/(n,no,c) the solution of (2.8)-(2.9)-, where
c= ( c i , c 2 , . . . , c f e ) eH f c . Thus, we have y(tiQ + j. n 0 , c) = Cj+i,
j = 0, 1. . . . , k - 1.
Definition 2.2.2 If for every n E N+Q, gn = 0, then the equation (2.8) is said to be homogeneous. Introducing the operator L defined by fc
*
(2.10)
z^O
ec{uation (2.8) assumes the form Lyn = 9n
(2.11)
and the homogeneous one becomes Lyn = 0.
Copyright © 2002 Marcel Dekker, Inc.
(2.12)
2.2. FUNDAMENTAL
THEORY
39
It is easy to verify that L is linear since
L(ayn + /3zn) = aLyn + (3Lzn, when a,/3 G IR and {yn}, {zn} are two sequences. Let <S be the space of solutions of (2.12). Because of linearity of L, we have the following result. Lemma 2.2.1 Any linear combination of elements of S lies in S. Let y(n, no, E\ ) , . . . , y(n, no, J5&) be solutions of (2.12), where
and therefore Ly(n,no,Ei) = 0,
i = 1, 2, . . . , k.
(2.13)
Lemma 2.2.2 Given a set of initial conditions c £ M fc , any other solution y(n, no,c) of (2.12) can be expressed as a linear combination of y(n,no,Ei}, i = 1,2,....A;. Proof. If y(n, no, c) is the solution with initial conditions c\, 0 2 , . . . , c^, then the sum k
zn = ^ Cfu(n, no, £"z) -^ ^V
t
C
7
\
"
U
7
t
/
\
(2.14) /
is a solution of (2.12) by Lemma 2.2.1 and also satisfies the same initial conditions zno = ci,z n o + i — C 2 , . . . ,2 no +fc-i = Cfc. By Theorem 2.2.1 the two solutions must coincide, n Let us now take /c functions f i ( n ) defined on 7V+o and define the matrix
h(n]
/ 2 (n)
fk(n)
\
K(n) = \fi(n
Definition 2.2.3 The functions fi(n},i = 1, 2 , . . . , k, are linearly independent if for all n, (2.15) implies c*7; — 0,
i — 1, 2 , . . . , k.
Copyright © 2002 Marcel Dekker, Inc.
40
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
Theorem 2.2.2 A sufficient condition for the functions f i ( n ) , i — 1, 2, . . . , k, to be linearly independent is that there exists n> UQ such that detK(n) 7^ 0. Proof.
If (2.15) holds, then
£}LI
(2.16) This linear homogeneous system of k equations in k unknowns has the coefficient matrix K(n). Thus, if for n = n, detK(n) 7^ 0, the unique solution of the system is aj = 0, i = 1 , . . . , k, and the functions are linearly independent. D Theorem 2.2.3 Suppose that /j(n), i — 1 , 2 , . . . , k, are solutions of (2.12), then detK(n) 7^ 0, for n > UQ, provided that detK(no) / 0. Proof.
From the definition of the matrix K(n), we get
-1)
/2(no + l)
...
= det f*
/
1 \
/ » /
.
7
\
t*
/
.
7
\
Since the columns are solutions of (2.12), the last row can be expressed as a combination of the same solutions at the previous points. By neglecting the zero determinants one arrives at det K(n0 + I) = (-l) f c p f c (n 0 )detK(no). Since Pk(no] ^ 0, it follows that detK(no + 1) 7^ 0. In general, for every n £ ./V+ we have detK(n+l) = (-l)kpk(n}detK(n]
(2.17)
and thus by induction detK(n] ^ 0. D Corollary 2.2.1 The solutions y(n,no, Ei),i = 1,2,..., A; are linearly independent. Proof.
In this case, det K(no) = 1, since
Copyright © 2002 Marcel Dekker, Inc.
2.2. FUNDAMENTAL THEORY
41
/I
0
... 0 \
o -, -, ; : '-. 0 ...
Q
'-. 0 0 1 )
In view of Corollary 2.2.1, the base y(n, no, Ei),i = 1, 2,..., k, is said to be a canonical base. Using Lemma 2.2.2 and Corollary 2.2.1, we have the following result. Theorem 2.2.4 The space S of solutions of (2.12) is a vector space of dimension k. If ai G IRfc, i = 1 , 2 , . . . , fc, are linearly independent, the set of solutions y(n,no,0i) can also be used as a base of the space S. In fact, in this case K(no) — (a\,..., a&) where a^ is the i-th column of the matrix K(HQ). Because they are linearly independent, it follows that detA'(n) ^ 0 for all no'
The matrix K(ri) is called the Casorati matrix and in the theory of difference equations it plays the same role the Wronskian matrix does in the theory of linear differential equations. Definition 2.2.4 Given k linearly independent solutions of (2.12). any linear combination of them is said to be a general solution of (2.12). The term general means that such a solution can satisfy any set of initial conditions. Lemma 2.2.3 The difference satisfies (2.12). Proof.
between two solutions yn and yn of (2.11)
One has Lyn — gn and Lyn — gn, which implies
L(yn - yn] = 0. n Theorem 2.2.5 Let y(n,no,ai),i — 1,2, ...,k, be k linearly independent solutions of (2.12), and yn be a solution of (2.11). Then, any other solution of (2.11) can be written as
k yn - yn + ^aiy(n,no,ai).
(2.18)
Proof. From Lemma 2.2.3, one has that yn — yn £ S and therefore it can be expressed as a linear combination of y(n, no,a?;).D The foregoing theorem also means that the general solution of (2.11) is obtained by adding to the general solution of (2.12) any solution of (2.11).
Copyright © 2002 Marcel Dekker, Inc.
42
CHAPTER 2. LINEAR DIFFERENCE
2.2.1
EQUATIONS
Adjoint and transposed equations
Associated with the operator L, one defines the adjoint operator L* by k
L*yn = ^P^n + ^Vn+i 7= 0
and the adjoint homogeneous and nonhomogeneous equations, L*yn = 0,
(2.19)
L*yn = 9n-
(2.20)
There are some interesting properties connecting the solutions of the equations (2.11)-(2.12) and their adjoints. We shall consider, however, because of its use in numerical analysis (see Section 6.4), the transpose operator LT defined b
and the transpose equation LTyn = gn.
(2.22)
Theorem 2.2.6 Let un be a solution of the homogeneous equation (2.12) and yn a solution of (2.22) with yN+j = 0 for j = 1 . 2 , . . . .k and N > k. Then N
n=0
Proof.
We have
N
N
n=0
n=0
N
k
n=0 i—0
Setting j — n + i and s = min(j, k), one has N +k
TV
E
s
urtOn iiyii — ^> / _j y-j JJ y^pi(? / ,rt'\J — k}u-j-i. / J "
fPIi = 0 for i > k.
Since yj = 0 for j > N and ^Ct^oP'O ~" k)uj-i — 0 for s > /c, the conclusion follows. D
Copyright © 2002 Marcel Dekker, Inc.
2.3. THE METHOD OF VARIATION OF CONSTANTS
2.3
43
The Method of Variation of Constants
It is possible to find a particular solution of (2.11) knowing the general solution of (2.12). This may be accomplished by the method of variation of constants. Let ?/(n, no,c) be a solution of (2.12) and y(n, no, £j), j = 1, 2, . . . , fc, be the canonical base in the space <S of solutions of (2.12). Then, k y(n, n 0 , c] = ^ Cjy(n, n 0 , E j ) . (2.23) j=i We shall consider now the GJ as functions of n with Cj(no) = Cj and require that the function fc y(n,n 0 ,c(n)) = ^Cj(n}y(n,n0,Ej) j=i satisfies the equation (2.11). From (2.24) it now follows that
(2.24)
fc y(n+ I,n 0 ,c(n + 1)) = ^CJ(H j=i fc fc
By setting
fc 5] Acj(n)y(n + 1, n 0 , ^) = 0, j=i
(2.25)
we have fc y ( n + I , n 0 , c ( n + 1)) = ^c3(n)y(n + l,n0,Ej). j=i Similarly, for i = 2, . . . ,k — 1, we can proceed recursively getting fc ?/(n + z,n 0 ,c(n + z)) = J^Cj(n)y(n + i,n 0 ,£j) j=i if we set fc J^ &Cj(n)y(n + i,no,Ej) = 0, i = 1, 2, . . . , k - 1. j=i Therefore, in the end, we obtain fc j/(n + Ar,n 0 ,c(n + /c)) = y^Cj(n}y(n + k,riQ,E3}
, n0,
Copyright © 2002 Marcel Dekker, Inc.
(2.26)
(2.27)
CHAPTER 2. LINEAR DIFFERENCE
44
EQUATIONS
By substituting in (2.11) the result is k k — z,no, c(n + k — 2)) ?:=o k
/\Cj(n]y(n + k , n Q , E j )
Since y(n,riQ,Ej} are solutions of (2.12), one has (2.28)
= gn-
The equations (2.25), (2.27), and (2.28) form a linear system of k equations in k unknowns Acj(n), whose coefficient matrix is the Casorati matrix K(n+ 1). The solution is given by I A Cl (n) \ Ac 2 (n) ,
f\r, (r>}
-«-.„.»
0 L
l
n..
(2.29) I
Denoting by M^(n +1) the (z, k} element of the adjoint matrix of K(n + 1). (2.29) becomes .—+T1)T v ' det A (n
t
1, Z, . . . , /v,
(2.30)
from which it follows that Ci(n) = A"1
dctK(n 4- 1)
(/n + Ui,
By substituting the values of c?(n) in (2.24), we sec that z(n,no,c) = y(n.n().c(n}} satisfies (2.11).
2.4
Linear Equations with Constant Coefficients
If in equation (2.8) the coefficients Pi(n) are constants with respect to n. we obtain the following important class of difference equations: = gn,
Copyright © 2002 Marcel Dekker, Inc.
(2.31)
2.4. LINEAR EQUATIONS WITH CONSTANT COEFFICIENTS
45
The corresponding homogeneous equation is k
(2.32)
^Plyn+k^ = Q. i=0
Theorem 2.4.1 The equation (2.32) has solutions of the form yn = zn,
(2.33)
where z £ C, z 7^ 0, and satisfies
i=Q
Proof.
Substituting (2.33) in (2.32), we have k
i=o
from which it follows that z is a root of 2.34. D Equation (2.34) is a polynomial and it has k solutions in the complex field. Furthermore, it is said to be the characteristic equation of (2.32) and the polynomial p(z) is called the characteristic polynomial. Theorem 2.4.2 If the roots z\, z^,..., zk ofp(z) are distinct, then z™, z % , . . . z% are linearly independent solutions of (2.32). Proof. It is easy to verify that in this case the Casorati determinant is proportional to the determinant of the matrix
1
I
z\
Z2
\
1
(2.35)
V z\ ~ zk
l
z2k'1 • • •
Z
zzk~l k
)
which is known as Cauchy-Vandermonde matrix (or Vandermonde matrix). Its determinant is given by (2.36)
det V(zi,Z2,...,zk) =
which is different from zero if zi ^ Zj for all i and j.
Copyright © 2002 Marcel Dekker, Inc.
D
46
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
From Theorem 2.2.4 it follows that if the roots of p(z] are distinct, any solution of (2.32) can be expressed in the form k
yn = .5>2?.
(2.37)
i=\
When p(z] has multiple roots, the solutions zf corresponding to distinct roots are linearly independent. But they are not enough to form a base in S. However,it is possible to find other solutions and to form a base. Theorem 2.4.3 Let ms be the multiplicity of the root zs ofp(z}. functions ys(n) = us(n}zns,
Then the
(2.38)
where us(n) are generic polynomials in n whose degree does not exceed ms — l, are solutions of (2.32), and they are linearly independent. Proof.
If zs has multiplicity ms as a root of p(z], we have p ( z s ) = Q.p'(zs} = 0 , . . . ,p(m'-1](za) = 0.
(2.39)
By substituting (2.38) in (2.32), one gets k ^Piu3(n + k- i}zks"1 = 0.
(2.40)
i=0
By the relation (1.4) we get k-i /, _ -\
us(n +k- i) = E f c - 7 u,(n) = V j=o\
J
) A J u s (n)
(2.41)
J
and, from (2.40). we have k
E
k—i k — i V~^ >
Tt •7 JL/7 A/ r. r
•'
/
Z j
j=0 k
(j)(-y
3=0
J
\
'
In view of (2.39), one has that the terms of (2.42) corresponding to j = 0 , 1 , . . . , ms — 1 are zero for all functions us(n). To make the other k — ms + 1 terms equal to zero, it is necessary that A J u s (n) = 0 for ms < j < k. This can be accomplished by taking us(n) as a polynomial of degree not greater than ms — 1. The proof that they are linearly independent is left as an exercise. D
Copyright © 2002 Marcel Dekker, Inc.
2.4. LINEAR EQUATIONS WITH CONSTANT COEFFICIENTS
47
Corollary 2.4.1 The general solution of (2.32) is given by d «jUrt tn—
"
d
M~^ \ 7
/
-j
/TT • 91 •/I IY\ \\ 'y n
\Jttl l Lit \ I V I As4
*\
/
I
—
/
/
d mi — 1
mi — 1
V^ \
^
/T •
*-*7
*
V^ \ /
/ ^
7 n f*-nnJv ^-i
^1 ' v
J
I
—
1 V"" V^ \ \ /
/
^
/7 _j
A • • ni>* Tn Zi ^t i
-**7 t I if
<-J
I '
/o I / AO\ /I. -\ \
\ " • Jtt-J I
V
/
where Aij = a^Cj, and d is the number of distinct roots. The next theorem is useful in recognizing if a sequence yn,n £ -W^ is the solution of a difference equation. Theorem 2.4.4 A sequence yn,n E A^0, satisfies the equation (2.32) iff, for alln E N+,,
yn+i yn+2 \ yn+k
• • • yn+k
yn+k+l
•••
yn+2k
\ J
and, moreover, D(yn, yn+i, . . . , yn+k-i] ^ 0. Proof.
Suppose that yn satisfies (2.32). One has
k yn+k+j = - ^Piyn+k+j-i,
j = 0, 1, . . . , k.
By substituting in the last row of D(yn, yn+\, • • • , Un+k), one easily obtains that D(yn, . . . , yn+k] = 0- Conversely, if this determinant is zero, one has, by developing with respect to the first row, Y^i=o yn+k-iAi(n) — 0, where Ai(n) are the cofactors of the (k — i) — th elements. If in D one substitutes to the first row, the second one, and then the other rows, one obtains determinants identically zero. By developing these determinants again with respect to the elements of the first row one obtains k
Y^ yn+j+k-iAi(n) = 0,
j = 1, 2, . . . , k - I .
i=0
The determinant Ak(ri) is not zero by hypothesis. One has, setting
k-l yn
=
-Y^ i-0 k-l
yn+i - -^2, i=0 k-l
yn+k-i
= -^ i=0
Copyright © 2002 Marcel Dekker, Inc.
48
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
By setting n + 1 instead of n in the first relation and subtracting the second one, we have k-l'
0=-
and proceeding similarly for the others, one arrives at an homogeneous system of equations
fc-i ^ Api(n)2/ n+fc +j-i = 0,
j = 1,2, . . . , f c - 1,
i=0
whose determinant ^4fc(n) is not zero by hypothesis. It follows that the solution is Ap z (n) = 0, that is the pi are constant with respect to n, and then the conclusion follows. D Example 13 Consider the equation yn+l -ayn = Q,
a 6 C.
(2.44)
The characteristic polynomial is p ( z ) = z — a, and its unique root is z = a. The general solution of (2.44) is then yn — can. Example 14 Consider the equation
yn+2 - yn+\ - yn = 0.
(2.45)
The characteristic polynomial is p(z) — z2 - z - 1, which has roots l-A/5
l + N/5
Therefore, the general solution of (2.45) is + V/5V + C2
which is known as the Fibonacci sequence. Example 15 Consider the equation 7 Ti -\-1
J Ti ~~~~
jTl'
From Example 13, it follows that the general solution of the homogeneous equation is can. Applying the method of variation of constants it follows that
Copyright © 2002 Marcel Dekker, Inc.
2.4. LINEAR EQUATIONS WITH CONSTANT COEFFICIENTS
49
and
from which we obtain
3=0
Example 16 The following equation often arises in discretization of second order differential equations, namely, yn+2 - 2qyn+i + yn = f n ,
(2.46)
where q E C. The homogeneous equation has the general solution yn = G\ZI + C2^2 , where z\ and 22 are the distinct roots of the second degree equation z2 — 2qz+l = 0. It is useful, in the applications, to write the general solution in two different forms. In the first form, the linearly independent solutions ,.(1) _ i/n
Z2-Z1
_ '
i/n H
_ 2~
i
are used, which give as a general solution of the homogeneous equation + c2yW.
(2.47)
In the second case, one uses the Chebyshev polynomials (see Appendix C) Tn and Un(q) as linearly independent solutions, obtaining yn = ciT n (g)+c 2 t/ n _i(g). The advantage of using (2.47) is that the base yk , yn that is, (i) _ 1
(2.48) is a canonical one,
(2) _
from which it follows that for the initial value problem (2.47) we have
Copyright © 2002 Marcel Dekker, Inc.
50
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
The advantage of the form (2.48) lies in the fact that the functions Tn(q) and Un(q) have many interesting properties that make their use especially helpful in Numerical Analysis and Approximation Theory. The solution of (2.46) can then be written in the following form,
Example 17 Consider the equation
poyn+2 + piyn+i + P2yn = 0.
(2.49)
The solution can be written in terms of the roots of the polynomial p\z +^2 as said above. It is interesting, however, to give the solution in terms of Chebyshev 1 /O
polynomials. Suppose pop2 > 0 and let p = ( — ] and q — — , P l \ \ / 2 One easily verifies that pnTn(q] and pnUn-\(q) are solutions of (2.49). It follows then that yn = cipvTn(q} -f c'2pnUn-i(q} is the general solution.
2.5
Use of Operators A and E
The method of solving difference equations with constant coefficients becomes simple and elegant when we use the operators A and E. Using the operator E. equations (2.31) and (2.32) can be rewritten in the form p(E)yn p(E}yn
= cjm = 0,
(2-50) (2.51)
'.
(2.52)
7=0
One can immediately verify that k
p(E)zn = z n p ( z )
and
p(E) = 1[[(E - zj),
(2.53)
z=l
where z\, z^, . . . . Zk are the zeros of p ( z ) . If there arc s distinct roots with multiplicity m,j,j = l , 2 , . . . , s , then p(E) can be written as p(E) = U*=1(E - zj)™* and (2.51) becomes ziiriyn = 0,
(2.54)
!•= 1
from which it is seen that the homogeneous equation can be split into s difference equations of order nij. In fact, the commutability of the operators (E — z-J] implies the following result.
Copyright © 2002 Marcel Dekker, Inc.
2.5. USE OF OPERATORS A AND E
51
Theorem 2.5.1 The solution xn of the equation (E -
m
Zjl)
>xn = 0
(2.55)
is a solution of (2.54)The problem simplifies further since it is possible to define the inverses of (E - zj),i = 1 , 2 , . . . , A;. Theorem 2.5.2 Let {yn} be a sequence, and let f ( z ) = Y^=Qaizl be a polynomial of degree m. Then, . Proof.
(2.56)
By definition of f ( z ) , it follows that
i—Q
i=0
m
= zn^ai(ziEi)yn = znf(zE)yn.
D
i=0
Definition 2.5.1 The inverse of the operator (E — zl) is the operator (E — zl)'1 such that (E - zI)(E - zl)-1 = / •
Theorem 2.5.3 Let z e C. Then, the inverse of E — zl is given by (E - zl)~l = zn-l/\-lz-n.
(2.57)
Proof. Applying (E — zl} to both sides of (2.57) and using the result of Theorem 2.5.2, one gets (E - zI}(E - zl}~1
=
(E-zI}zn-l&-lz-n
= zn-z-nl = l.
D
Corollary 2.5.1 For m — 1 , 2 , . . . , one has (E - zl)-m = zn'm^-mz'n.
(2.58)
The equation (2.58) allows us to very easily find the solutions of (2.55) and then of (2.50). In fact, from (2.55) and (2.58) we have m
i&-mi . 0.
Copyright © 2002 Marcel Dekker, Inc.
52
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
But we know that A~ m ^ • 0 — qj(ri), where qj(n] is a polynomial of degree less than rrij. Hence, the solution xn is given by xn = z- jqj(n}, and this can be repeated for j = I , 2. . . . , s. Usually, because JH-J is independent of n, one prefers to consider the previous solution multiplied by z • J . The solutions corresponding to the multiple root Zj are then z^qj(n). and hence, the general solution of (2.51) is Hn — Yl'j=\ ajQi(n}z^ which is equivalent to (2.43). In general, to get a solution of the nonhomogeneous equation (2.50), one can proceed as described in the previous section by applying the method of variation of constants. Usually this way of proceeding is too long and in some cases can be avoided using the definition of p~l(E}. Theorem 2.5.4 Let /(z) be a polynomial of degree k and z G C with /(z) ^ 0. Then = -(2.59) Proof.
By applying f(E] to both sides of (2.59), one obtains
=zn.
D
Theorem 2.5.5 Let /(z) be a polynomial of degree k and z\ G C be a root of multiplicity rn. Then, setting g(z) — (z — z i ) ~ m / ( z ) . one has m
Proof.
By applying f ( E ) to both sides and using (2.56), one obtains
Theorem 2.5.6 Let /(z) be a polynomial of degree k and yn be a sequence. Then for every n e N. we have f~l(E}znyn = znf~l(zE)yn. These results can be used to obtain particular solutions of the equation p(E}yn = g ( n ) . Let us consider the most frequent cases: (a) g(n] — g constant. If p ( \ ] / 0, from (2.59) one obtains
Copyright © 2002 Marcel Dekker, Inc.
2.6. METHOD OF GENERATING FUNCTIONS
53
(b) g(n] = £i=i aiz? with p(zi) ^ 0. From (2.59) one has
i=\
(c) Same as in (b), but Zj is a root of p ( z ) of multiplicity m. From (2.59) and (2.60) we obtain
E where ^(z) =
(
„ rn ttiZj ^i
lln-m
~ 7 \ + ai
e
(m)
\—T'
^ . ) m.
(d) g(n] = eina. This case can be treated as in (c) by putting z = eta. (e) g(ri) = cosna.g(n) = sin no;. We can proceed as in case (d) taking the real or imaginary part.
2.6
Method of Generating Functions
The method of generating functions is another elegant method for solving linear difference equations with constant coefficients. Definition 2.6.1 Given a sequences {yn}, we shall call a formal series generated by it the expression ^Tyix\
(2.61)
i=0
where x is a symbol. Only in the case where x will be a complex value, the problem of convergence of (2.61) will arise. The formal series are often called generating functions of {yn}- In the set of all formal series, we can define operations that make such a set algebraically similar to the set of rational numbers. Definition 2.6.2 Given two formal series Y and Z, their sum is defined by
n-O
Copyright © 2002 Marcel Dekker, Inc.
54
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
Definition 2.6.3 The product of two formal series Y and Z is given by OG
YZ = YL c^n
( 2 - 62 )
n=0
where
n
n yn-i.
(2.63)
We list some simple properties of formal series: (i) the product of two formal series is commutative; (ii) given three formal series Y, Z, T we have (Y + Z}T = YT + ZT\ (iii) the unit element with respect to the product is the formal series / = 1 + Ox + Ox2 + . . .; (iv) the zero element is the formal series 0 + Ox + Ox2 + . . .; (v) let Y — Y;i^=oyixl- If Vo T^ Oi then there exist the formal series Y~l such that Y~1Y = I. The polynomials are a particular formal series with a finite number of terms. Consider now the linear difference equation
fc ^Piy-n+k-i = 0,
with
po = I-
(2.64)
i=0
We shall associate with it the two formal series P
= PO + PIX + ...+pkxk,
Y
= yo + yix + ....
(2.65)
P is different from the characteristic polynomial. In fact, one has p(z),
(2.66)
where p(z) is the characteristic polynomial. The product Q of the two series is where
n
qn = ^piyn-i.
(2.68)
In view of (2.64) it is easy to see that qn = 0, for n > k, which means that Q is a formal series with a finite number of terms. Moreover, because P is invertible (po — 1)> one has Y = P~1Q. (2.69)
Copyright © 2002 Marcel Dekker, Inc.
2.6. METHOD OF GENERATING FUNCTIONS
55
If we consider the symbol x as an element z of the complex plane, then (2.69) gives the values of Y as a ratio of two polynomials y 1 _
~~ -i z \
p( )
where p(\) is the characteristic polynomial and q(z) — Yli^oQiZ1roots of p(z) are z"1, where Z{ are the roots of p(z). It is known in the theory of complex variables that every expression like (2.70) is equal to the sum of the principal parts of its poles. The poles of (2.70) are the roots jof the denominators, and therefore, OO
5
Y = E^
rn
j
n=Q
where s is the number of distinct roots of p(z), and nij their multiplicity. The coefficients a?;j are the coefficients in the Laurent series of (2.70). For \z\z\ < 1, i — 1, 2. . . . , s, (1 — Ziz}~i can be expressed as
(
00
\ J
E^" n=0
/
°°
/
=E n=0 V
_|_
•_
' "
1 \
H?2"-
(2-72)
/
and substituting in (2.71), we get CO
S
OO
"lj
' _ 1
'"
n J V^ y"yV-(-l) T"^ J ~ ^ \zn+j / ^ ty'i n = y^2 / j / j / j 2J \
n=0
n=0
z=lj—\
If we write
we obtain
which is equivalent to (2.43). Theorem 2.6.1 Suppose that the roots of the characteristic polynomial p(z] are inside the unit disk of the complex plane. Then the formal series Y converges inside the unit disk.
Copyright © 2002 Marcel Dekker, Inc.
56
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
Proof. The polynomial p(z) has zeros outside the unit disk and q(z}/p(z) has no poles in it. Y must then coincide with the Taylor series in the unit disk. D Theorem 2.6.2 // the characteristic polynomial p(z) has no roots outside the unit disk and those on the unit circle are simple, then the coefficients yn of Y are bounded. Proof. From (2.73), it follows that qi(n) corresponding to \Zi\ = I are constants with respect to n. d The method of generating functions can also be used to obtain solutions of the nonhomogeneous equation (2.11). One can proceed as before with the difference that q^, q^+i-, • • • are not zero. In fact from (2.68) and (2.11) one has q^ = Y^i=oPiyk-i — 9o and, in general, for n — 1, 2 , . . . qn+k
k = ^plyn+k-i = gn-,
(2-74)
i=0
since pi — 0 for i > k. The series (2.67) can be written as k —l
£
n
oo
=
^Zn^piyn-i n=0 z=0
+ ZkY, n=0
=
Q i ( z ) + zkQ2(z],
(2.75)
where as (2.69) becomes
i W + W .
D
(2.76)
The polynomial Q\(z] depends only on the initial values y$, j/i, . . . , yk-i, while Q'2(z) is a formal series defined by the sequence {gn}- Proceeding as in (2.70), (2.71), and (2.72), one obtains the solution {yn}. This procedure can be further simplified by considering that inside the region of convergence, it represents a function /(z), which is said to be the transformed function of the sequence {yn}. For example, in the unit disk, the function (1 — z}~1 is the transformed function of the constant sequence {!}, since
In Table 2.6, transformed functions of some important sequences are given. Now suppose that Q^(z] is the transformed function of {gn}. After
Copyright © 2002 Marcel Dekker, Inc.
2.6. METHOD OF GENERATING FUNCTIONS
57
doing the necessary algebraic operations, one obtains from (2.76)
where G(z) is the function resulting in the right-hand side of (2.76). By expanding G(z) in Taylor series and equating the coefficients of the powers of the same orders on both sides, we arrive at the solution {yn}. Example 18 Consider the equation 2/n+i +Vn = -(n+ 1),
j/o = l.
Here n=0
and 1 V
'
1+ 2
(l + 2 ) ( l - 2 )
2
41 + 2
41-2
2
2(l-2)2'
From Table 2.6 we find that -,
00
*
OO
n=0
1 —£
^_ Q
and
oo
£
- V nzn
(1 — 2) 2 ~
"
'
Therefore,
from which we obtain yn = |( — l) n — | — |n. We give now an example showing how the generating function may help in establishing the difference equation which a sequence satisfies. Example 19 Consider the sequence of polynomials defined by
Does such a sequence satisfy a difference equation? If yes, which one? One may be tempted to use Theorem 2.4.4, which is devoted to solving such
Copyright © 2002 Marcel Dekker, Inc.
58
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
kinds of questions. However, the number of calculations required is huge. The use of eneratin functions is easier. In fact we have
n=Q OO
n =0j=0 CO
i
tXJ
•\
+ J \
7 n
<-~
V^ /
/
x 7 V^ /
' - 7zn ~ ~ ^_—/ /> ,V( — iizV i/'6'/ Z—/ /> , \ I
j=0
S
>
r» •
+ 2j
s=0 V
c 5
II z' ^
j=0
havin set 5=0
One recognizes easily that /o(z) = y^. Moreover we also have
It then follows that l-z
Despite the difficult expressions of the original sequence, the generating function turns out to be very simple. Looking at Table 2.6, one recognizes that the Chebyshev polynomials have a similar, although not the same, generating functions. The denominator may become the same if we change the variable y by setting y = 2(1 — x}. We now leave it as an exercise to show that gn(x) = 2T n (x) - Un-i(x). Being a linear combination of Chebyshev polynomials, the sequence gn(x] will satisfy the same difference equation of such polynomials. In the original variable y. the sequence gn(y) satisfies the equation
9n(y] + (y - 2)# n +i(y) + gn+i(y) = 0. In some applications, especially in system theory, instead of generating functions defined in (2.61), generating functions called Z transform defined by X(z] — Y^^Loynz"n are used. It is evident that X(z] = Y ( M and therefore,
X(z) =
- +z P5^
Copyright © 2002 Marcel Dekker, Inc.
Z-kP(z»
2.7. STABILITY OF SOLUTIONS
59
where Qi (~j is the Z transform of {gn} and p(z) is the characteristic polynomial. Using the tables of Z transforms, everything goes similarly as before.
2.7
Stability of Solutions
The stability problem will be studied in a more general setting in a later chapter (see Chapter 4). In this section we shall consider only the stability problem for linear difference equations, which is very important in applications. Definition 2.7.1 The solution yn of (2.11) is said to be stable if, for any other solution yn of (2.11), the following difference is bounded: (2.77)
no'
Definition 2.7.2 The solution yn. of (2.11) is said to be asymptotically stable if for any other solution yn of (2.11), one has lin^-nx, en = 0.
Definition 2.7.3 The solution yn of (2.11) is said to be unstable if it is not stable. From Lemma 2.2.3 it follows that the difference en satisfies the homogeneous equations (2.12). In the case of linear equations with constant coefficients we have the following results. Theorem 2.7.1 The solution yn of (2.31) is asymptotically stable iff the roots of the characteristic polynomial are within the unit circle in the complex plane. Proof.
From (2.43) we have s
m,; — 1
Inn \yn - yn\ = lira ]T ]T \Al3\ri\z?\. If \Zi\ < 1 one has iimn_+00 \yn — yn\ = 0 and vice versa.
D
Theorem 2.7.2 The solution yn of (2.31) is stable if and only if the roots of the characteristic polynomial have moduli less then or equal to 1 and those having unit moduli are simple.
Copyright © 2002 Marcel Dekker, Inc.
60
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
Proof. From (2.43), it is evident that the terms coming from roots with modules less than 1 gives a vanishing contribution for n —»• oo, while the terms coining from roots with unit moduli give a bounded contribution to e n , since j = 0. D It can happen that for some initial conditions, the solution remains bounded even in the presence of multiple roots on the unit circle, as shown in the next example. Example 1 Consider the equation yn+2 - 2yn+i + yn = 0,
yo = yi = c.
This equation admits the solution yn = c. Often in applications it becomes necessary to study the stability of a constant solution which exists, as we have seen, if gn is constant. Example 2 Consider the equation Vn+2 - 2/n+l + T2/n = 24
We have p(z] = (z — ^) 2 . The constant solution y — 8 is asymptotically stable. In fact, the general solution is given by yn = (c\ -f C2'ri)2~ n + 8 and Hindoo (yn -8) = 0. From Definitions 2.7.1, 2.7.2, and 2.7.3 we see that the properties of stability and instability are usually referred to a particular solution yn. In the case where all solutions tend to a unique solution yn as n —>• oo, it is often said (especially in numerical analysis) that the difference equation itself (or the numerical method represented by it) is asymptotically stable. Moreover, in some branches of applications, a special terminology is used that is becoming more and more popular and worthwhile to mention here. Definition 2.7.4 A polynomial with roots within the unit disk in the complex plane is called a Schur polynomial. Definition 2.7.5 A polynomial with roots in the unit disk in the complex plane with only simple roots on the boundary is called a Von-Neumann polynomial. Using this terminology, the Theorems 2.7.1 and 2.7.2 can be recast as follows.
Copyright © 2002 Marcel Dekker, Inc.
2.8. ABSOLUTE STABILITY
61
Theorem 2.7.3 The solution yn is asymptotically stable, if and only if the characteristic polynomial is a Schur polynomial.
Theorem 2.7.4 The solution yn is stable if and only if the characteristic polynomial is a Von-Neumann polynomial.
2.8
Absolute Stability
One of the main applications of linear difference equations is the study of discretization methods for differential equations. The difference equations, as we have seen, can be solved recursively. This is not possible for the differential equations, and these are usually solved approximately using difference equations that satisfy some suitable conditions. Let us consider the scalar equation j/ = /(t,y), y(*o) = J/o, (2-78) where t G [tQ,T), and suppose that this continuous problem has1 a unique t solution y ( t } . Let h > 0 arid ^ = t0 + ih with i = 0 , 1 . . . . . N = T —tp h • Let the discrete problem approximating (2.78) be denoted by Fh(yn,yn+i,...,yn+k.fn,...Jn+k}
= 0,
(2.79)
where yi = y ( t i ) -\-O(hq),q > I and i = 0, 1, . . . , k — l,n + k < N. Moreover, we have posed /?, = f ( t i , y i } - We suppose that (2.79) has a unique solution yn. As the discrete problem is represented by a difference equation of order ft, it needs k initial conditions, only one of which is given from the continuous problem. Definition 2.8.1 The problem (2.79) is said to be consistent with the problem (2.78) if
, y(tn+i), • • • , y(tn+k}, f ( t n , y(tn}), . . . , f ( t n + k , y(tn+k))) = rn, (2.80) where rn = O(hp+l) with p>l. The quantity rn is called the truncation error. The equation (2.80) can be considered as a perturbation of (2.79). Definition 2.8.2 The discrete problem (2.79) is said to be convergent to the problem (2.78) if the solution yn of (2.79) tends to the solution y(i] of (2.78) for n -> oo, and tn - t0 ~ nh < T.
Copyright © 2002 Marcel Dekker, Inc.
62
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
Since the solution of the continuous problem satisfies (2.80), which is a perturbation of (2.79), the convergence will occur when (2.79) will be insensitive to such a perturbation, that is, when (2.79) is stable under perturbation. As a consequence, the consistency is not enough to guarantee the convergence. We shall study the problem in some detail for the main class of methods called linear multistep methods (LMM). These methods are obtained when Fh is linear in its arguments, namely, k
k
y^ aiyn+i - h^ flifn+i= 0, •^ -^ t ,•/ ?:=o
(2-81)
1=0
with ah = 1 and coefficients a,:, /3t are real numbers. Using the shift operator E and the two polynomials p and a given by (2-82)
a(z] -]T$z\
(2.83)
i=0
equation (2.81) can be written as p(E)yn - ha(E}fn = 0.
(2.84)
The two polynomials p(z) and a(z] characterize the method (2.79) uniquely, and one often refers to them as the (p, a] method. The relation (2.80) becomes p(E)y(tn) - ha(E)f(tn, y ( t n ) } = r n .
(2.85)
Theorem 2.8.1 Suppose that f is smooth enough. In order to have rn = O(h }, the following two conditions need to be satisfied: _ ?:=o
= 0.
(2.86)
k
(2-87)
and k
-X> = o. i-O
(For the proof of Theorem 2.8.1, see Problem 2.19). The conditions (2.86) and (2.87) arc called consistency conditions. If / is nonlinear, then the study of stability of (2.84) is generally difficult. Usually.
Copyright © 2002 Marcel Dekker, Inc.
2.8. ABSOLUTE STABILITY
63
one studies the behavior of solutions of (2.84) for particular linear functions /, which are called test functions. The most used test functions are f ( y ) = 0,
(2.88)
and
f ( y ) = Ay,
Re (A) < 0.
(2.89)
The use of test function (2.88) is justified by considering that in (2.84) the values of / are multiplied by h, and then, in the limit as h —> 0, the contribution of the terms containing fn+l can be disregarded. Also, one sees that the methods which are well-behaving on this test equation, and give good results when applied to the simple equation y' = 0, i.e. they are able to reproduce the constant solutions. The use of test function (2.89) is justified by considering that in the neighborhood of an asymptotically stable solution of (2.84), the first order approximation theorem (see Chapter 4) ensures that the behavior of any solution is established by the linear part that looks like (2.89). Let us first consider the test equation (2.88). Then (2.84) becomes p(E)yn = 0.
(2.90)
Definition 2.8.3 The method (p, a) is said to be 0-stable if the solution yn - 0 of (2.90) is stable. As a simple consequence of Theorem 2.7.2, we have the following result. Theorem 2.8.2 The method (p,cr) is 0-stable if p(z] is a Von-Neumann polynomial. The next theorem states a connection among the above defined concepts, i.e. consistency, 0-stability, and convergence. It is worth noting that it holds under the tacit assumption that real numbers are used. It no longer holds when using finite precision, as is always the case in practice. Theorem 2.8.3 The method (p. a) is convergent in the finite interval (0, T) if and only if it is consistent and 0-stable. Proof. Let us write f(tn,y(tn}} - fn = Cnen, where en = y(tn] - yn. Then, subtracting (2.84) from (2.85), one obtains the error equation p(E}en = ha(E}Cnen + rn = gn, with ,
Copyright © 2002 Marcel Dekker, Inc.
g>l,
j = 0 , 1 , . . . , fc- 1.
CHAPTER 2. LINEAR DIFFERENCE
64
EQUATIONS
The necessary part of the proof will be left as an exercise (see Problems 2.21 and 2.22). Suppose now that the method is 0-stable and consistent; then we can prove the convergence. We will use the formal series method, from which we get oo
E<
n=0
where
fc-i j=o
7=0
OO
By Theorem 2.6.2 and by 0-stability, we see that yn is bounded. By multiplying the formal series we get
s = mm(n,k- 1),
Qi(z)Q3(z) = n=0
0,
n = 0,l,...,fc-l
V n-fc
(2) 2_y?r=0 9». ~1n — k—ii
By equating the coefficients, we have e ra+ fc — %+/c "*~ %+fc «l + I^St • But k-1 i
EE ;=o j=0 ?
where
Y — max
// =
max le, 0
and
Copyright © 2002 Marcel Dekker, Inc.
fc-i
fc-i
?=on j=on
1=
an<
^ \en+k
2.8. ABSOLUTE STABILITY
65
n
k
and
L=
max 0 < i
Moreover,
s=0
n+fc-l
n+k-1
6n + hLB E 5=0
where J3 = X)?=o I ft I and 0 I s —n
T'\ —— \
if s < n if s > n '
T1) ——
s I A;
if s < k if s >
Finally, n+fc-l
^^ + 0n + /iL|/3 fc ||e n+fc | + /iLB ]T |es| or /
ra+fc-1
\
(l-hL\(3k\T}\en+k\
s=0
/
Supposing now that h < L,g ,r and letting F* = l_h^\g i r , we get n+fc-l s=0
^
Copyright © 2002 Marcel Dekker, Inc.
66
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
Corollary 1.6.3 is now applicable and we obtain n+fc-l
\en+k\
< rAHeY*LD(n+^h + ^
/n+k-1 A
^ exp
5=0
n+k-l
<
rAHeT*LB(n+k]h +
max
IT,
Considering that (n -f k}h < T, we get T'LBT _ i
max r. The first term in the right-hand side vanishes in the limit as n —» oo because we have supposed lim/^0 H = 0 and h < T/n. The second term becomes O(hp] because rs =O(/i p+1 ) and the denominator is O(/i). In view of the hypothesis, we have convergence and the theorem is proved. D The above result holds in the limit when h —> 0. It is not very useful in practice because one must use a finite number of steps of integration. Then. it is more useful to use the test function (2.89). In this case, the equation (2.84) becomes k
Y,(<*i - h\Pi)yn+i - 0.
(2.91)
i=0
Setting hX = q and TT(Z, q) = p(z) - q a ( z ] ,
(2.92)
equation (2.91) reduces to *(E,q)yn = Q
(2.93)
with Re (q)< 0. The polynomial Tr(z.q) is called the Dahlquist polynomial. Definition 2.8.4 The method (p, <j) is said to be absolutely stable if the solution yn = 0 of (2.93) is asymptotically stable. As a simple consequence of Theorem 2.7.1, we have the following result. Theorem 2.8.4 The m.ethod (p,cr) is absolutely stable i f n ( z , q ] is a Schur polynomial. Definition 2.8.5 The set of values q G C for which the method (p. a) is absolutely stable is called the region of absolute stability.
Copyright © 2002 Marcel Dekker, Inc.
2.9.
BOUNDARY VALUE PROBLEMS
67
Definition 2.8.6 If the region of absolute stability of the method (p. a) contains the negative complex half-plane, the method is said to be A-stable. Example 3 The method defined by yn+l = yn + h[(l - e}fn+l
+ Ofn]
(2.94)
with 0 < 9 < 1 is called ^-method. One has
p(z) = z - l
a ( z ) = (1 - 9)z + 9.
and
It is easily verified that (2.86) and (2.87) hold. Moreover, n(z,q) = (l-q + qO)z-l-q0 1 -\-nf)
whose unique root is z — 1 _ ? $• The locus of points satisfying \z\ — 1 is the circumference with center (jz^O) and radius l/jzWl- It is seen that for 0 < 0 < \, the region of absolute stability is external to the above circle, while for | < 9 < 1, such a region is internal. For 9 — ^, the circumference degenerates into the imaginary axis, and the region of absolute stability becomes the negative half-plane. As a consequence, we have that the ^-method is A-stable for 0<0<±. In the cases 9 — 0. 0 — 1/2,0 = 1, the 0-method assumes respectively the names of the implicit Euler method, the trapezoidal method and the explicit Euler method.
2.9
Boundary Value Problems
Here we shall discuss in some detail the most common cases arising in the discretization of the linear second-order differential equations. The general problem will be discussed in Chapter 3. Consider the equation yn+i - 2zyn + y n _i = 0
(2.95)
and the three different boundary conditions
yo = yN = Q,
(2-96)
yo
=
0,
(2.97)
yo
=
VN,
yw_i - zyN = 0, Vi = VN+I-
(2-98)
In order to consider these problems, it is very useful to use the general solution of (2.95) in the form presented in Example 16, i.e. yn = c\Tn(z) + c2Un-i(z).
Copyright © 2002 Marcel Dekker, Inc.
68
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
First, let us consider the conditions (2.96) that give d=0,
c2UN-i(z) = Q,
(2.99)
and
l/n = c 2 £/ n _i(z). Condition (2.99) is satisfied if z is a root of UN-\(Z), that is,
(2.100)
/C7T
Zfc = cos — , fc = 1, 2, . . . , JV - 1.
corresponding to which one has N — 1 solutions TT
tk]
/
y{n ' = C2Un-l(Zk)
\
~
•
.
= C2 Sin
The parameter c2 is usually chosen by imposing a normalizing condition. Let us next consider the mixed conditions (2.97). which give rise to ci = 0,C2(t7jv_2(z) - zUN^(z}} = 0 arid using property (7) of Appendix C, the last equation becomes TN(Z] — 0, which is satisfied if
corresponding to which one has solutions (k);
TT i N (fc) . n ( 2 f c + l)?r c2Un-i(zk) = 4 sin -—-
Finally, the periodic conditions (2.98) yield ci(l-TN(z))-c2UN-i(z)
=0
and
ci(T;v+i(z) - z) + c 2 (-l + UN(z}} = 0. This system is a homogeneous one and it has nontrivial solutions corresponding to the roots of its determinants which, after some manipulation using the properties of Tn and Un given in Appendix C, is equal to 2(1 — T/v(z)). This expression is zero for ;,- = cos^,
k = 0,l,...,N-l.
One verifies then that the corresponding solutions arc (k)
n
(k)
—
']
COb
_
CW cog
Ikirn r j\
2fc7rn
L,
; + (t(k) g j n
"' —
2kwn
n N_
' 2 '
^ _ ^ y
N k + —
In applications, equation (2.95) is usually derived from the discretization of -^ + Xy = 0, and then the parameter z has the form z = 1 — ^A(A,/;) 2 . The results show that the solution exists onlv for the values of A given solutions arc called eigcnfunctions.
Copyright © 2002 Marcel Dekker, Inc.
2.10. PROBLEMS AND REMARKS
2.10
69
Problems and Remarks
2.1 Solve the equation yn+2 — ^xyn+\ + yn = O.yo = 1,3/1 = x. where x is a real parameter, and show that yn(x] is a polynomial of degree n in x having the coefficient of xn equal to 2 n ~ 1 . 2.2 For |x| < 1, by setting x — cost, in the previous problem show that (a) the solution is yn — cosnt, (b) the roots of the polynomials yn(x) are simple and lie in the interval [—1,1]. The polynomial yn(x) (usually denoted by Tn(x}} are called Chebyshev polynomials and they are of primary importance in numerical analysis, see Appendix C. 2.3 Show that the solutions (2.38) are linearly independent. 2.4 Suppose we deposit a sum s0 in a bank that will pay at the end of each year an interest proportional to the initial sum. Determine the amount at the end of r-th year. 2.5 As in Problem (2.4), if the interest paid is proportional to the sum of the deposit, find the amount at the end of rih year. 2.6 Suppose we have to pay an initial debt A with a constant rate R, paying an interest rate i. Determine the rate R if the debt has to be paid in 20 years. 2.7 Equation (2.46) deserves a special mention both for its historical importance and for its applications in many different fields. (a) Take y\ — 1 and 7/2 — 1- Determine c\ and c'2 and verify that the sequence obtained is the Fibonacci sequence 1, 1, 2, 3, 5, 8, 13, . . . . (b) Show that lim^^oo ^^ = l "y5 (this number was called the golden section by ancient Greek mathematicians and has been important in the arts through the centuries.) (c) Show that YJi=i Vi = yn+2 ~ I2.8 Find the solution of (a) yn(l + ayn-i] = 1,
(b) yn(b + yn-i] = 1. (Hint: set yn =
Copyright © 2002 Marcel Dekker, Inc.
.)
70
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
2.9 The best known methods for finding the roots of a single nonlinear equation f(x] = 0 are the Newton method and the secant method, defined
by __ n+\ — xn
x
f(xn) , _ f(xn)xn-i - f(xn-i)xn ,., , , and xn+i — — r -, .
If a is the root, denoting the error \xn — a\ with en show that, for sufficiently smooth /, en+i — /e n Cn and en+i — k'nen€n-i, respectively kn and k'n uniformly bounded. The order of convergence of an iterative method is defined by p = lim n _^ 00 1°1^e""1"1 . Show that in the first case p = 2, while in the second case p =
1+ 2
•
2.10 Solve the difference equation zn+\ + zn = — (n + 1). 2.11 Let xn and yn be two different integer solutions of the equation l + Zn — 0,
with a 2 — n\fd — 0. where a. 6, d are integers. Show that, for all values of n, x^ — by^ — 1 (Pell's equation). 2.12 (Bernoulli Method). The solution of the linear difference equation has been obtained considering the polynomial p(z}. It also happens that for finding the roots of a polynomial it is useful to consider a linear difference equation. In fact, supposing that the roots z\. z ^ , . . . . z^ arc all simple and that z\\> z-2 > . . . from (2.37). we have
from which it follows ,. yn+i _ iim — z\.
n->oo
yn
Solving then the difference equation recursively, the ratio of two successive values of the solution will give an approximation of the first root. (a) How can we approximate the root of minimum modulus? (b) What happens if z\ ~ 22? (c) How do we choose the initial conditions in order to avoid the effect of multiple zeros of the characteristic polynomial?
Copyright © 2002 Marcel Dekker, Inc.
71
2.10. PROBLEMS AND REMARKS
2.13 Let A be an s x s matrix. Show that the entries of An+s satisfy a linear difference equation with constant coefficients. 2.14 Let A^, i — 1 , . . . , s be the eigenvalue of an s x s matrix A. Deduce, from the result of previous exercise that, if max \\i < 1, An —> 0. 2.15 Suppose one has to perform the sum Sn — £^=1 ai by using the following algorithm: SQ = 0> <%+! = Si + a^+i. If one performs the sum not using real numbers, but an approximation of them (floating point numbers) that is, instead of the number a = m • 109, where 0.1 < ra < 1, one uses the number a = m • 109, where 0.1 < m < 1 but fh has only t digits. This implies that \a — d\ < \m — ra| • 109 < Wq~l. Study the behavior of the errors, considering that s — (s^-i + a^)(l + lO^ 1 ).
2.16 If
( -2 1
1 -2
1
An =
1
-2 )
Show that Dn — det An satisfies the equation Dn+2 + 2Dn+i + Dn — 0 and that Dn = ( - l ) n ( n + l ) .
2.17 If 2a b
V
b 2a
b 2a
and Dn(X) = det(^4 n — A/), find the eigenvalues of An in the case a2 < be. 2.18 If {yn} has generating function /(z), for z\ < R, show that yn/(n+l) has generating function l/z JQ f ( t ) in the same region. (Hint: integrate term by term.) 2.19 Prove (2.86-2.87). (Hint: expand in Taylor series starting from y(tn) and equate to zero the coefficients of h° and h.) 2.20 Show that the solution of yn+2 - %yn+i + \yn = is unbounded.
, yo = 0, y\ = 0,
2.21 Prove that if the method (/?, a) is convergent (for all / satisfying the hypothesis stated in the text), then it is 0-stable. (Hint: take / — 0.)
Copyright © 2002 Marcel Dekker, Inc.
72
CHAPTER 2. LINEAR DIFFERENCE
EQUATIONS
2.22 With the hypothesis of Problem 2.21, prove that the method is consistent. 2.23 Suppose that the relation (2.85) is given by p(E}yn — hcr(E}fn = e n , where e n , is a small bounded quantity but not infinitesimal with respect to h (this happens in practice when we solve the difference equation on the computer). Using a similar procedure used in the text, prove that |en+^| < E\ + £2, where EI, and E<2 have respectively a zero and a pole for h — 0. Deduce that it is not convenient in practice to use h arbitrary small. 2.24 Find the region of absolute stability for the following methods: (a) yn+2 -yn = 2/?-/n+i (midpoint), (b) yn+2 -yn = |(/n+2 + 4/n+i + fn) (Simpson rule). 2.25 Find the solution of the boundary value problem yn+2 — 2zyn+i +Vn = 0: yi - zyo = 0; yjv-i - zyN = 0.
2.11
Notes
The material of Sections 2.2 to 2.5 is classical and can be found in many classical texts. Most of the results of Sections 2.4 and 2.5 are due to F. Casorati (sec [36], [35]). Theorem 2.2.6 is essentially a compact form of a result in Clenshaw [40] and Luke [115], (see also Section 6.4). The content of Section 2.6 is also classical, but the notation is adapted from Henrici's book [88]. Section 2.7 consists of a collection of results that are scattered in many publications, essentially dealing with numerical analysis. The material of Section 2.8 is based on Dahlquist's papers starting with the fundamental one [45] (see also references given in Chapter 7). The books of Lambert [107] and Gear [71] give more detailed arguments on the subject. A more recent book on the subject is by Brugnano and Trigiante [24], where one can find the proof of the linear independence of (2.38). Theorem 2.8.3 can be found in Henrici [89]. Section 2.9 deals with material that can be found either in some books on numerical analysis (see for example [159]) or in some books on difference equations, see Fort [65]. The classical reference is the book of Atkinson [16], where both the continuous and discrete cases are discussed in addition to many applications.
Copyright © 2002 Marcel Dekker, Inc.
2.11. NOTES
73
TABLE 2.1: Generating Functions Domain of Convergence \z\
mlzm(l- z)-m~l zpm(z)(l-z)-n~l(*) (l-kz)~l m\(l - kz}-m-1
ean
(l-aaz)-1
kn cos an
l—kz cos a l-2kzcosa+k2z2
kn sin an
fcz sina 1 — 2kz cos a4-fc 2 z 2
Brr.
n!
n V rn/
zm(l -
\z\<\ \Z\<1
(m)
(-i) • O
\Z\<1
(1-2ZZ - zx)(l - 2
(*) Pm(z) is a polynomial of degree m satisfying the recurrence relation Pm+i(z) = (rnz + l)pm(z) + z(l - z)p'm(z),pi = 1. (**) Tn(x) and Un(x) are the Chebyshev polynomials (see Appendix C).
Copyright © 2002 Marcel Dekker, Inc.
Chapter 3
Linear Systems of Difference Equations 3.0
Introduction
In this chapter, we shall treat systems of linear difference equations. Some results discussed in Chapter 2 are presented here by using the matrix notation. After investigating the basic theory, where the classical theorem of Poincare is also included, the method of variation of constants and systems representing higher order equations are treated in Sections 3.1 to 3.4. The case of periodic solutions is discussed in Section 3.6. Boundary value problems are dealt with in Section 3.7. The elements of matrix theory that are necessary for this chapter may be found in Appendix A. Some useful problems are given in Section 3.8.
3.1
Basic Theory
Let A(ri) be an s x s matrix whose elements a^(n) are real or complex functions denned on A^0, and yn £ IRS (or C s ) with entries that are functions denned on the same set N+Q. A linear equation yn+i = A(n)yn + bn, (3.1) where bn e IRS, is said to be a nonhomogeneous linear difference The corresponding homogeneous linear difference equation is yn+1 = A(n)yn.
equation.
(3.2)
When an initial vector yno is assigned, both (3.1) and (3.2) determine the solution uniquely on the set A^ as can be easily seen by induction. For 75 Copyright © 2002 Marcel Dekker, Inc.
76 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
example, it follows from (3.2) that the solution takes the form n-1
yn =
(3-3)
-n0
from which the uniqueness of the solution passing through yno follows from nr=Tn0 -^( 2 ) uniquely defined for all n. Since the matrices A(i] may not commute, their order in the product is important: the matrix with lowest index is always the rightmost. The above product should be read as A(n ~ l)A(n-2)...A(no). Sometimes, in order to avoid confusion, we shall denote the solution of (3.1) or (3.2) having yno as initial vector by y(n,nQ,yno). Let us now consider the space S of solutions of (3.2). It is a linear space, since by taking any two solutions of (3.2), it is easily shown that any linear combination of them is a solution of the same equation. Let EI, E<2, . . . . Es be the unit vectors of JRS and y(n, no, Ei), i = 1, 2, . . . , s, the s solutions having Ei as initial vectors. Lemma 3.1.1 Any element of S can be expressed as a linear combination of y(n,riQ, Ei), i = 1, 2, . . . , s. Proof. Let j/(n,no,c) be a solution of (3.2) with yno = c € IRS- From the linearity of S and from c = Y^i=i ci^i-, it follows that the vector
satisfies (3.2) and has c as the initial vector. Then, by uniqueness, zn must coincide with y(n, HQ, c). D Definition 3.1.1 Let fi(n),i = 1,2, . . . , s be vector-valued functions defined on N£Q- They are linearly dependent if there exists constants a,i,i = 1, 2, . . . . s, not all zero such that YA=I aifi(n) = 0 , for all n > UQ. Definition 3.1.2 The vectors /i(n), i = 1, 2, . . . , s are linearly independent if they are not linearly dependent. Let us define the matrix K(n) — (/i(n), /2(n), . . . , / s (n)) whose columns are the vectors f i ( n ) . Also, let a be the vector (ai, 02, • • • , as) . Theorem 3.1.1 // there exists an n e A^+o such that det A'(fi) ^ 0, then the vectors f , ( n ) . i — 1, 2, . . . , s are linearly independent.
Copyright © 2002 Marcel Dekker, Inc.
3.1. BASIC THEORY Proof.
77
Suppose that for n > no s
K(ri)a = ^aj/j(n) = 0. Since detK(n) ^ 0, it follows that a = 0 and the functions /(n) are not linearly dependent. D Suppose now that f i ( n ) , i — 1 , 2 , . . . , s, are solutions of (3.2). One has K(n+l) = A(n)K(n).
(3.4)
Theorem 3.1.2 If fi(n),i = 1, 2 , . . . , s are solutions of (3.2) with det A(n) ^ 0 for n G N£Q and if det A"(no) 7^ 0, tfoen det AT(n) ^ 0 /or a// n G -/V+,. Proof.
For n > no,
detA-(n-fl)
- det(/i(n + 1), / 2 (n + 1),... ,/ s (n + 1))
= detA(n)detK(ri),
(3.5)
from which it follows that
(
n-l
\
JJ det^(z) i=no
det A"(n 0 ).
n
(3.6)
/
Corollary 3.1.1 The solutions y(n, no, Ei),i — 1 , 2 , . . . , s of (3.2) with det A(n) 7^ 0 /or n > no are linearly independent. Proof. In this case detK(no) = I, the identity matrix, and by Theorem 3.1.1, the result follows. D Corollary 3.1.2 // the columns of K(n) are linearly independent solutions of (3.2) with det.A(n) ^ 0, then detK(n] ^ 0 for all n > no. Proof. The proof follows from the fact that there exists an n at which detK(n] ^ 0 and from the relation (3.5). D The matrix K(n}1 when its columns are solutions of (3.2), is called the Casorati matrix or fundamental matrix. We shall reserve the name of fundamental matrix for a slightly different matrix, and call K(n) the Casorati matrix. Its determinant is called Casoratean and plays the same role as the Wronskian in the continuous case. Theorem 3.1.3 The space S of all solutions of (3.2) is a linear space of dimension s. The proof is an easy consequence of Lemma 3.1.1 and Corollary 3.1.1.
Copyright © 2002 Marcel Dekker, Inc.
78 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
Definition 3.1.3 Given s linearly independent solutions of (3.2), and a vector c € IRS of arbitrary components, the vector valued function yn = K(n]c is said to be the general solution of (3.2). Fixing the initial condition yno, it follows from Definition 3.1.3 that c = K~l(n0}yno and y(n,nQ,yno} = K(n)K~l(n0)yno
(3.7)
or, in general, for s G N£Q, ys = c, y(n,s,c) =K(n}K-l(s}c.
(3.8)
$(n,s) = K(n}K-l(s)
(3.9)
The matrix
satisfies the same equation as K(ri), i.e.,
Moreover, $(n,n) = / for all n > no- We shall call <£ the fundamental matrix. In terms of the fundamental matrix, (3.7) can be written as y(n.riQ.yno} = $(n,no)y n o - Other properties of the matrix $ are
(ii) if $(n..s)~ 1 exists, then $-l(n,s] = $(s,n).
(3.10)
The relation (3.10) allows us to define $(s, n), for s < n. Let us now consider the nonhomogeneous equation (3.1). Lemma 3.1.2 The difference is a solution of (3.2). Proof.
between any two solutions yn and yn of (3.1)
From the fact that
one obtains lln+i ™ yn+\ = A(n)(yn - y which proves the lemma.
Copyright © 2002 Marcel Dekker, Inc.
D
3.2.
METHOD OF VARIATION OF CONSTANTS
79
Theorem 3.1.4 Every solution of (3.1) can be written in the form yn = Vn + $(n,n 0 )c, where yn is a particular solution of (3.1) and
(n,no) is the fundamental matrix of the homogeneous equation (3.2). Proof. From Lemma 3.1.2, yn — yn G 5 and an element in this space can be written in the form <&(n,no)c. D If the matrix A is independent of n, the fundamental matrix simplifies because 3>(n, no) = 3>(n — no, 0).
3.2
Method of Variation of Constants
From the general solution of (3.2), it is possible to obtain the general solution of (3.1). The general solution of (3.2) is given by ?/(n,no,c) — <3>(n,no)c. Let c be a function defined on N^ and let us impose the condition that y(n,no,Cn) satisfy (3.1). We then have y(n+ I,n 0 ,c n + i) = $(n + I,n 0 )c n+ i = A(n)$(n, n 0 )c n+ i
from which, supposing that det A(n) ^ 0 for all n > no, we get cn+i — cn + 3>(no, n + l)&nThe solution of the above equation is n-l
j=n 0
The solution of (3.1) can now be written as n-l
j=n 0 n-l
—
<£(n,no)c no + ^ 3>(n,j + l)&j, j=n0
from which, since cno = yno, we have n-l
y(n, no,yno] = $(n,n 0 )y reo + 5Z ^(^,j + l)6j.
Copyright © 2002 Marcel Dekker, Inc.
(3.11)
80 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
By comparing (3.4) and (3.9), it follows that 3>(n,n 0 ) = n?=Tno "^(*)> where n^no* ^(^) ~ ^• We can rewr ite (3.11) in the form /n-l
\
n-l / n-l
\
7—no
In the case where >1 is a constant matrix, 4>(n, no) — An $(n. no) = $(n — no,0). The equation (3.12) reduces to
n
°, and of course
n-l
n 0 , ynQ) =
Let us now consider the case where A(n) as well as bn are defined on N±. Theorem 3.2.1 Suppose that
| < 6,6 G IR + ,j G A r± . Then, n-s-i
(3.14)
is a solution of (3.1). Proof.
For m € N^ consider the solution, corresponding to ym = 0, n-l
y(n,m,0)= and the sequence y(n, m — 1,0), ?/(n, m — 2 , 0 ) , . . . . This sequence is a Cauchy sequence since, for r > 0. e > 0 and rn\ are chosen such that mi
E
<
and
<
Copyright © 2002 Marcel Dekker, Inc.
3.2. METHOD OF VARIATION OF CONSTANTS
81
It follows that the sequence will converge as m —> oo. Let yn be the limit given by yn= £ K(n}K-l(j + l)b^ j=-oo
which is again a solution of (3.1). By setting s = n — j - 1, we obtain
In the case of constant coefficients, this solution takes the form 00
yn = £X&»-3-i,
(3.15)
s=0
which exists if the eigenvalues of A are inside the unit circle. D Let us close this section by giving the solution in a form that corresponds to the one given using the formal series in the scalar case. Let Ayn + bn. (3.16) By multiplying by zn, z G C, and summing formally from zero to infinity, one has i 00 JL ^ •>
n 4-1
00 4 \^
n
00 X ^ 7
n
- y ^ yn+iz ^ = A 2_^ ynz 4- ) bnz . n=0
n=0
n^=0
Letting oo
Y(z) = ^T ynzn, n=0
oo
B(z] = Y^ b^n n=0
and substituting, one obtains
from which
and
Y ( z ) - z-l(z~ll - A)-l(y0 + z B ( z ) ) .
(3.17)
When the formal series is convergent, the previous formula furnishes the solutions as the coefficient vectors of Y ( z ) . The matrix R(z~l,A) = (z~ll — A}~1 is called resolvent of A. Its properties reflect the properties of the solution yn.
Copyright © 2002 Marcel Dekker, Inc.
82 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
3.3
EQUATIONS
Autonomous Systems
The system
yn+\ = Ayn where A is a real n x n matrix deserves a special mention because of its importance in the applications. Naturally, the properties of the solutions depend on the particular properties of the matrix A. Apart from those related to stability that will be discussed in the next chapters, there are other important properties that we need to mention since they are related to important fields of applications such as, for example, discrete Markov chains. Here we consider the following two cases: 1. The matrix A has a leading eigenvalue, i.e. an eigenvalue, say AI, larger in modulus than the remaining ones; 2. The matrix A has an even number of leading eigenvalues having the same modulus. In the first case the solution yn tends (for large values of n) to align with the eigenvector of A I . The assertion is readily proved by considering that if yo is the initial value of the solution, we have yn = Any0. From the expression of An obtained in the Appendix A, we also have
(
<J
\
rnk-l
\ (0
^i + E(vT E " (Afc)^M<+i)h/ok=2 l 1=0 ]
Since, by hypothesis, the ratios -^ are smaller than one in modulus, the proof completes by considering that Znyo is a vector in the direction of the eigenvector corresponding to A I . The above result is important because there are many classes of matrices having such a property. The positive matrices for which the mentioned property is established by the Pcrron-Frobcnius theorem is an example (sec Appendix A). In the second case, each couple of complex conjugate eigenvalues may give rise to a periodic solution. An example of such a case will be given in Chapter 8 (Leslie model).
3.4
Systems Representing High-Order Equations
Any /c-th order scalar linear difference equation yn+k + pi(n}yn+k-]. + . . . + Pk(n}yn = gn
Copyright © 2002 Marcel Dekker, Inc.
(3-18)
3.4.
SYSTEMS REPRESENTING HIGH-ORDER EQUATIONS
83
can be written as a first-order system in IR by introducing the vectors
yn
\ (3.19) V Vk-l
/
\9n J
and the matrix
(3.20)
A(n) = \ -Pk(n)
-Pk-:
-Pi(n) I
Using this notation, equation (3.18) becomes = A(n)Yn + Gn,
(3.21)
where YQ is the initial condition. The matrix A(n) is called the companion (or Frobenius) matrix, and some of its interesting properties that characterize the solution of (3.21) are listed below. (i) The determinant of A(n)-XI is the polynomial (-l)k(Xk+pi(n)Xk~1 + . . . + Pfc(n)). When A is independent of n, this polynomial coincides with the characteristic polynomial of the scalar difference equation; (ii) det.A(n) = ( — l ) k p k ( n ) and A(n) is nonsingular if (3.18) is really a fc-th order equation; (iii)
There are no semisimple eigenvalues of A(n] which are not simple (see Appendix A. This implies that the algebraic and geometric multiplicity of the eigenvalues of A coincide. This property is important in determining the qualitative behavior of the solutions;
(iv) When A is independent of n and has simple eigenvalues zj, 22, • • • , Zfc, it can be diagonalized by the similarity transformation A = VDV~l, where V is the Vandermonde matrix V(z\,Z2,. D = The solution of (3.21) is deduced by (3.11), which in the present notation becomes n-l
The fundamental matrix <£>(n,no) is given by
Copyright © 2002 Marcel Dekker, Inc.
84 CHAPTER 3. LINEAR SYSTEMS
OF DIFFERENCE
EQUATIONS
where the Casorati matrix K(n) is given in terms of k independent solutions f i ( n ) - . h(n)^ • • • • f k ( n ) of the homogeneous equation corresponding to (3.21), i.e.. fk(n) A(n+l)
h(n]
fi(n)
K(n \h(n + k-l]
f2(n + k-l)
...
\
fk(n + k-l) J
Example 4 Consider the second-order equation used to define the Chebyshev polynomials (see Appendix C for the properties of such polynomials) yn+2 - 2zyn+i + yn = 0. In matrix form it becomes
yn+2
o
i
-1
2z
yn
The fundamental matrix of the problem is 0 1 -I 2z
The solutions then have the form
= Cn
yo
2/n+l
It is not necessary to compute Cn directly. It can be obtained more easilv bv considering that K(n\ -
T -1 n Tn+l(z)
Ur.
is the Casorati matrix of the system (see Appendix C). Considering that
1 0 z I \ it follows that Cn — K(n}K~l(0). By using simple properties of Chebyshev polynomials, we obtain
solutions of the above difference equation. We leave it as an exercise to show that there exist values of z and N > 0 such that Cn+N = Cn (see Problem 3.5).
Copyright © 2002 Marcel Dekker, Inc.
3.4. SYSTEMS REPRESENTING
HIGH-ORDER
EQUATIONS
85
Example 5 Consider the same equation of the previous example and suppose that 2z = p = 1+2 . We already know that p2 = p+ I (see Chapter 2). This implies that all the polynomials of degree greater than two in the variable p will be expressible as a first degree polynomial. In order to evaluate yn(p/2), we put yn(p/2) = anp + fin. We then have fin+l) ~ &nP ~ fin,
• fin+2 ~
i.e.,
an+i +fin+lfin+2
— fin-
~
By introducing the matrix 1 1 1 0 / '
P
we can write =F fl n+2
This second-order equation can be transformed as a first-order one by doubling the dimension of the space, i.e. by setting ( fin+l
an \ fin
= B
\fin+l)
\fin+2)
( an \ fin
\fin+l)
where In is the n-dimensional identity matrix. We leave it as an exercise to show that B10 - /4 (see Problem 3.6).
3.4.1
One-sided Green's functions
The solution Yn of (3.21) has redundant information concerning the solution of (3.18). It is enough to consider any component of Yn, for n > no + k, to get the solution of (3.18). For example, if we take the case YQ — 0, from (3.11) we have n-l
(3.22)
Y= j—no
where, by (3.9), $(n,j + 1) = K(n}K~l(j + 1). To obtain- the solution y(n + /c, no, 0) of (3.18), it is sufficient to consider the last component of the vector Yn+i and get
Copyright © 2002 Marcel Dekker, Inc.
CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
Vn+k
EQUATIONS
=
(3.23) where Ek = (0, 0, . . . , 0, 1)T. Introducing the function H(n + kJ) = E^K(n+l}K~l(j
+ l)Ek,
(3.24)
the solution (3.23) can be written as (3.25)
yn+k = J=T10
The function H(n + k, j), which is called the one-sided Green's function, has some interesting properties. For example, it easily follows from (3.24) that H(n + k,n) = l
(3.26)
In order to obtain additional properties, let us consider the identity k J1 — Y^ F FT — 2^-Zl-Zi
C\ 97} W-Z' )
;
i—l
where Ei are the unit vectors in IRfc and / is the identity matrix. From (3.24), one has
=
H(n
EKn
l)Ek,
(3.28)
which represents the sum of the products of the elements of the last row of K(n + 1) and the elements of the last column of K~l(j + 1). By observing that the elements in the last column of the matrix K~l(j + 1) are the cofactors of the elements of the last row of the matrix K(j + 1), it follows that
I detK(j + I)
H(n
x
dct
Copyright © 2002 Marcel Dekker, Inc.
(3.29)
3.4. SYSTEMS REPRESENTING
HIGH-ORDER EQUATIONS
87
As a consequence, we get #(n + fc,n) = 1,
H(n + k--i,n) = Q,
(3.30)
z = 1,2, . . . , fc - 1,
(3.31)
and (3.32) v '
pk(n + k)
Theorem 3.4.1 For fixed j, the function H(n,j) satisfies the homogeneous equation associated with (3.18). That is,
i=0
Proof. It follows easily from (3.29) and the properties of the determinant. D The solution (3.25) can also be written as n— k
(3.33)
with the usual convention ^n0 — 0 if s < no- For the case of arbitrary initial conditions together with equation (3.18), one can proceed in a similar way. From the solution Yn+l = K(n + l)K-l(n0)Y0 +
K(n j=no
by taking the k-th component we have / ^ j=n0
In the case of constant coefficients, the expression for H ( n , j ] can be simplified. Suppose that the roots Z{ of the characteristic polynomial are all distinct. We then have, from (3.28),
/ i
iii=lZi
[Til 2-
-i-i
det^(O)
I
- • •
1
21
•••
zk
j
t LlcL
zK
i
£
(n-7')+fc-l
fc
Copyright © 2002 Marcel Dekker, Inc.
\
"'
z ra-j+fc-1
2 K
fc
Z
(n-?)+/c-l
88 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
where Vi(z\, . . . , ZA,-) are the cofactors of the i-th elements of the last row and p'(zi) is the derivative of the characteristic polynomial evaluated at z\.. In this case, as can be expected, one has H(n + k , j ) = H(n + k — j, 0). By denoting with H(n + k — j) the function H(n + k — j. 0), the solution of the equation k .PiUn+k-i — 9n
such that yr = 0. i — 0 , 1 , . . . . k — 1 is given by n —k
yn ~~ / •*•* \fi
3)9
(3.35)
The properties (3.29), (3.30), and (3.31) reduce to H(k) = l,H(k - s) = 0, s = 1 , . . . , k — 1, and -fiT(O) = — — respectively.
3.5
Poincare Theorem
We shall now state two classical theorems on the growth of solutions of (3.18) with gn = 0. — Pi:i = 1 . 2 . . . . , A:, and if
Theorem 3.5.1 (Poincare). // li the roots of
= l
(3-36)
have distinct moduli, then for every solution yn, ,.
Vn+l
x lirri - = As,
where \s is a solution of (3.36). Proof. Let pl(n] — pl + r/i(n). where by hypothesis r/i —> 0 as n —->• oo. The matrix A(n) can be split as A(n) = A 4- JE/c?7T(n), wrhere 0 0
1 0
0 ... 1 0
0
0
1
\
A= 0
V -Pk -Pk-i
-Pi J
(-7/ f c (n), -r/ f c _i(n),. .. , -rji(n)) and E% = (0, 0,. .. , 1). Equation (3.21) then becomes Yn+i = AYn + Ek7jT(n)Yn. Now A = VAV~l. where
Copyright © 2002 Marcel Dekker, Inc.
3.5. POINCARE THEOREM
89
V is the Vandermonde matrix made up of the eigenvalues AI, A2, . . . , \k of A (which are the roots of (3.36)) and A = diag(Ai, A2, . . . , A&). We suppose that |Ai| < |A2J < ... < |Afc|. Changing variables u(n] — V~1Yn and letting Tn = V~lEkr]T(n)V ', we get u(n + 1) = Au(n) + Tnu(n). The elements of F n , being linear combinations of n^(n), tend to zero as n —> oo. This implies that for any matrix norm, we have ||rn|| —> 0. Suppose now that maxi<; no, the function s(n) is not decreasing. In fact, we know that for i < j, M < 1. Take e > 0 small enough such that L i i < 1 and choose oose no sso that for n > no, Hindoo < e. Setting s(n + 1) — j, it follows that \u8(n + 1)| > |A s ||u s (n)| - IIFnHooMn)! = (|A a | and
|A||'Uj(n)| + e|w s (n)| < (|Aj| + e)\us(n)\ |wj(n + l)|
>
|Aj||uj(n)| — e|w s (n)|.
Consequently, if s(n -f 1) = j were less than s(n), the following would be true:
which is in contradiction with the definition of j. For n > N suitably chosen, the function s(n) will then assume a fixed value less than or equal to k. We shall show now that the ratios usn u n tend to zero. In fact we know that for n > N, lu.si^nji ^ '\ < a < 1. This means that a is an upper limit for (3.37). We extract for n > N a subsequence ni,n2, . . ., for which (3.37) converges to a. Suppose first that j > s. Then
|A s | + e We take the limit of the subsequence, obtaining a lower limit
Copyright © 2002 Marcel Dekker, Inc.
90 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
This implies
|A s | + e
lim
\u-(n + 1)1 \u-(n +1)1 ; — < lim -—; — = cv
P^OQ us(np + 1)|
P-*oo|Ws(np +
for arbitrary small e. Since |^4 > 1, the previous relation holds only for a = 0. In the case j < s, similar arguments, starting from +e
us(np + 1)|
|A,S| - e
lead to the same conclusion that a = 0. Consider now the original solution yn. Considering the first two rows of Yn •= Vu(n), we have,
/ \ s(n)
= \ Xsus/(n)\
One easily verifies, by using the previous results, that
r
hm
Vn+l
— \x s.
r-,
D
Vn
We shall now state, without proof, the following theorem due to Perron, which improves the previous one. Theorem 3.5.2 (Perron). Suppose that the hypotheses of Theorem 3.5.1 are verified, and moreoverPk(n] 7^ 0, for n > HQ. Then k solutions f\. /2,. . . , //can be found such that
3.6
Periodic Solutions
Let 7V be a positive integer greater than 1. A solution yn of a first order difference system is said to be periodic of period N if yn+N — Un-
Copyright © 2002 Marcel Dekker, Inc.
3.6. PERIODIC SOLUTIONS
91
Example 6 Consider the difference system defined in Example 5. Since Bw = /, any solution of such a system has period 10. This implies that the solutions of the difference equation y n+2 (p/2) - pyn+i(p/2) + yn(p/2) = 0 have the same period. In particular we have T\o(p/2) — To(p/2) = 1. This in turn implies p — 2cos(27T/10), which essentially reflects an old result of elementary geometry relating the golden ratio to the regular decagon (see Problem 3.20). In the previous example the periodic solution is only one, but in other cases there may be many, as shown by the following example. Example 7 Consider Example 5. In order to have CN = /, it should be that UN-I(Z) — 0 and UN(Z) = I . This is accomplished by taking rT IT
z — cos( — ). iV
k G f l , N — I}, J
k even.
Fixing TV, one may find many values of z to which periodic solutions of period TV correspond. Consider now the case of non-autonomous first-order systems. Let A(ri) be real nonsingular s x s matrices and bn vectors of ]RS. Suppose that A(n) and bn are periodic of period N. That is, A(n -f TV) — A(ri) and 6n+yv = bn. The trivial example of periodic solutions of the homogeneous equation (3.2) is yn = 0. For the nonhomogeneous equation (3.1), if there exists a constant vector y such that (A(n) — I}y + bn — 0 for all n, then y is trivially periodic. In this section, we shall look for the existence of nontrivial periodic solutions. Theorem 3.6.1 If A(n] is periodic of period TV, then the fundamental matrix 3> satisfies the relations
l
j
Proof. The proof follows easily from the definition of $ (see (3.9)) and the hypothesis of periodicity of A(n). D Theorem 3.6.2 If the homogeneous equation (3.2) has only yn = 0 as periodic solution, then the nonhomogeneous equation (3. 1) has a unique periodic solution of period N and vice versa.
Copyright © 2002 Marcel Dekker, Inc.
92 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
Proof. If yn and yn are periodic solutions for (3.1) and (3.2) respectively, both having the same initial point yg, they must satisfy
N-l
, 0)7/0 + £ 3=0
from which it follows that T/Q must satisfy
By0
= 0
Byo
=
(3.39)
N-l
^^(NJ + Vbj,
(3.40)
j=0
where B = 7 - $ ( 7 V , 0 ) . The solution of each of the two equations (3.39) and (3.40) will give the initial condition for the periodic solutions of the equations (3.2) and (3.1). If (3.39) has the unique solution yo = 0, it follows that det B ^ 0 and this implies that (3.40) has a unique nontrivial solution. The converse is proved similarly. D Suppose now that detB — 0 and N(B) (the null space of B) has dimension k < s. This means that (3.39) has k solutions to which k periodic solutions of (3.2) will correspond. Problem (3.40) has solutions if the vector 7V-1
Y^*(NJ + l)bj j=o is orthogonal to N(BT). Suppose that t/ 1 ), i/ 2 ), . . . , v^ is a base of N(BT), then we have , 0) = 0,
i = 1, 2, . . . , k,
(3.41)
from which T
vd)
= vW
T
$(jv ? o),
i = 1, 2, . . . , k.
(3.42)
By imposing the orthogonality conditions we get JV--1
N-l
_
_
J=0
j=0
+ l)6j; = 0.
(3.43)
Let
1,2,...,*,
Copyright © 2002 Marcel Dekker, Inc.
(3.44)
3.6. PERIODIC SOLUTIONS
93
so that we obtain N-l
]T xf j=o
bj = 0,
i = 1,2, . . . , / c .
(3.45)
From this result we get the following theorem. Theorem 3.6.3 If the homogeneous equation (3.2) has k periodic solutions of period N, and if the conditions (3-45) are verified, then the nonhomogeneous equation (3.1) has periodic solutions of period N. Let us now consider the functions defined by (3.44) and let X j , j 6 be one of them. Then, 1, j).
The vector x3 are periodic with period TV and satisfy the equation (3.46) which is called the adjoint equation of (3.1). The fundamental matrix ^(t, s for such equation satisfies the equation *(s,s)=/.
(3.47)
Using (3.46), Theorem 3.6.3 can be restated as follows. Theorem 3.6.4 If the homogeneous equation (3.2) has k periodic solutions of period N, and if the given vector b — (69, 6 1 , . . . . 6,/v-i)T is orthogonal to the periodic solutions of the adjoint equation, then the nonhomogeneous equation (3.1) has periodic solutions of period N. Theorem 3.6.5 Suppose A(ri] and bn are periodic with period N. If the nonhomogeneous equation (3.1) does not have periodic solutions with period N, it cannot have bounded solutions. Proof. If (3.1) has no periodic solutions, by Theorem 3.6.2 the equation (3.1) has such solutions and of course the conditions (3.45) are not verified. Let VT be a solution of vTB — 0. Then N-l j=0
Consequently, for every solution yn of (3.1), we have N-l j=0 N-l
=
Copyright © 2002 Marcel Dekker, Inc.
V Vo + VT Y
94 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
Moreover, by considering the periodicity of bj and (3.38), we get N-l
+ VT ]T j=o N-l
and in general, for k > 0, N-l
vTykN = vTyo + kvT ]T j=0
showing that yn cannot be bounded. D The matrix U = &(N, 0) has relevant importance in discussing the stability of periodic solutions. From
and (3.38). it follows that: $(n + 7V,0) = $(n,0)t/.
(3.48)
$(n + /cTV, 0) = $(rc, 0)f/ f c .
(3.49)
and in general, for k > 0,
Suppose that p is an eigenvalue of U and v the corresponding eigenvector. Then $(n + TV, 0)t; = $(n, Q)Uv =
Letting $(n..Q}v = vn for n > 0, we get (3.50) This means that the solution of the homogeneous equation having initial value vn, after one period, is multiplied by p. For this reason the eigenvalues of U are usually called multiplicators . The converse is also true. If yn is a solution such that yn+N — PVn for all n, then in particular y,v — pyo and that means Uyo — pyo from which follows that yo is an eigenvector of U. As seen in Example 4 periodic solutions may also arise when the matrix A(n) is constant.
Copyright © 2002 Marcel Dekker, Inc.
3.7. BOUNDARY VALUE PROBLEMS
3.7
95
Boundary Value Problems
The discrete analog of the Sturm-Liouville problem is the following: ) + (qk + \rk)yk = 0,
+
(3.51)
(3.52)
= 0,
where all the sequences are of real numbers, rk > 0, QQ 7^ 0, OLM 7^ 0 and 0 < k < M. The problem can be treated by using arguments very similar to the continuous case. We shall transform the problem into a vector form, and reduce it to a problem of linear algebra. Note that the equation (3.51) can be rewritten as
+Pk-\}yk + Pk-iyk-i + (Qk + Ar fc )yfc = o. Let =
Pk+Pk-i-Qk,
po,
PM +PM-I -
and -pi -P2
(3.53)
A= -PM-I
-PM-I
\
a-M
)
Then the problem (3.51)-(3.52) is equivalent to Ay = XRy.
(3.54)
This is a generalized eigenvalue problem for the matrix A. The condition for existence of solutions to this problem is
dct(A - XR) = 0,
(3.55)
which is a polynomial equation in A. Theorem 3.7.1 The generalized eigenvalues of (3.54)
Copyright © 2002 Marcel Dekker, Inc.
are
96 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE Proof. roots of
EQUATIONS
Let S = R~l/2. It then follows that the roots of (3.55) are det(SAS - XI) = 0.
(3.56)
Since the matrix 5^45 is symmetric, it will have real eigenvalues. D l For each eigenvalue A7;, there is an eigenvector y which is the solution of (3.54). By using standard arguments, it can be proved that if yl and yj are two eigenvectors associated with two distinct eigenvalues, then M
(y%V) = £M/X = 0.
(3.57)
5=1
Definition 3.7.1 Two vectors u and v such that (u, Rv) — 0 are called R-orthogonal. Since the Sturm-Liouville problem (3.51)-(3.52) is equivalent to (3.54), we have the following result. Theorem 3.7.2 Two solutions of the discrete Sturm-Liouville problem corresponding to two distinct eigenvalues are R-orthogonal. Consider now the more general problem yn+i = A(n]yn + bni
(3.58)
where yn,bn E IRS and A(n) is an s x s matrix. Assume the boundary conditions are given by N
Y^L,yni=w
(3.59)
i=0
where n% 6 A^. n, < nl+\, n$ = 0, w is a given vector in IR'S and Lr are given s x s matrices. Let $(n, j) be the fundamental matrix for the homogeneous problem = A(n)yn, (3.60) such that $(0,0) = /. The solutions of (3.58) are n-l yn =
(3.61)
where 2/0 is the unknown initial condition. The conditions (3.59) will be satisfied if N
N
Copyright © 2002 Marcel Dekker, Inc.
N
rii-1
3.7. BOUNDARY
VALUE PROBLEMS
97
which can be written as N
TV
nN-l L
£]L z $(n z ,(%o + E * E z=0
i-0
$fcJ + l)T(j + l , n 0 6 j = « > ,
(3.62)
j=0
where the step matrix T(j, n) is defined by
for j < n, for j > n.
/ ', n) — -i . ' ^ 0
r
By introducing the matrix Q = ^ = 0 Lj(nj,0), the previous formula becomes TV
l,ni)6j.
Y,
(3.63)
Theorem 3.7.3 // the matrix Q is nonsingular, then the problem (3.58) with boundary conditions (3.59) has only one solution given by 71./V-1
2/n = $(n,0)g-W £ C?(n,j)6j, j=o
(3.64)
where the matrices G(n,j) are defined by: TV
^ L t $(n z , j + l ) T ( j + 1, n.).
(3.65)
i=0
Proof. Since Q is nonsingular, from (3.63) we see that (3.61) solves the problem if the initial condition is given by TV nyv-1 l
1
yo = Q~ w - Q" E E i=Q
L
^(^3 + 1)^0' + l,ni)6j.
j=Q
By substituting in (3.61) one has TV nAr-1
yn
= t=0 j=0
Copyright © 2002 Marcel Dekker, Inc.
(3.66)
98 CHAPTER 3. LINEAR SYSTEMS
OF DIFFERENCE
EQUATIONS
from which, by using the definition (3.64) of G(n, j), the conclusion follows. D
The matrix G(n,i) is called Green's matrix and it has some interesting properties. For example, (1) for fixed j, the function G(n, j) satisfies the boundary conditions N
X;^G(ni,j) = 0, i=0
(2) for fixed j and n ^ j, the function G(n,j] satisfies the homogeneous equation
(3) for n = ] , one has G(j + l,i) = A ( j ) G ( j J ) + I. The proofs of the above statements are left as exercises (see Problems 3.26 and 3.27). If the matrix Q is singular, then the equation (3.65) can have either an infinite number of solutions or no solution. Suppose for simplicity we indicate the righthand side of (3.63) by b. then the problem is reduced to establishing the existence of solutions for the equation Qyo = b.
(3.67)
Let R(Q) and N(Q) be respectively the range and the null space of Q. Then (3.67) will have solutions if b € R(Q}. In this case if c is any vector in N(Q) and t/o any solution of (3.67), the vector c + yo will also be a solution. Otherwise if b 0 R(Q), the problem will not have solutions. In the first case (b e R(Q}), a solution can be obtained by introducing the generalized inverse of Q, defined as follows. Let r — rank Q. The generalized inverse Q1 of Q is the only matrix satisfying the relations -P, QQ!Q = Q,
QIQ = Pl,
(3.68)
Q'QQ1 = Q 7 ,
(3.69)
where P and P\ are the projections on R(Q) and R(Q*) (Q* is the conjugate transpose of Q} respectively. It is well known that if F is an s x s matrix whose columns span R(Q), then P is given by )-1F*.
(3.70)
By using Q1 . the solution yo of (3.67) when b G R(Q] is given by ^o = Q!b.
Copyright © 2002 Marcel Dekker, Inc.
(3.71)
3.8. PROBLEMS
99
In fact, we have Qyo — QQ!b — Pb = b. A solution yo of the boundary value problem can now be given in a form similar to (3.64) and (3.65) with Q~l replaced by Q1. This solution, as we have already seen, is not unique. In fact if c € N(Q), y^ — $(n, 0)c 4- yo will also be a solution satisfying the boundary conditions since N
TV
n
Y^ Liyni = ^ $(rii, 0)c + ^ Llyni = Qc + w = w. i=0
i=0
i-0
When b $. -R(Q), the relation (3.71) has the meaning of the least square solution because yo minimizes the quantity \\Qyo ~b\\2 and the sequence yn defined consequently may serve as an approximate solution.
3.8
Problems
3.1 Prove that if A(n) is nonsingular for n > no, then 3>~l(n, s) exists for n, s > HQ. (Hint: write $(n, s) = Yl™=s A(i] and invert.) 3.2 Show that if K(n) satisfies (3.6), with K(ri) ^ 0 for all n, its columns form a set of linearly independent solutions of (3.2). 3.3 Let xn and yn be two different integer solutions of the equation zn+2 - 2azn+i + zn ~ 0
a2 - by(d) = 0,
where a, 6, d are integers. Show that, for all values of n, x2n -byl = l (Pell's equation). 3.4 Prove that if A is a constant matrix, $(n, no) = $(n — no) either directly by the explicit expression of $(n,no) or as a solution of (3.6). 3.5 Show that the matrix C in Example 4 is such that CN = ±1 if one chooses z = cos(^). 3.6 Show that B10 = I in Example 5. 3.7 Prove the relations (3.31) and (3.32). 3.8 Prove Theorem 3.4.1 directly by using the expression (3.28). 3.9 Verify that (3.33) satisfies equation (3.18). 3.10 Verify that yn = Y^7=-oo Asbn^s-i is a solution of yn+i = Ayn + bn When docs this solution have a meaning?
Copyright © 2002 Marcel Dekker, Inc.
100 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE 3.11
EQUATIONS
Verify that H(n) is solution of Y^i=oPiyn+k-i = 0-
3.12 Supposing that X^?=o-^( n ~ J}9j nas a meaning, show that it is the solution of the autonomous linear scalar difference equation. When does it have a meaning? 3.13 As in the previous exercise with the function yn = Yl^L-oo H(n~~j}9j^ where H(n] is the solution Y^i=oPiH(n + k — i) = 5~l, find the conditions on the roots of p ( z ) in order to have yn as the only bounded solution. 3.14 Consider the second-order difference equation yn+2 + P\yn+\ +P2yn = gn. Construct the function H(n) of the previous exercise. 3.15 Find the function H(n — j) and the solution of yn+2 ~ 2(cost)y n+ i +
yn = 9n,yo = 2/1=0. 3.16 By using the one-sided Green's function H defined by (3.34), find the inverse of the matrix:
/ a (3 7
\
''•
13 a / When does the inverse exist for N —> oo (infinite matrix)? 3.17 Deduce the adjoint equation in the scalar case as defined in section 2.2 from the Formula (3.46). 3.18 Find the periodic solutions for the problem considered in Example 4. 3.19 By using the fundamental matrix found in Example 4, prove the following identity for Chebyshev polynomials: 77nu 77. 77n — iuj ,77.— i, — — <_y 77 n _f_j. , u j _ u
3.20 The parameter p in Example 5 is the golden ratio. A very ancient result in Eucleadean geometry is that p is equal to the ratio between the radius of the circumcircle and the edge of the regular decagon. By using the result that 2cos(^) — p, prove the theorem. 3.21 If in Example 5 one chose p — \/2. show that all solutions have period eight.
Copyright © 2002 Marcel Dekker, Inc.
101
3.9. NOTES 3.22 Let zi,Z2,... z^ be a sequence of numbers. Show that k
^s-1
1
s=k
3.23 Show that A 2 y n + 4y sin2 ^yn = 0, yo> DM = e(^ 0) has no solution. It is interesting to notice that the continuous analog of this problem namely, = y(M) — e, has the solution y ( t ) — ~j^ = 0, yn — 0, y sin(2Af sin 5^7)
3.24 Find the solution of A2yn + f^yn = 0, y(Q) = y ( M ) = 0, forM > 2. 3.25 Find the eigenvalues of A 2 y n + Xyn = 0, yo — 0, T/M+I = 0. Show that the problem is equivalent to finding the eigenvalues of the matrix: /
2
-1
0
-1
2
'-.
-.
-1
-i 2 y M x M
V
3.26 Show that for fixed jf and n 7^ j,G(n,j) satisfies the homogeneous equation G(n + i , j ] = AG(n, j). 3.27 Show that for j + 1 and j in the same interval [n,:_i, r^ — 1] one has
3.28 Show that, for fixed j, G(n,j) satisfies the boundary conditions
i=0
3.29 Verify that (3.64) satisfies Equation (3.58) and the boundary conditions (3.59).
3.9
Notes
Most of the material of Sections 3.1, 3.2, and 3.4 appears in several different books. The major references are Miller [129] and Halm [81]. The Poincare and Perron theorems can be found in Gclfond [74, 75]. More recent works
Copyright © 2002 Marcel Dekker, Inc.
102 CHAPTER 3. LINEAR SYSTEMS OF DIFFERENCE
EQUATIONS
on the subject can be found in Mate and Neva! [117], and Trench [172]. An overview is provided by Elaydi [61]. The periodic solutions of scalar difference equations have been studied in recent years by Ladas and his group, see for example [102], Filho and Carvalho [135], Cook and Ladeira [42], Agarwal and Popenda [5], Dannan, Elaydi and Liu [53], Pang and Agarwal [146]. The periodic solutions for linear systems are treated in Halanay [85], Halanay and Wexler [86], and Cordimeanu [44 . For the boundary value problems see Fort [65], Mattheij [119, 120], Agarwal [6, 7], and Hildebrand [92]. Theorem 3.7.2 is taken from Agarwal [6].
Copyright © 2002 Marcel Dekker, Inc.
Chapter 4
Stability Theory 4.0
Introduction
In Section 4.1, we define various stability notions and give some simple examples. Sections 4.2 to 4.4 are devoted to the theory of stability of linear difference equations. Using the norm as a candidate for measure and the comparison principle, we develop general results on stability in Section 4.5. Nonlinear variation of constants formula is obtained in Section 4.6, and stability by first approximation is treated in Section 4.7. Sections 4.8 and 4.9 investigate stability theory in terms of Liapunov functions and the comparison principle and offer several direct theorems of importance. Section 4.10 contains a discussion of converse theorems. Section 4.11 deals with the concepts of total and practical stabilities that are important in all applications for example in numerical analysis. Several problems are included in Section 4.12 that complement the theory presented in the chapter.
4.1
Stability Notions
Let us denote by B ( y , 5 ] the open ball having its center at y and radius 6. If y = 0, we shall use the shorter notation B$. Let D C JRS, y0 e D and / : 7V+o x D —» D. In the following the origin as well as all the balls such as B§, Be needed for the discussion will always be supposed to be contained in D. Consider the difference equation yn+i = /(n,y n ),
yno = yo-
(4.1)
Definition 4.1.1 The points y e IRS, which satisfy the algebraic equation /(n,y) = y for all n are called fixed points (or critical or equilibrium points) of (4-1)103 Copyright © 2002 Marcel Dekker, Inc.
104
CHAPTER 4. STABILITY THEORY
We shall suppose, for simplicity, that there is a fixed point at the origin. If the fixed point is not at the origin, changing coordinates zn — yn — y, equation (4.1) is transformed into zn+i = /(n, zn + y] - /(n, y) = F(n, zn), which has the fixed point at the origin. Definition 4.1.2 The solution y = 0 of (4.1) is said to be (i) Stable if, given e > 0, there is a £(e, no) > 0 such that for any y$ G B$, the solution yn G J3e; (ii) Uniformly stable if it is stable and 6 can be chosen independent of no; (iii) Attractive if there is S(UQ) > 0 such that, for y$ G B$, one has lim yn = 0;
n—>oo
(iv) Uniformly attractive if it is attractive and 5 can be chosen independent of no; (v) Asymptotically stable if it is stable and attractive; (vi) Uniformly, asymptotically stable if it is uniformly stable and uniformly attractive; (vii) Globally attractive if it is attractive for all starting points yo G IR'S; (viii) Globally, asymptotically stable if it is asymptotically stable for all
yo e IR'S: (ix) Uniformly, exponentially stable if there exists 5 > 0, a > 0, 77 G (0,1) such that if yo G B^. then \\Vn\\ < a\\yo\\rf (x) /p-stable if it is stable and if moreover for some p > 0,
(xi) Uniformly /^-stable if the previous summation converges uniformly with respect to no-
Copyright © 2002 Marcel Dekker, Inc.
4.1. STABILITY NOTIONS
105
Example 8 Consider the difference equation
where a^ are real numbers. The solution of (4.2) is n-l
(4.3) i=no
We then have the following cases:
a) If n-l
JJ |ai| < M(n 0 ) i=no
then |xn| < |x 0 |M(n 0 ), and it suffices to take 8(e.no) — Mf
b) If
s to get stability.
M
n-l
n N^ '
then it suffices to take 8 — e/M to get uniform stability. For example, this condition is satisfied if a, = cos i.
c) If n-l
lim TT lad = 0, n—».rv~i JL J* then, as this is a particular case of case a), the stability follows. Moreover, n-l
lim \xn\ = XQ\ lim FT laA = 0
n—»oo
n—*oo _•*•-*•
from which the asymptotic stability of the zero solution follows. This condition is not satisfied when ai = cos i as in the previous case, showing that uniform stability and asymptotic stability are two different concepts.
d) If H ai\ 0 and 0 < 77 < 1, then x = 0 is exponentially stable.
Copyright © 2002 Marcel Dekker, Inc.
106
CHAPTER 4. STABILITY
THEORY
There is a sort of hierarchy among all these different kinds of stabilities. For example, uniform asymptotic stability implies asymptotic stability, and uniform stability implies stability. Furthermore, asymptotic stability implies stability. Equation (4.1) is said to be autonomous if / does not depend explicitly on n. For autonomous equations, uniform stability concepts coincide with the stability ones. This can be seen by observing that if y(n, no, yo) satisfies the autonomous equation y(n + l,no,yo) = f ( y ( n , n Q . y o ) ) . then y(n — no,0,?/o) satisfies the same equation. Since the two solutions assume the same value for n = no, they must coincide for n e N^0- This means that for autonomous equations we can always fix no — 0. If there is stability for no — 0, the same will be true for all no, which means that stability is uniform. Attractivity is a different concept than stability, as the following example demonstrates. Example 9 Consider the system
9, rn + T™
r, I <-> I /. .
O™
yr>\yTl
^""^71J
'n '
i . A\ (4.4)
\
*>
/
^
\
ri
where r^ — :c^ + yT2r The origin of (4.4) is globally attractive but unstable (see Problem 4.3).
Of course lp stability implies asymptotic stability because the convergence of the series X] \\y(j, nO: yo) p will imply that y(j, no, yo) tends to zero. The converse is not true. Exponential stability, however, is sufficient for lp stability as the following proves. Theorem 4.1.1 If the solution y — 0 is exponentially stable, then it is also lp- stable. Proof.
By definition we have: \\yn\\ < a\\yo\\rrno
with 0 < r; < 1 and a > 0. Therefore
£
Copyright © 2002 Marcel Dekker, Inc.
p
OO
\ £
4.2.
4.2
THE LINEAR CASE
107
The Linear Case
We shall discuss two results that characterize both uniform stability and uniform asymptotic stability in terms of the fundamental matrix. Consider yn+i = A(n)yn,
yno = 2/o,
(4-5)
where A(n] is an s x s matrix. Theorem 4.2.1 Let $(n, no) be the fundamental matrix of (4-5). Then the solution y = 0 is uniformly stable iff there exists an M > 0 such that ||$(n,n 0 )|| < M ,
forn>n0.
(4.6)
Proof. The sufficiency follows from the fact that yn = <£(n,no)yo5 hence we have
Hence ||yn|| < e if |y0|| < To prove necessity, if there is uniform stability, then
for \\yo\\ < d. Taking XQ = yo/ \\yo\\, the previous formula shows that sup ||$(n,n 0 )£o||
(4-7)
is bounded. But (4.7) is just the definition of the norm of $(n, no) (see Appendix A). D Theorem 4.2.2 The solution y — 0 of (4-5) is uniformly asymptotically stable iff there exist two positive numbers a, 77 with 77 < 1 such that
Proof. The proof of the sufficiency is as simple as before. The necessity follows by considering that if there is uniform asymptotic stability, then by fixing e > 0, there exists 5 > 0, N(e) > 0 such that for y^ G B§,
for n > no 4- N ( e } . As before, it is easy to see that
Copyright © 2002 Marcel Dekker, Inc.
108
CHAPTER 4. STABILITY
THEORY
for n > no + N ( e ] , where this time 771 can be chosen arbitrarily small. Moreover, because the uniform asymptotic stability implies the uniform stability, we obtain the result that ||<3>(n, no)|| is bounded by a positive number a\ for all n > HQ. We then have, for n G [no 4- mN(e), n0 + (rri + l)JV(e)],
\r(e\
with r] — r ^ ( f } < 1 and a — ainf 1 , and this proves the theorem.
D
As a result of the above theorem, we see that for linear systems uniform asymptotic stability is equivalent to exponential stability.
4.3
Autonomous Linear Systems
In this section we shall be concerned with linear autonomous equations because of their importance in applications. Of course, the results of Theorems 4.2.1 and 4.2.2 hold true, but we can give more explicit results here. The solution of the homogeneous autonomous equation
yn+i = Ayn,
yno = yo
(4.8)
is given by yn = An^yQ.
(4.9)
From the matrix theory (see Appendix A), we know that
A
r
mit — l / \
x l A " - k=l E E m r ( - Wzki i=0
(4.10)
where r is the number of distinct eigenvalues of A, m\~ is the multiplicity of A/c and Z^\ are component matrices of A. From (4.10) it follows that if the eigenvalues of A arc in the unit disk, then lirnn^oo An~n() — 0 and vice versa. This leads to the following result. Theorem 4.3.1 The solution y = 0 of (4-8) is asymptotically stable iff the eigenvalues of the matrix A are inside the unit disk. We recall that an eigenvalue A^ is said to be sernisimple if (A — \kI)Zki — 0 (see Appendix A). Theorem 4.3.2 The solution y = 0 of (4-8) is stable iff the eigenvalues of A have modulus less or equal to one and those of modulus one are sernisimple.
Copyright © 2002 Marcel Dekker, Inc.
4.3. AUTONOMOUS LINEAR SYSTEMS
109
Proof. From (4.10) it is easy to see that for semisimple eigenvalues the term containing q is (fy =1, which does not grow for q —> oo. D If the matrix A is a companion matrix, then it is known (see Appendix A) that there are no semisimple eigenvalues that are not simple. In this case Theorem 4.3.2 assumes the following form. Theorem 4.3.3 If A is a companion matrix, the solution y = 0 is stable iff the eigenvalues of A have modulus less or equal to 1 and those of modulus 1 are simple. Example 10 The trivial equation 2/n+i = Iyn,
(4.11)
where I is the s x s unit matrix is an example of a matrix with a multiple eigenvalue that is 1, but it is semisimple and the zero solution is stable. Let us now consider the nonhomogeneous equation yn+l = Ayn + b,
(4.12)
where A is an s x s matrix and b a nonnegative vector. The critical point y is given by the solution of the equation y = Ay + b.
(4.13)
For such difference equations, there is a relation between the existence of nonnegative solutions of (4.13) and the stability behavior. For the notion used in the next two theorems see Appendix A. Theorem 4.3.4 // A > 0, b > 0 and p(A) < 1, where p(A) is the spectral radius of A, then (4-13) has a nonnegative solution. Proof. Since p(A) < 1, (I — A)~l exists and is given by (/ — A)~l — X]^o ^ fr°m which we obtain y = Xli^o Alb, which is nonnegative. By assumption on p(A), we also see that y is asymptotically stable. D Under stronger assumptions on b, there is a converse of the above theorem. Theorem 4.3.5 Suppose that b is positive and A > 0. Then, if (4-13) has a positive solution y, we have p(A) < 1. Proof. By the Perron-Frobenius theorem the matrix AT has a real eigenvalue equal to p(A), to which corresponds a nonnegative eigenvector UQ such that ATUQ = p(A)uQ. Multiplying the transpose of the relation (4.13) by UQ we get [1 — p(A)]yTUQ = bTUQ from which (since both yTuo and bTu0 are positive) one gets p(A] < '1. D The foregoing results have important applications in the study of iterative methods for linear systems of equations.
Copyright © 2002 Marcel Dekker, Inc.
110
4.4
CHAPTER 4. STABILITY
THEORY
Linear Equations with Periodic Coefficients
The results obtained in the previous section cannot be extended to nonautonomous equations, as the following example shows. Consider the equation yn+l = A(n)yn,
(4.14)
where (n) - g The eigenvalues of A(n] are ±2~ ! / 2 for all n and they are inside the unit disk, but this is not enough to ensure even the stability of the null solution. With no — 0, the fundamental matrix is »-2n
V
'n
°
Q \
29n
)
(4'16)
if n is even and *(^0)=(2_°2n
2 Q
1
(4.17)
if n is odd. In any case, this is a solution that will grow exponentially away from the origin. Consequently there must be an additional condition on A(n) in order to get stability. There is an intermediate case, however, that we can treat, namely the case of linear equations with periodic matrix A(n). Equation (3.49) shows the central role played by the matrix U = &(N, 0) in such a case. Any solution of the equation will have the form yn+jN = ^(n,0)UjyQ
(4.18)
for 0 < n < N. The behavior of the solution will then be dictated by the behavior of U^yo. This leads to the following theorem, which is analogous to Theorem 4.3.1.
Theorem 4.4.1 The zero solution of the equation yn+i — A(n]yn, where A(n) is periodic of period N, is asymptotically stable if the eigenvalues of the matrix U are inside the unit disk. When'only semisimple eigenvalues are on the boundary of the unit disk, then the solution is stable. In the other cases there is instability. The similarity of the results in the two cases of autonomous and periodic equations suggests a deeper connection between them, as the following theorem shows.
Copyright © 2002 Marcel Dekker, Inc.
4.4. LINEAR EQUATIONS WITH PERIODIC COEFFICIENTS
111
Theorem 4.4.2 // A(i) is, for all i, nonsingular and periodic, then it is possible to transform the periodic system into an autonomous one. Proof.
By hypothesis, the matrix N-l
U = $(N,Q) = fl A(i)
(4.19)
i=0
is nonsingular. It is possible to define (see Appendix A) the matrix C such that C = U1/N. (4.20) Since <&(n + N, N) — $(n, 0), it turns out that the matrix P(n) = $(n,0)C- n is periodic. In fact, one has P(n + N) = $(n + 7V, Q)C~NC~n = $(n + N, 0)$(0, N)C~n = P(n). Using such a matrix to define the new variable xn = p-l(n)yn,
(4.21)
we have xn+\ = P~l(n + \}A(n)P(n}xn. Since p-1(n+l)A(n}P(n) = C, we get
xn+i = Cxn, which proves the theorem.
(4.22)
D
The solutions having the eigenvectors of C (or U) as initial values have the property xn = iinv, where v is an eigenvector and \JL the corresponding eigenvalue. But JJL — pl/N where p is the eigenvalue of U. The corresponding solution of the original equation is yn = $(n,OK (4.23) and this agrees with what was stated in Chapter 3. The solutions in the form of (4.23) are said to be Floquet solutions in analogy with the terminology used in the continuous case.
Copyright © 2002 Marcel Dekker, Inc.
112
CHAPTER 4. STABILITY
4.5
THEORY
Use of the Comparison Principle
Most of the results on the qualitative behavior of the solutions of difference equations can be obtained using the comparison theorems stated in Section 1.8. This theory is parallel to the corresponding theory of differential equations. Theorem 4.5.1 Let g(n,u) be a nonnegative function non decreasing in u. Suppose that (1) / : N+Q xBp^Bp, (2) f(n, 0) = 0,
(3)
p> 0,
g(n, 0) = 0 and
\\f(n.y)\\<9M\y\\)Then the stability of the trivial solution of the equation un+i = g(n,un)
(4.24)
implies the stability of the trivial solution of (4-1)Proof.
From (4.1) we have \\yn+i\\<\\f(n,yn)\\
and hence the comparison equation is (4.24). Theorem 1.6.1 can be applied (provided that ||yo|| < UQ) to get \\yn\\ < un for n > HQ. Suppose now that the zero solution of (4.24) is stable. Then for e > 0, there exists a <5(e.no) such that for UQ\ < 5 we have \un\ < €. This means the trivial solution of (4.1) is stable. D The assumption (3) can be replaced by (4)
/ ( n , y ) | | < y +w(n,
where g(n,u) = u + w(n,u) is nondecreasing in u. We note that w in (4) needs not be positive, which could be useful in some situations. The assumption (4) is analogous to the condition when we utilize the Liapunov functions as a measure rather then the norm. The conclusion of Theorem 4.5.1, with this change, remains the same. This version of Theorem 4.5.1 is more useful, and we shall denote it by 4.5.1*. Of course, the proof needs minor modifications. From Theorems 4.5.1 and 4.5.1*, we can easily obtain several important variants. Theorem 4.5.2 Let
Copyright © 2002 Marcel Dekker, Inc.
4.5. USE OF THE COMPARISON PRINCIPLE
113
(i) 3>(n, no) be the fundamental matrix of the linear equation zn+i = A(n}zn;
(4.25)
(ii) F : N+Q x Hs -> 1RS, F(n, 0) = 0 and
$(n,n 0 )|/n)||<5(^l|yn||),
(4.26)
where the function g(n,u) is nondecreasing in u; (iii) the solutions un of un+i = un + g(n, un)
(4.27)
are bounded for n > noThen the stability properties of the linear equation (4-25) imply the corresponding stability properties of the null solution of xn+i = A(n)xn + F(n, xn). Proof.
(4.28)
The linear transformation xn — 3>(n,no)j/ n reduces (4.28) to 2/n+i = yn + $~l(n+ l,n 0 )F(n, $(n,n 0 )y n )-
We then have Il2/n+l II < ||2/n|| + S ( n , ||2/n||).
(4.29)
If Ill/oil < uo we obtain ||yn|| < u n , where un is the solution of un+\ = un + g(n,un). It then follows that
If the solution of the linear system is, for example, uniformly asymptotically stable, then from Theorem 4.2.2 we see that ||$(n,no)|| < ar]n~no for some suitable a > 0 and 0 < r? < 1. Then ll-r < "'/ rvnn~n°n ||^n||II _^ «n
and this shows that the solution x = 0 is uniformly asymptotically stable because un is bounded. The proof of the other cases is similar. D We shall state another important variant of Theorem 4.5.1* which is widely used in numerical analysis. Theorem 4.5.3 Given the difference
equation
yn+i =yn + hA(n}yn + /(n, yn): where h is a positive constant, suppose that
Copyright © 2002 Marcel Dekker, Inc.
(4.30)
114
CHAPTER 4. STABILITY
THEORY
(1) /(n, 0) = 0 for n > ?7o and
(2) ||/(n,y)|| < g(n, \\y\\) with g(n,u] nondecreasing in u and g(n, 0) = 0. Then the stability properties of the null solution of un+i = \\I + hA(n)\\un + g(n, un}
(4.31)
imply the corresponding stability properties of the null solution of (4-30). In this form the foregoing theorem is used in the estimation of the growth of errors in numerical methods for differential equations. Instead of (4.31), one usually uses the comparison equation un+l = (1 + h\\A(n}\\}un +flf(n,u n ),
(4.32)
which is less useful because (1 + /i||A(n)||) > 1. The form (4.31) is more interesting because when the eigenvalues of A have all negative real parts, \\I + /i^4(n)|| can be less than 1. This will happen, for example, if the loarithmic norm
is less than zero. From the definition it follows that ||/ + hA\\ = I + h^i(A(n}} + O(h2A(n}}. Lettin
the comparison equation becomes un+i = (1 + fi}un + g(n, un}.
(4.33)
The next theorem requires essentially a condition on the variation of /, but it does not require the a-priori knowledge of the existence of the critical point. Let us consider xn+i - f ( x n } . (4.34) Theorem 4.5.4 Suppose that (i) / is continuous in Nno x D and g is a positive function nondecreasing 'with respect to its arguments.: defined on J\ x J^ x J$, where J{ are subsets of IR+ containing the origin:
Copyright © 2002 Marcel Dekker, Inc.
4.5. USE OF THE COMPARISON PRINCIPLE
11 5
(ii) for XQ 6 D, the sequence xn is contained in D; (iii) such sequence satisfies \\xn+2 - Xn+iH < g(\\xn+i - xn||, ||xn+i - x0||, ||xn - x 0 ||);
(4.35)
(iv) the comparison equation I
n
n— 1
\
un+i = g I u n , J^ Uj, ^ Uj V j=o j=o /
(4.36)
has an exponentially stable fixed point at the origin. Then (4-34) has a fixed point that is asymptotically stable. Proof.
Let yn = ||xn-i-i — £n||- Then we have H X J+1 ~ X3\
~ Zo|| <
3=0
j=0
Since g is nondecreasing, it follows that /
n
n-1
yn+i
j=0
j=0
By Theorem 1.6.5 we then obtain
yn < w n ,
where un is the solution of (4.36), provided that T/Q < ^o- If the origin is exponentially stable for (4.36), it follows that for suitable UQ the sequence un will tend to zero and the same will happen to yn — ||xn+i — x n ||. Moreover for all p > 0
— / ^ Hn+j •
Now by exponential stability of the origin of (4.36) and by Theorem 4.1.1 it follows that it is also /i-stable. The series Y^V] 'ls convergent, and then, for suitable n, Y^=i Vn+j can be taken arbitrarily small, showing that Xk is a Cauchy sequence. D
Copyright © 2002 Marcel Dekker, Inc.
116
CHAPTER 4. STABILITY THEORY
4.6
Variation of Constants
Consider the equation yn+i = A(n}yn + /(n, y n ] ,
yno = 2/o,
(4.37)
where A(n] is an s x s nonsingular matrix and / : 7V+0 x IRS —>• ]RlS.
Theorem 4.6.1 77/.e solution y(n,no.yQ} of (4-37) satisfies the equation n-\
yn = $(n. n0)yno + £ $(n, j + l)/(j, y,),
(4.38)
j = "0
where $ (n.no) satisfy the matrix equation $(n + I . n 0 ) = yl(n)$(n,n, 0 ).
(4.39)
Proof. Let y(n.n0,y0) = $(ri, n 0 )c n ,
cno = ?/o-
(4.40)
Then substituting in (4.37), we get $(rz + 1. 7? 0 )c n+ i = A(n}$>(n, n 0 )c n + /(n, y n ) from which we see that
and
From (4.40) it follows that
v ' <• •> "'U y yu '
/ j
^ \' *i j
Consider now the equation xn+i = / ( n , x n ) , where / : /V+ x ]RS -* IR'S.
Copyright © 2002 Marcel Dekker, Inc.
(4.42)
4.6. VARIATION OF CONSTANTS
117
Lemma 4.6.1 Assume that f : N+0 x ]RS —» ]RS and f possesses partial derivatives on N+0 x IRS. Let the solution xn = ;r(n,no,£o) of (4-4%) exist for n > no and let rrf ^ df(n,x(n,no,xo}} //(n,n 0 ,z 0 ) = —--^— --.
(4.43)
Then
*.( , , $(n, n 0 , XQ)\ = —^-r '-
(4.44)
OXQ
exists and is the solution of $(n + 1, n 0 , XQ} - H(n, n 0 , x 0 )$(n, n0, z 0 ),
(4.45)
$(n 0 ,n 0 ,o;o) = /,
(4.46)
o/^en called the variational equation. Proof.
By differentiating (4.42) with respect to XQ we have <9zn+i
df dxn
Then (4.45) follows from the definition of 3>. D We are now able to generalize Theorem 4.6.1 to the equation yn+l = /(n, yn} + F(n, yn}.
(4.47)
Theorem 4.6.2 Let /, F : JV+o x IRS -> Hs, and Je* 5//5x exzs^ and be continuous and invertible on N^Q x H5. If x(n,no,xo} is the solution of /(n,x n ),
x no = x 0 ,
any solution of (4-47) satisfies the equation
=x
•0(n, no. i>j, I'j+i) — / $(^, no, svj+ij + (1 — s}vj}ds Jo and Vj satisfies the implicit equation (4-50).
Copyright © 2002 Marcel Dekker, Inc.
(4.48)
118
CHAPTER 4. STABILITY THEORY
Proof.
Let us put ?/(n,no,xo) = x(n,no,v n ) and ^o = XQ. Then
y(n 4- l,no,xo)
=
x(ra + 1, n 0 , vn+\) - x(n + 1, no, vn) + x(n + 1, no, vn)
=
/(n, z(n, n0, v n )) + F(n, x(ra, n 0 , vn}}
from which we get x(n 4- 1, n 0 , v n +i) - ^( n + 1, no, vn) = F(n, yn}Applying the mean value theorem we have fl dx(n+ I,n 0 ,sv n + i + (1 - s)vn) . . , . \ —--«-—--—-ds(vn+i - vn] = F(n, yn) Jo OXQ and hence by (4.44)
o which is equivalent to ^(n + I,n 0 ,v n , Un+i)(^n+i - ^;n) = F(n,yn}.
(4.50)
It now follows that
and
n-l
vn = v0+ Y^ ^~~l(J + l,no>vj,vj+i}F(j,yj}
(4.51)
j=n0
from which the conclusion results.
D
Corollary 4.6.1 Under the hypothesis of Theorem 4-6.2, the solution y(n, no, can be written in the following form: y(n,no,xo)
=
x(n,no, XQ) + i}>(n,nQ, v n , XQ) n-l
Proof.
Apply the mean value theorem once more to (4.49).
Corollary 4.6.2 7//(n,x) = >l(n)x, i/ien ^.5^j reduces to (4-38). Proof.
In this case xn = <E>(n, no)xo,
Copyright © 2002 Marcel Dekker, Inc.
4.7. STABILITY BY FIRST APPROXIMATION
119
and therefore we have vn+i -vn = i/J~l(n + l,rao,v
and n-l
vn = yo + ]T] ^~l(J
+
!' n O'
j-no
from which the claim follows.
4.7
D
Stability by First Approximation
Consider the equation yn+i = A(n]yn + f ( n , yn),
(4.53)
where y 6 IRS, A(ri) is an s x s matrix, / : ./Vno x Ba —> Ba and /(n, 0) = 0. When / is small in the sense to be specified, one can consider (4.53) as a perturbation of the equation xn+i = A(n)xn.
(4.54)
The question arises whether the properties of stability of (4.54) are preserved for (4.53). The following theorems offer an answer to such a question. Theorem 4.7.1 Assume that \,
(4-55)
where gn are positive and Y^<^=n09n < °°- Then if the zero solution of (4-54) is uniformly stable (or uniformly asymptotically stable), then the zero solution of (4-53) is uniformly stable (or uniformly asymptotically stable). Proof.
By (4.38) we get n-l
j=n 0
Because of Theorem 4.2.1 we have, using (4.55), n-l
||yn|| <M||yo|| + M £
9j\\yj\\.
j=n0
Corollary 1.6.2 yields /
n-l
\
||yn|| < M||yo|| exp M £ # ,
Copyright © 2002 Marcel Dekker, Inc.
120
CHAPTER 4. STABILITY
THEORY
from which follows the proof, provided that ||yo|| is small enough such that M ||yo|| e x p ( A f £ ~ n o ^ ) < ^ . In the case of uniform asymptotic stability, it follows that for n > N ( e ) , || $(71,710)2/0 1| < e, for every e > 0. The previous inequality can be written, for such values of n, /
00
\
\\yn\\ < eexp M E \
9j
J=no
from which we conclude that lim?;n = 0.
)
D
Corollary 4.7.1 // the matrix A is constant such that the solutions of (4-54) are bounded, then the solutions of the equation yn+l = (A + B(n))yn
(4.56)
are bounded provided that oo
(4-57)
E ii£fc)H < °°n—no Theorem 4.7.2 Assume that |,
(4.58)
where L > 0 sufficiently small and the solution xn — 0 of (4-54) is uniformly asymptotically stable. Then the solution yn = 0 of (4-53) is exponentially asymptotically stable. Proof.
By using the result of Theorem 4.2.2, we have ||$(n,n 0 )|| <#T7 n ~ n °,
#>0,
0<77<1,
and because of (4.58), we get 1
^ ^ \Vi\\3 = n0
By introducing the new variable pn = r)~n\\yn\\, we see that n-l no
Pn
l
\\yo\\+LHrr
^ Pij=no
Using Corollary 1.6.2 again, we arrive at n-l Pn
Copyright © 2002 Marcel Dekker, Inc.
<
#TT
4.8. LIAPUNOV FUNCTIONS
121
which implies \\yn\\
If 77 + LH < 1, that is L < ^-jj , the conclusion follows.
(4.59) D
The following particular case is more often used in the applications. Corollary 4.7.2 (Perron). Consider the equation yn+1 =Ayn + f(n,yn),
(4.60)
where A has all the eigenvalues inside the unit disk and moreover
»-°
IMI
uniformly with respect to n. Then the zero solution of (4-60) is exponentially asymptotically stable. We can similarly prove the following result. Theorem 4.7.3 Assume that the zero solution of — •**••£ n -i
where A is an s x s matrix, is asymptotically stable. If 00
E
)|| <+oo,
then the zero solution of (4-56) is asymptotically stable.
4.8
Liapunov Functions
The most powerful method for studying the stability properties of a critical point is Liapunov's second method. It consists of the use of an auxiliary function, which generalizes the role of the energy in mechanical systems. For differential systems the method has been used since 1892, while its use is much more recent for difference equations. In order to characterize such auxiliary functions, we need to introduce a special class of functions. Definition 4.8.1 A function 0 is said to be of class K if it is continuous in [0,a), strictly increasing, and 0(0) = 0. It is easy to check that the product of any two functions of class K is in the same class and that the inverse of such a function is in the same class. Let V(n, x] be a function defined on N+0 x Ba, which assumes values in IR+.
Copyright © 2002 Marcel Dekker, Inc.
122
CHAPTER 4. STABILITY
THEORY
Definition 4.8.2 The function V(n, x) is positive definite (or negative definite) if there exists a function £ K such that
(or V(n,x) < -^(||x||))
for all (n,or) G A+ x B0. Definition 4.8.3 A function V(n,x) > 0 is said to be decrescent if there exists G K such that for all (n,x) € Ar+0 x £0. Let us consider the equation yn+i = /(n,y n ),
(4.62)
where / : N.^Q x Ba —> Ba.f(n,Q) — 0 and f(n.x) is continuous in x. Let y(n.nQ.yo) be the solution of (4.62), having (no,j/o) as the initial condition and defined for n e "•0 . We shall now consider the variation of the function V along the solutions of (4.62) / n ) = y(n + l,j/ n + 1 ) - V(n,2/ n ). (4.63) If there is a function u : N+ x IR —> IR such that I i(J
y n )
<
V(n, yn] + u(n, V(n, yn}}
= 9(n,V(n,yn))
(4.64)
to which we shall associate the comparison equation un+i = g(n, un} = un + u;(n, un).
(4.65)
The auxiliary functions V(n, x) are called Liapunov functions. In the following, we shall always assume that such functions are continuous with respect to the second argument. Theorem 4.8.1 Suppose there exist two functions V ( n , x ) and g(n,u) satisfying the following conditions: (1) g : N£O x IR+ —>• IR + , ^(n,0) — 0 7 g(n.u) is nondecreasing in u; (2) V : N*0 x Ba —> ]R+ , 1/(n,0) = 0 and V(n,x) is positive definite and continuous with respect to the second argument:
Copyright © 2002 Marcel Dekker, Inc.
4.8. LIAPUNOV FUNCTIONS
123
(3) V satisfies (4.64). Then (a) the stability of un — 0 for (4-65) implies the stability of yn = 0; (b) the asymptotic stability of un = 0 implies the asymptotic stability of yn - 0. Proof.
By Theorem 1.6.1, we know that V(n, yn)
n e 7V+o,
provided that V(no,?/o) < UQ. From the hypothesis of positive defmiteness we obtain for a 0 e K, 4>(\\yn\\}
from which we get Ill/nil < e-
(4-66)
By using the hypothesis of continuity of V with respect to the second argument, it is possible to find a £(e, no) such that \\yno\\ < ^(e, no) will imply V(n0,yno} < UQ. In the case of asymptotic stability, from (t>(\\yn\\)00 0(||y n ||) = 0 and consequently lim n _ >00 7/ n = 0.
D
Corollary 4.8.1 If there exists a positive definite function V(n, x) such that on N+Q xBa, V is continuous with respect to x, and moreover AV(n, yn] < 0, then the zero solution of (4-62) is stable. Proof. In this case u(n,u) = 0, and the comparison equation reduces to un+i — un, which has stable zero solution. D Theorem 4.8.2 Assume that there exist two functions V(n,x] and g(n, u] satisfying conditions (1), (2), and (3) of the previous theorem and moreover suppose that V is decrescent. Then (a) uniform stability of un — 0 implies uniform stability of yn = 0;
Copyright © 2002 Marcel Dekker, Inc.
124
CHAPTER 4. STABILITY THEORY
(b) uniform asymptotic stability of un = 0 implies uniform asymptotic stability ofyn = 0. Proof. The proof proceeds as in the previous case except that we need to show that £(e,no) can be chosen independent of no- This can be done by using the hypothesis that V(n, x] is decrescent because in this case there exists a ^ € K such that V(n, yn] < ^(\\yn\\). In fact, as before, we have 4 > ( \ \ y n \ \ } < V ( n , y n ] n 0 . The uniform asymptotic stability follows similarly. D Corollary 4.8.2 // there exists a positive definite and decrescent function V such that then yn = 0 is uniformly stable. Corollary 4.8.3 // there exists a function V such that
and &V(n,yn)<-v(\\yn\\), where 4>. ^, v 6 K then yn = 0 is uniformly asymptotically stable. Proof. Clearly ||yn|| = 0 is uniformly stable. Take e = p and designate by SQ — 5(p) so that we have \\yno\\ < $o implies \\yn\ < p, n > no- To get uniform asymptotic stability, let 0 < e < p be given and let 5(e) > 0 to be the number we get by uniform stability. Choose an integer
Then, it is enough to show that there exists an n* such that no < n* < TT-O + NQ(C) and \\yn*\\ < £(e). If this is not true, it follows that \\yn\\ > <5(e), for all n £ [HQ, no + No(e)}. As a result, we obtain 0 < V(n + l,2/ n +i) < ^(n,y n )-7(||2/ n ||) < F(n 0 ,y n o )-n7(e) < n(60)-ni(e), for all n G [no,^o + ^ol 6 )]- This leads to a contradiction because of the definition of NQ(C). Hence there exists an n* such that | yn*\ < ^( e ) 5 which implies ||yn|| < t f°r n ^ n*• In other words, \\yn\\ < e for n > HQ + A^o(e), and the proof is complete. D
Copyright © 2002 Marcel Dekker, Inc.
4.8. LIAPUNOV FUNCTIONS
125
With minor changes, one can show that the condition on A^ can be substituted with AV r (n,?/ n ) < — z/(||?/ n +i||). If in Theorem 4.8.2 the condition that V is decrescent is removed, the asymptotic stability will remain. Theorem 4.8.3 Assume that there exists a function V such that (1) V : N+0 x Ba —>• IR+; V(N, 0) = 0; V is positive definite and continuous with respect to the, second argument; (2)
Then the origin is asymptotically stable for (4-62). Proof. By Theorem 4.8.1 we know that the origin is stable. Suppose it is not asymptotically stable, then there exists a solution y ( n , n 0 , y o ) and an infinite subset Jno C N+o such that ||?/(n, no, yo)|| > e > 0 for n 6 J no . Let be n — min Jno
kn — cardjj e Jno : n < j < n}.
we then have V(n+ l , 2 / ( n + I,n 0 ,yo)) < V^(n,yn) - fc n /^(e), if n G J no . Summing, we have V(n + 1, y(n + 1, n 0 , yo)) < ^(n 0 , y no ) - ne,
where n can be arbitrarily large in J no . Taking the limit over all n £ J no . we get
lim V(n,yn) = -oo,
n —>oo
which contradicts the hypothesis that V is positive definite.
D
The next theorem concerns the lp stability and is related to the previous one. Theorem 4.8.4 Assume that there exists a function V such that (1) V : N+0 x Ba —> IR + ;y(n,0) — 0; V(n,y) positive definite and continuous with respect to the second argument: (2)
where p,c are positive constant. Then yn = 0 is lp stable.
Copyright © 2002 Marcel Dekker, Inc.
126
CHAPTER 4. STABILITY
THEORY
Proof. By Theorem 4.8.3 we know that yn = 0 is asymptotically stable, that means that for n > no, e > 0, there exists £(e, no) such that for \\yno\\ < 6, \\yn\\ < £• Let us define the function n-l
j=n0
Then
Therefore G(n) < G(UQ) = V(riQ,ynQ) for n > no and n-l
0 < G(n] = V(n,yn) + c ^ from which it follows n— 1
i
and
The next theorem is a generalization of the previous one. It is the discrete analog of LaSalle's invariancc principle. Let us consider the solution y(ri,no,yno} of 2/n+i = /(n,2/ n ),
yn0 = 2/0,
(4.67)
and suppose that it is continuous with respect to the initial vector yo. Theorem 4.8.5 Suppose that, for y € D C IRS, (1) i/iCTe exisi iwo reo,/ valued functions V(n,y), oj(y) > 0, 6ot/i continuous in y, with V(n,y] bounded below such that . yn) < -u(yn)
for
n > n0;
(4.68)
(2) y(n, n 0 , ;y no ) ^ D /or n > n 0 .
Then either y(n,riQ,yno] is unbounded or it approaches the set E = {x G D\u(x) = 0}.
Copyright © 2002 Marcel Dekker, Inc.
(4.69)
4.8. LIAPUNOV FUNCTIONS
127
Proof. By assumption, yn E D for n > HQ and V(n,yn) is decreasing along it. Because V is bounded below, it must approach a limit for n —> oo and u(yn) must approach zero. Then either the limit is finite and must lie in E, or it is infinite. D Corollary 4.8.4 Suppose that u(x) and v(x) are continuous real valued functions such that u(x] < V(n,x) HQ. Fixing r/, consider the sets D(rj} = [x\u(x) < 77}, 1)1(77) = {x\v(x) < 77}. Under the hypothesis of Theorem 4-8.5 with D = D(r/), all solutions that start in D\(ri) remain in D(r/) and approach E for n —>• oo. Proof.
Let yo E DI(TJ), then u(yn] < V(n,yn) < V(n0,y0} < v(y0) < J]
showing that u(yn] < 77 for n > HQ.
n
Example 11 Consider the equation yn+i = M(n,yn)yn,
(4.71)
where M is an s x s matrix. Define V(n, yn] as V(n,yn} = \\yn\\.
(4.72)
Then AV - \\M(n,yn}yn\\ - \\yn\\ < (\\M(n,yn}\\ - l)||yn||. Let u(yn) = v(yn) = V(n,yn). Then D(rf) = D^} = {y\\\y\\ < r,}. For all y € D, let ||M(n,j/)|| < a(y) and u(y) - (1 - a(y))\\y\\. It then follows that If a(y) < 1, for y e D(ri), uj(y} is positive. The set E is the origin and possibly something on the boundary of D(rj). Because V is decreasing on D(rj), it follows that this last possibility cannot occur. Then the solution starting in D(rf] cannot leave this set, and it will tend to the origin. ^(77) is a domain of asymptotic stability of the origin. Different choices of 77 and of the norm used will give different domains of asymptotic stability. Of course, the union of all these domains is still a domain of asymptotic stability. If M is independent of n with a spectral radius less than one, then it is possible to choose a vector norm such that c*(0) < 1 with a(x) continuous. The previous result shows that it is possible to define a nonempty domain of asymptotic stability.
Copyright © 2002 Marcel Dekker, Inc.
128
CHAPTER 4. STABILITY
THEORY
Definition 4.8.4 The positive limit set Q(yno) of a sequence yn,n G is the set of all the limit points of the sequence. That is, y G Q,(yno) if there exists an unbounded subset Jno C N+0 such that yni —>• y for nl G Jno. Definition 4.8.5 A set S C IRS is said to be invariant if for yo G S it follows that y(n.nQ.yo) G S. In the case of the autonomous difference equation yn+i = /(y n ),
(4.73)
where / is a continuous vector valued function with /(O) = 0, Theorem 4.8.5 assumes the following form. Theorem. 4.8.6 Suppose that for y G D C IR'S, (1) there exist two real valued functions V(y], u(y) both continuous in y with V bounded below, and
(2) yn G D. forn > n0. Then either yn is unbounded or approaches the maximum invariant set M contained m E . Proof. Same as before and moreover the positive limit set of any bounded solution of an autonomous equation is nonempty, invariant and compact (see Problem 4.14). D Corollary 4.8.4 can be rewritten as follows. Corollary 4.8.5 // in Theorem 4-8.6 the set D is of the form, = {x\V(x) < i!} for some r; > 0 7 then all the solutions that start in D(r/) remain in it and approach M as n —>• oo. The next result, again for autonomous equations, imposes conditions on the second differences of the Liapunov function. Theorem 4.8.7 Suppose that V : IRA> —>• IR is a continuous function with ^2V(lJn} > 0 foryn 7^ 0. Then for any yo G IR'S7 either y ( n . y o ) is unbounded or it tends to zero for n —>• oo. Likewise if /\2V(yn] < 0, yn ^ 0.
Copyright © 2002 Marcel Dekker, Inc.
4.9. DOMAIN OF ASYMPTOTIC STABILITY Proof.
129
Put A 2 F > 0 in the form V(yn+2) - V(yn+l) > V(yn+l] - V(yn}.
If there exists a k £ N+Q such that V(yk+\) — V(yk) > 0, then V(yn+\) — V(yn) > 0 for n > fc. Otherwise, we get V(yn+i) < V(yn] for all n. In both cases V is a monotone function. Suppose that V is not increasing. Consider the positive limit set fi(yo) °f y( n 5^o 5 2/o)- If ^(2/0) is empty, then y(n,nQ,yo) is unbounded and the theorem is proved. If fi(yo) is not empty, V(yn) must be constant on fi(yo) because the limit of a monotone function is unique. But this is impossible because A 2 F > 0, unless f2(yo) — {0}- The other cases are proved similarly. D
4.9
Domain of Asymptotic Stability
All the previous results say that if the initial value is small enough, then the origin has some kind of stability. In applications one is more interested in the domain of asymptotic stability since one needs to know where to start with the iterations. In other words, one needs to know the domain in Rs containing the initial values, starting from which solutions will eventually tend to the fixed point. This problem is a difficult one and will also be discussed in the next chapter. We shall give some results in this direction in the case of autonomous difference equations yn+i = f ( y n ) ,
(4.74)
where / <E C[B 0 ,IR S ] and /(O) - 0. Theorem 4.9.1 Suppose that there exists a continuous function V : Ba —»• M + , V(0) = 0, V(y) ^Qfory^Q and AV(yn) < 0. Then the origin is asymptotically stable. Moreover if Ba = IRS and V(y) —> oo as y —•> oo, then the origin is globally asymptotically stable. Proof. The proof of stability follows from the same arguments used in Theorem 4.8.1 and Corollary 4.8.1, except that continuity of V is used instead of positive definiteness of V. To prove asymptotic, stability, we observe that for yn £ B€ where 0 < e < a, V(yn) is strictly decreasing and it must converge to zero. Again by continuity of V, the sequence yn itself must converge to zero. Now suppose that the last hypothesis is true. Then it is clear that we are not restricted to take \\yn\\ < a, and hence the proof is complete. D As an example, consider (4.74) when / is linear, = Ayn
Copyright © 2002 Marcel Dekker, Inc.
(4.75)
130
CHAPTER 4. STABILITY
THEORY
and let us take V(yn) = ylByn,
(4.76)
where B is a symmetric positive definite matrix. The demand that AF(y n ) < 0 becomes ylATBAyn - y^Byn < 0 that is, we must have ATBA -B = -C,
(4.77)
where C is any positive definite matrix. This leads to the following result. Corollary 4.9.1 // there is a symmetric positive definite matrix B such that (4-77) is verified, then the origin is globally asymptotically stable. The converse of this corollary is also true. Theorem 4.9.2 Suppose the origin is asymptotically stable for (4-75); then there exist two positive definite matrices B and C such that (4-77) is verified. The statement analogous to (4.77) in the continuous case is STG + GS = -Gi,
(4.78)
where G and GI are symmetric positive definite matrices and S has the eigenvalues with negative real part. Equation (4.78) is called the Liapunov matrix equation. There is, of course, a correspondence between (4.77) and (4.78). In fact by putting
S = (A + I}(A - 7)"1, (4.78) is transformed into an equation of the form (4.77) and vice versa by putting A
i T
O\ — 1 / 7"
,/i —- I J ~~~ O I
( J.
i
f\
i~ O ) •
To find the matrix B, one needs to solve the matrix equation (4.77) where A is given and C is chosen appropriately. The methods of solution of the equation (4.77) (or alternatively (4.78)) have been studied extensively in the last years. The following theorem also gives the region of the asymptotic stability for the zero solution of (4.74), and it is the discrete version of the Zubov's theorem. Theorem 4.9.3 Consider the equation (4-74), and assume that there exist two functions V and 0 satisfying the following conditions: (1) V : C[MMR+].
V(Q) = 0,
V(x) > 0
for x ^ 0,
(2) 0 : C[IR S . IR+],
0(0) = 0.
0(:r) > 0
for x ^ 0,
Copyright © 2002 Marcel Dekker, Inc.
4.9. DOMAIN OF ASYMPTOTIC STABILITY
131
(3) V(yn+1) = (1 + (yn))V(yn) - <j>(yn). Then D — {x\V(x) < 1} is the domain of asymptotic stability. Proof.
By condition (3), which can be written as
one obtains n-l
1 - V(yn) = I] (! + ^'))(1 - V(j/o)).
(4-79)
j=0
Suppose now that yo lies in the region of asymptotic stability. Then, the left-hand side tends to one by hypothesis, the right-hand side will also be convergent and will tend to C(l — V(yo}), where
3=0
It follows then that V(yo) — (C — 1)/C < 1, which shows that yo £ D. Conversely if yo £ D, then again from (4.79) we see that V(yn) will remain always outside D and V will never be zero. This implies that yn will not tend to zero. D To use this theorem, one needs to know the solution yj of (4.74) and then from (4.79) find the function V", which will define the set D. Unfortunately this can be done only for some cases. Theorems 4.8.5 and 4.8.7 and their corollaries also give the asymptotic stability domains in terms of D(rj) for the set E. When the maximum invariant set contained in E, reduces to a point, Theorem 4.8.7 gives the asymptotic stability domain for a critical point. Theorem 4.8.7 can also be used to obtain the domain of asymptotic stability. Suppose that the condition A 2 V > 0 holds true only in an open region H containing the origin. Then put Amaz = max{AK(x)|x e boundary of H }
and
for j = 0, 1, 2, . . . , where y ( j , x) = y(j, n 0 , x). Theorem 4.9.4 If the regions Ej are bounded and nonempty, then they are domains of asymptotic stability for (4-74).
Copyright © 2002 Marcel Dekker, Inc.
132
CHAPTER 4. STABILITY
THEORY
Proof. If Ej is not empty and x E Ej, y(j + k.x) E H since AV is not decreasing along any trajectory in Ej. Moreover &V(y(j + k + 1, x)) - AV(y(j + k, x)) = A2 V (y(j + /c, x)) > 0 and
fc, x)) > AV(y(j, x)) > A max . This means that y(j -i- k.x) E E1, for all fc, from which we see that y(n. x) is bounded. By Theorem 4.8.7 it follows that y(n, x) —+ 0. D 2 Similar results hold in the case where A V^ < 0 in H and the reions ' ) ) / ^ '-^mini
Amin = min{Al/(x)|x E boundary of H}. As an example, we consider the following system arising in biomathematics Example 12 Xn-\-[
—
J/ni
Vn+\
~ tt-En ~~ yni
.2
i „ -"
Consider
One obtains A 2 F(x,y) = •
J
2 2 (1
- 3a2 + 4 a x - 2?/ 2 ) + (ay- (ax - y 2 ) 2 ) 2 .
Here // - {(x, y)|l - 3a2 + 4ax - 2y2 > 0}
contains the origin. For a = 1/4. one shows that A max = —0.0364 and then IN
* 47
4.10
2
^ 17
^ > -0.0364 i . 17
Converse Theorems
This section will be devoted to the construction of Liapunov functions when certain stability properties hold. In this construction of Liapunov functions, however, one uses the solutions of the problem, and this implies that this construction is of little use in practice. The real importance of converse theorems lies therefore in the fact that by means of such theorems it is possible to prove some results on total stability, which is a very important concept in applications. For this reason we shall present only those results that we shall use later.
Copyright © 2002 Marcel Dekker, Inc.
4.10.
CONVERSE THEOREMS
133
Theorem 4.10.1 Suppose that the zero solution of (4-62) is uniformly stable. Then there exists a function V, positive definite and decrescent, such that AT/ < 0 along the solutions. Proof.
Consider the function (4.80) k>n
As usual, yn = y(n, no, yo). From (4.80) it is immediate that V(n. yn] > \\yn\\ showing that V is positive definite. From the definition of uniform stability we know that \\y(k,n, yn}\\ < e if \\y\\ < <5(e). Without loss of generality, we shall suppose that 6 e K (See Problems 4.8 and 4.9). Let 0 = 5~l. Then we can write \\y(k,n,yn)\\ < <j>(\\yn\\), from which we obtain V(n, yn) < (/>(\\yn\\), showing that V is decrescent. On the other hand, for every solution one has sup||j/(fc,n,2/(n,ra 0 ,2/o))ll = sup ||y(fc,n 0 ,2/o)|| k>n
k>riQ
and
V(n + 1, j/n+i)
SU
—
P '\\y(k,nQi yo}\\ ^ sup ||j/(fc, no, ?/o)|| fc>n
fc>n+l
=
V(n,y n ),
and this completes the proof.
D
Theorem 4.10.2 Suppose that the zero solution of (4-62), where /(n,x) is locally Lipschitz around the origin, is uniformly asymptotically stable. Then there exists a function V, positive definite and decrescent, such that AF(n,y n ) < —//(||?/ n+ i||) along the solutions and V is locally Lipschitzean. Proof. Consider a function G(r) defined for r > 0, G(0) = 0, 6"(0) = 0, G'(r] > 0, G"(r) > 0 and let a > 1. Since G(r] = I du I G"(v}dv Jo Jo and
(
r
\
rr/a
- 1= / a/ Jo
ru
du
Jo
G"(v)dv,
we have, setting u = u>/a, / r \
1
fr
/"W/Q
G ( - ) = - / dw / \aj a Jo Jo
I
rr
ruj
Define V(n, yn) - V C?(||j/(n + k, n, i/n)!!)^?-
Copyright © 2002 Marcel Dekker, Inc.
1
G /7 (v)dT; < - / do; / G"(v)dv = -G(r}. a Jo Jo a
134
CHAPTER 4. STABILITY
THEORY
For k — 0, we have G(\\yn\\)(\\yn\\) and therefore (See previous theorem). Consequently, since
1 + ak -;1+ k r- < a, V(n,yn} AT(e), \\y(n+k, n, yn)\\ < e and we get k,n,yn)\
\\yr
for k > N
Thus. a
which in turn leads to < V(n,yn).
aG
This shows that it is sufficient to consider k
G(
\ 11 N JL ~~T~ K Oi.
(4.81)
0
Let ki. be the index where the sup is achieved so that 1+
(4.82)
1+
x
l+o^+l)
a -1 a
a <
Copyright © 2002 Marcel Dekker, Inc.
4.10. CONVERSE THEOREMS
135
from which V(n+l,yn+i)-V(n,yn)
< N
i + a + aN
*l
where L^o is the Lipschitz constant. The last inequality follows from \\yn+\ \\ = The function /i belongs to K because it is strictly increasing being TV (||2/ n ||/a) a decreasing function and G(0) = 0. To complete the proof, we must show that we can choose a function G such that V is a Lipschitz function. By hypothesis of uniform asymptotic stability, it follows that for r > 0 there exists a <5(r) such that for y',y" £ -B<5(r) and y(n,riQ,y'},y(n,no,y") £ Br. Because / is a Lipschitz function, one has ||3/(n,n 0 ,3/') - y(n.n^y"}\\ < L?~n°\\yf - y"\\.
We let q<mm(Lr,L-1},
q < 1,
(4.83)
and
G(r)=A
Jo
q d
S
(4.84)
j
where N(r) is the same function defined before. G(r) satisfies the condition required because N(r) is decreasing and limT._>o N ( r ) = oo. We have seen that +kl^y'}\\}~,
(4.85)
where 0 < ki < N (Ml\ . V a )
(4.86)
For simplicity, let us put r\ — \\y(n + k\,n,y'}\\ and TI = \\y(n + ki,n,y"}\\. Suppose r\ < r^. Then 0 < G(n) - G(r2) < G 7 (ri)(r! - r 2 ) < But
Copyright © 2002 Marcel Dekker, Inc.
a
( n - r 2 ).
136
CHAPTER 4. STABILITY THEORY
By substituting, we have 0 < G( n ) - G(r 2 ) < AqN^r^L^\\y' - y"\\ < A\\y' - y"\\.
(4.87)
The last inequality follows from (4.86) and (4.83). Multiplying (4.87) by (1 + aki)/(l + ki) we get
from which, because of the fact
we obtain V(n, y'} — V(n, y"} < aA\\y' — y"\\. By interchanging the roles of y' and y" one similarly gets V(n,y")-V(n,y')>-aA\\y'-y"\\. This shows that \V(n,y')-V(n,y")\ IR+ is positive definite, decrescent, and such that Proof. Let
oc
n v(n,yQ)^Y,\\y( k=Q It follows that
+k n
^ -y
n^z/n)>iwi p ,
which shows that V is positive definite. Moreover,
and
AV(n,yn)
=
which completes the proof.
Copyright © 2002 Marcel Dekker, Inc.
4.11.
4.11
TOTAL AND PRACTICAL STABILITY
137
Total and Practical Stability
Let us consider the equations 2/n+i = f ( n , yn] + R(n, yn) yn+i = f(n,yn)
(4.88) (4.89)
where R is a bounded, Lipschitz function in Ba, and J?(ra, 0) = 0. We shall consider (4.88) as a perturbation of equation (4.89). Suppose that the zero solution of (4.89) has some kind of stability property. We want to know under what conditions on R the zero solution preserves some stability. Definition 4.11.1 The solution y = 0 of (4.89) is said to be totally stable (or stable with respect to permanent perturbations) if for every e > 0, there exist two positive numbers 81 = #i(e) and 82 = bi(t) such that every solution y(n,nQ,yo) of (4.88) lies in Ba for n > no, provided that Ill/oil < S1 and
\\R(n,yn)\\<62
for
yn G B€,
n > n0.
Theorem 4.11.1 Suppose that the trivial solution of (4-89) is uniformly asymptotically stable, and moreover
\\f(n,y') -/(n,i/")ll < W-y"l
y',y" ^B,C Ba.
Then it is totally stable. Proof. Let ?/(n,no,2/o) be the solution of the unperturbed equation. The hypothesis of uniform asymptotic stability implies (see Theorem 4.10.2) that for 0 < <5o < a, there exist <5(<5o) > 0, 8(80} < a such that yn G -B<$(<$0), limyn ~ 0 and moreover there exist functions a, 6, c G K and V such that forne7V+0, (a)
a(m\)
(b) AV(n,y n )<-c(||y n + 1 ||), and
(c) \V(n,y'} - V(niy")\ < M\\y' - y"||, for yf,y" e Bs(So),M > 0. Let 0 < e < 8(80). Choose 81 > 0 and 82 > 0 such that < a(c),
Copyright © 2002 Marcel Dekker, Inc.
($2<<Ji,
^
<
-
(4.90)
138
CHAPTER 4. STABILITY THEORY
Suppose that e is sufficiently small so that L e e + $2 < 5 (80}. Let \\R(n,y} | < 82 for y E Bf. One then finds for ||y|| < e, \\y(n + l,n,y)|| — |/(n,y)| < L e e, | y ( n + l , n , y ) - y(n + l,n,y)|| - ||fl(n,y)|| < £ 2 , and ||y(n + 1, n, y)|| < Lee + 82 < 8(80). Thus, for ||y|| < e, we have V(n + l,y(n + l , n , y ) )
- ^(n,y) = F(n + l,y(n 4- l , n , y ) ) - V(n,y) + V(n + l , y ( n + l,n,y)) - F(n + l , y ( n + l , n , y
+ c^).
(4.91)
Now suppose that there is an index n\ > HQ and a y( e B$1 such that y(n\. HO, y') ^ ^e and y(n, no, y') E 5e for n < ni. It then follows that
and V(no, y') < 6((5i). Then there exists an index n^ e [no, ni — 1] such that
Vfa, y(n 2 , n 0 , y 7 )) < b(8i + 5 2 ) and V(n2 + I,y(n2 + I,n 0 ,y ; )) > K^i + ^)-
(4.92)
Thus ||y(n2 + l,no,y')|| > 81 + 82, from which we get ||y(n 2 + l , n 2 , y ( n 2 , n o , y / ) | | >
||y(n2 + 1, n 0 , y')!! -||y(n 2 + I , n 2 , y (n 2 ,n 0 , y')) -y(n 2 + I,n 2 ,y(n2,no,y'))||
> 5i + 82 — 82 = 8\ . From (4.92) and (4.91), it follows that 0 < V(n 2 + I,y(n 2 + I,n 0 ,y')) - ^(n 2 ,y(n2,no,y')) < -c(||y(n 2 + I , n 2 , y ( n 2 , n 0 , y / ) l l ) + c(<5i) < 0, which is a contradiction.
D
Corollary 4.11.1 Suppose that the hypothesis of Theorem 4.11.1 is verified and moreover for yn E Ba one has |J?(n,y n )|| < gn\ yn \ with gn —> 0 monotonically. Then the solution of the perturbed equation is uniformly asymptotically stable.
Copyright © 2002 Marcel Dekker, Inc.
4.11. TOTAL AND PRACTICAL STABILITY Proof.
139
From (4.91) one has , yn} < -c(\\y(n + 1, n, t/ n )||) 4- Mgn\\yn\\.
Suppose 0 < r < <5(<5o) and r < \\y(n + l,n, yn)|| < £(5o), by the hypothesis on g n , it can be chosen an n\ G N+0 such that for n > n\ one has ||2/n|| < 2~ 1 c(r), and then
Then apply Theorem 4.8.5. D Connected to the problem of total stability is the problem of practical stability of (4.89) and (4.88). In this case we no longer require that R(n, 0) = 0 so that (4.88) does not have the fixed point at the origin, but it is known that ||/2(n, 0)|| is bounded for all n. This kind of stability is very important in numerical analysis, where certain kinds of errors cannot be made arbitrarily small. Definition 4.11.2 The solution y — 0 of (4.89) is said to be practically stable, if there exists a neighborhood A of the origin and n > UQ such that for n > n the solution y(n, no,yo) of (4.89) remains in A. Theorem 4.11.2 Consider the equation (4-88) and suppose that in a set D C IRS the following conditions are satisfied (1) \\f(n,y)-f(n,y')\
L < 1,
(2) \\R(n,y)\\ < 5. Then the origin is practically stable for (4-89). Proof. Let yn and yn be the solution of (4.88) and (4.89) respectively. Set mn — \\yn — yn\\, then by hypothesis mn+i < Lmn + 6 from which it follows n-l
\\yn ~ yn\\ < Ln\\yo - y0\\ + 6 T V < Ln\\yo - y0\\ j=
, l
~L
.
If yo = yo, we see that the distance between the two solutions will never exceed j^;- Thus choosing n > n suitably, both of the solutions will remain in the ball B (0, yzx) , and the proof is complete. n The next theorem generalizes the previous result. Theorem 4.11.3 Consider the equation (4-88) and a set D C. IRS. Suppose there exist two continuous real valued functions defined on D such that, for all x € D,
Copyright © 2002 Marcel Dekker, Inc.
140
CHAPTER 4. STABILITY THEORY
(1) V ( z ) > 0 7 (2) AV = V ( f ( n , x) + R(n. x ) ) - V(x) < w(x] < a, for some constant a > 0. Let S = {x e D\w(x) > 0},6 = sup{l / (x)|x G 5} and A = {x e D\V(x] < b + a}. Then every solution, which remains D and enters A for n = n, remains in A for n > n. Proof. Suppose that yn = y(n,riQ,yo) e A then V(yn) < b + a and V(yn+\) < V(yn}+w(yn). liw(yn) is less than zero, yn £ 5, then, V(yn+\) < b + a, from which it follows that yn+i G A. If y^ G 5, then, because V(yn) < b, again it follows that V(yn+i) < b + a. The proof is complete by induction. D Corollary 4.11.2 If 6 = sup{w(x}\x e D — A} < 0 then each solution y(n) of (4-88), which remains in D enters A in a finite number of steps. Proof. From AV < w(y) we get V(yn) < V(ynQ] + E"=no u>W) < V(yno) + 8(n — no), from which it follows that V(y(n}) —>• — oo as n —> oo, and this is a contradiction because V(yn] > b + a for yn € D — A. D
4.12
Problems
4.1 Show that if in the equation xn+\ — anxn the product |n^n 0 a ? l ^s bounded, then the zero solution is uniformly stable. 4.2 Consider the following equation, yn+i = TL2^y^- Compute (by using a calculator), the two solutions starting from T/Q — 10~ 21 ,n = 0 and y$ — — l,no — 2 • 102. Conclude that the zero solution is not uniformly stable. 4.3 Show that for the system (4.4), the origin is the only fixed point, and that it is attractive but not stable 2
2
4.4 Let y = ( x , z ] be such that yn+\ = ('21"'")- Show that the set D — { ( x , z)\x2 + z2 < 1} is the region of asymptotic stability of the origin. (Hint: Apply Theorem 4.9.3. Take 4>(y) = x2 + z2 and show that if there is convergence, then I"I?Li(l + 0(^?)) = i—2—2 from which V(y0) — XQ + z{j.) '
Q
4.5 Consider the linear scalar equation yn+i — io g |n+3)^ n- ^m(^ ^ne solutions and show that the zero solution is asymptotically but not exponentially stable. (Hint: put zn — ynlog(n + 2).) 4.6 Show that the zero solution of the previous problem is not lp stable for any p > I .
Copyright © 2002 Marcel Dekker, Inc.
4.12.
PROBLEMS
141
4.7 Show that the function <5(e, no) in the definition of stability can be taken of class K. (Hint: take <5i(e) = sup<5(e,n). The function ^i(e) is positive and strict increasing. Take ?/> G K such that ^(e) < <5i(e). This function can be used instead of 6 in the definition.) 4.8 Show that an alternative definition of stability is: The trivial solution is stable if there exists 0 G K and ni G N*0 such that for n > HI, \\y(n,no,yo)\\ < 0(||?/o||, ™o)- (Hint: Take as 0 the inverse of ifj defined in the previous problem.) 4.9 Using Theorem 4.8.5, study the behavior of the solutions of the equation yn+i = y~2. 4.10 Show that y — 1 is asymptotically stable for the equation
yn+i = yn(yn - 2) + 2. Such equations arise in number theory (see Section 9.4). 4.11 Show that the equation 2/n+i = Vn(yn ~ 2) has chaotic behavior. 4.12 Consider the equation yn = Ayn^\ + b, where \
/ 0 0
\
u
and study the stability of the critical point. The problem arises in economics treating oligopoly systems. The case s — 2 was treated by Cournot, finding asymptotic stability. 4.13 Show that the function g(n, u} = un — w(un) in Corollary 4.8.3 is not decreasing, and prove that the origin is asymptotically stable. 4.14 Show that ft(yo}, the limit set of yo G IRS, is invariant and closed.
Copyright © 2002 Marcel Dekker, Inc.
142
CHAPTER 4. STABILITY
THEORY
ri--y2 4.15 Consider the equation yn+\ = l _ 2 n , y o = rj. Determine the values of 77 for which yn converges and find the limit point. 4.16 Consider the system
yn+i
= xn + /On),
and suppose that A(x n /(x n )) > 0 for all n. Show, by using Theorem 4.8.7, that the solutions are either unbounded or tend to zero. (Hint: Take Vn =
4.13
Notes
The definitions in Section 4.1 are modified versions of those given for differential equations, for example, see Corduneanu [43] and Lakshmikantham and Leela [104]. Theorem 4.1.1 is adapted from Gordon [79]. The results of Section 4.2 are adapted from Corduneanu [43]. Most of the material of Section 4.4 has been taken from Halanay [84, 86], see also [58]. The contents of Sections 4.5, 4.7 and 4.8 have been adapted from Halanay [86] and Lakshmikantham and Leela [104]. For Theorem 4.8.4, see Gordon [79]. Theorems 4.8.5 and 4.8.6 are due to Hurt [95] (see also LaSalle [108]). Theorem 4.9.3 is the discrete version of Zubov's theorem and can be found in O'Shea [133] and Ortega [138]. Theorem 4.9.4 is due to Diamond [56], [57]. The converse theorems are adapted from Lakshmikantham and Leela [104] and Halanay [86]. The total stability in the discrete case is treated by Halanay [86] and Ortega [138]. Practical stability is discussed in Hurt [95] and Ortega [138].
Copyright © 2002 Marcel Dekker, Inc.
Chapter 5
Difference Equations as Banded Matrices 5.0
Introduction
It is a well-known fact that second-order differential problems are very often encountered in the applications, especially among those derived from physics. A more frequent appearance of second-order problems is also true in difference equations. The latter, however, may appear either in their classical form or under the form of algebraic systems of linear equations having as their coefficient matrix a tridiagonal one. Moreover, in recent years there is the tendency to transform all linear algebraic problems (i.e. with a generic matrix) to tridiagonal form. As matter of fact, many classical methods to solve symmetrical linear systems, such as conjugate directions or Lanczos methods, are nothing but the transformation of the original matrix to tridiagonal form. Since tridiagonal systems result from second-order difference equations, it is worthwhile to analyze the principal results in this setting simply because of their increasing importance. There is, however, another reason to consider such questions. In fact many classical methods used to treat tridiagonal matrices such as LU factorization or cyclic reduction, lead to nonlinear first order difference equations whose study is interesting in itself. Of course the terminology used in linear algebra is often different from that used in the difference equations setting. For this reason, we shall start with the simplest example in order to introduce these concepts.
5.1
Initial Value Problems
We begin with the simplest example in order to become acquainted with the new notation. Let 143 Copyright © 2002 Marcel Dekker, Inc.
CHAPTER 5. BANDED
144
MATRICES
yn+i - ayn = 0. If ?/o is fixed, the problem is an initial value one. Choosing N > 0, letting
y = (yi,y2,---,yN)T and
\
-a V
1 -a
1/
Nxl
(where the nonevident entries are zero), the problem assumes the form
Ay = b.
(5.1)
When N -+ oo, A becomes an infinite dimensional matrix. It is a lower triangular matrix. Generalizing to less simple cases, one may deduce that: 1. The matrix A is lower triangular whenever the problem is an initial value one; 2. The matrix A will have constant value on each diagonal whenever the difference equation is of constant coefficients. The question of existence of the solution is trivial in this case since A"1 exists for all values of N and y — A~lb. The problem of stability assumes the following form. Suppose that the initial condition yo is perturbed so as to have b = b + 8b. Then the question is how the solution y is affected? Obviously, we have 6y = A-
a5y0A~
(5.2)
There will be stability whenever the entries of the first column of A l will be bounded with respect to N. We will have asymptotic stability if the entries of such a column decreases exponentially as the row index increases. The inverse of A is not difficult to find. In fact, by introducing the shift matrix K, already defined in Chapter 1, the matrix A can be written as A — I ~ aK, i.e. A is a polynomial, say p(K], of the matrix K and its inverse A~l will also be a function of the matrix K. Actually it is the inverse of p(K). considered as a formal series (see Chapter 2). In fact we have JV-l 1
A' = ]T <JK\ 2-0
It then follows that the stability corresponds to the case |a| — 1 and the asymptotic stability to the case a < 1, according to the results in Chapter
Copyright © 2002 Marcel Dekker, Inc.
5.1. INITIAL VALUE PROBLEMS
145
2. This can be generalized to generic lower triangular-banded Toeplitz matrices. In fact, to the characteristic polynomial of linear difference equations corresponds the polynomial associated with the Toeplitz matrix, defined as follows. Suppose A — p(K}. The polynomial of degree m associated to A is defined by p(z) = zmp(l/z), and this coincides with the characteristic polynomial of the corresponding difference equation. A further change in the terminology arises when one introduces norms to measure the perturbation. Suppose we use the || • ||i- It then follows that 1- \\dy\\i < c\N\5yo\ in the stability case; 2- \\fiy\\i < 02 1^2/0 1 in the asymptotic stability case; 3. \\6y\\i < C3\a\N\6yQ\ in the instability case. where ci, 02,03 are constants. In the terminology of linear algebra , where the problems are classified according to the behavior, with respect to the dimension N, of the parameter
= \\A\\\\A~ll the first case corresponds to a weakly conditioned problem, the second case to a well-conditioned problem, and the third one to an ill-conditioned problem. The parameter k(A) is called the condition number of the matrix A. Its importance lies in the fact that from (5.2) we have \\8y\\ <\\A~l\\\\5b\\, and from (5.1) we get Finally, putting together the last two results, we obtain
i.e. k(A) is the amplifying factor between the relative error in the initial data and the relative error in the solution. The concept of conditioning is, however, more general since it is defined for a generic matrix, not only for lower triangular Toeplitz ones. Moreover, it plays a similar role when the perturbation is of more generale kind. However it gives less information with respect to various stability concepts regarding the qualitative behavior of the solutions. In this chapter we shall adopt it and we shall return to stability in the following chapters. In the more general case, i.e. when the linear difference equation is not with constant coefficients, the matrix is still lower triangular, but it is no more a Toeplitz one. We leave it as an exercise to show that upper triangular matrices arise when dealing with problems where the conditions are imposed at the end of the interval.
Copyright © 2002 Marcel Dekker, Inc.
146
5.2
CHAPTER 5. BANDED
MATRICES
Boundary Values Problems
More interesting is the case of boundary value problems, leading to more general banded matrices. We shall consider in some details the case of second-order difference equations, starting with a few examples taken from the main applications. Example 13 Orthogonal Polynomials A set of polynomials p n ( x ] , n — 0 , 1 , . . . whose degree is equal the their index is said to be orthogonal in an interval / — [a, 6] of the real axis with respect to a weight function w(x) > 0 if they satisfy a relation of the type rb (Pn,Pm) = I pm(x]pn(x]w(x)dx Ja
= hnSnm
For simplicity we shall assume hn = 1 for all n, i.e. all the polynomials are normalized (for the more general case, see Problem 5.3). Orthogonal polynomials satisfy a second-order difference equation depending on a parameter x, such as bn)pn(x)
(5.3)
= 0
(see Problem 5.1). Let kn be the coefficient of the higher degree term in each polynomial pn(x}. It is easily seen that 71+1
and then an ^ 0 for all n. Moreover it is possible to obtain that 1
Cn
(5-4)
0-n-l
(see Problem 5.2). To show the deep relation between orthogonal polynomials and second-order difference equations of the previous type, one must consider that the Favard theorem (see [39]) also states the converse, i.e. polynomials satisfying (5.3) are orthogonal with respect a suitable weight function. Special importance in the applications, for example in the construction of quadrature formulas, is to consider the roots of such polynomials. Let N > 0, then (5.3) can be stated as follows in matrix form /
/
xD
\
= A )
Copyright © 2002 Marcel Dekker, Inc.
/
147
5.2. BOUNDARY VALUES PROBLEMS where D — diag(ao,ai, • • • , a^v-i), and
A=
-60
1
GI
—61
CN-I
1 —t>N-i J
Usually in the applications COP-I(X) = 0. In this case, it is immediately seen that if a: is a root of PN(X], it is also an eigenvalue of the matrix D~1A. Due to (5.4), such a matrix is symmetric. The problem then assumes the form \
A) +
\ PN-I )
\ PN-I
\
(5.6) T where & — — ^-, a; = ^-, and e/v = (0, 0 , . . . . 1) . For future reference, we Oj (Z?' ^ ' ' shall write the above equation in the simplified form, xp(x) = Tp(x)
(5.7)
where p(x) is the vector whose entries are the polynomials Pi(x) and T is the tricliagonal matrix appearing in (5.6). Since T is tridiagonal and symmetric with Q>J ^ 0, by a well known theorem of linear algebra, it follows that the roots of orthogonal polynomials are real and simple. Analogous to the previous example is the case of the Euclidean Algorithm. Example 14 Euclidean Algorithm The Euclidean algorithm is used to find the greatest common divisor (gcd) between two polynomials. Recently it has been proposed to find the roots of a generic polynomial. The procedure is as follows: let po(z) and p\(z) be two polynomials of the complex variable z. We suppose that the degree of po(z] is N. The degree of p i ( z ) may be any value between 0 and AT — 1, if one is only interested in the gcd(po,Pi)- Since our scope is different, we suppose that its degree in exactly N — I . By dividing the two polynomials one obtains a quotient q\(z) and a remainder P2(z). One continues by dividing p i ( z ) and P2(z}. The n-th step is then described by
(5.8)
Copyright © 2002 Marcel Dekker, Inc.
CHAPTER 5. BANDED MATRICES
148 while the last step will be
Pm-l(z) =
Qm(z)pm(z)
with m < Ar. Moreover we have'O < deg(pn+\) < deg(pn), n = 1,2, . . . m. It is easy to check that pm is a common factor of all the polynomials pn(z). It is then the gcd(p 0 -Pi)- According to the supposed degree of the initial polynomials, we are sure that deg(q\(z}} is one, and as a rule the same should be true for the other quotients, i.e. it would be deg(qn(z)} — 1, and deg(p n (z)) — Ar — n. When this happens we say that the procedure terminates regularly. Suppose now that po(z) and p\(z] are coprime and monic. We obtain
where the (3n are chosen so as to have all polynomials monic. In matrix form we have 0
P2(
(5.9)
rri
where
T=
/ c*i 1
\
,81 a 2 02 l
From (5.9), it follows that if z is a root of po(z), it will be an eigenvalue of T. This way the problem of finding the roots of a polynomial is reduced to the problem of finding the eigenvalues of a matrix. With judicious choices of p i ( z ) , one may also obtain information on the multiplicities of the roots (see [22], [25]). Example 15 Conjugate Directions Method. Let A be an N x N symmetric real matrix. Suppose moreover that it is also positive definite, i.e. for all u G IR^, uTAu > 0. Consider the second-order vector difference equation i
(5.10)
where HJ G IRA . UQ given, w _ i = 0. We need to specify the set of coefficients 7, and /37. Usually they are obtained by exploiting the properties of the matrix A. There are many ways to do this. The most important choices are obtained by imposing one of the following conditions:
Copyright © 2002 Marcel Dekker, Inc.
5.2. BOUNDARY VALUES PROBLEMS
149
i) uf AUJ = diSij, ^4-orthogonality or conjugancy ii) uj Uj = di§ij, orthogonality. In the first case we obtain the conjugate directions method, while in the second case we obtain the so called Lanczos method. It is not difficult to verify that in the case i) we get _ Pi —
It can be proved that all the vectors Ui are linearly independent and since in JRN there cannot exist more than N linearly independent vectors, UN = 0 (see Problem 5.5). In matrix form, by setting U = (UQ. u\,... , UTV-I), i.e. the columns of the matrix U are just the vectors HI obtained from (5.10), AU = UT, 70 1
71
PN-I 7;v-i
Since the columns of U are linearly independent, such a matrix is invertible and A = UTU~1, i.e. the process leads to transform the original matrix A in tridiagonal form through a similarity transformation. Similar considerations can also be made in the second case.
5.2.1
Invert ibility of tridiagonal matrices
Once the problem is stated in matrix form, the problem of the existence of solutions is transformed into the problem of existence of the inverse of the tridiagonal matrix. We note, however, that in many applications the existence of the inverse of T is not enough. \Ve need to know whenever T is well-conditioned. We shall consider such questions in the subsequent subsections, while in this section we shall deal with getting the solution. In order to discuss only the necessary parameters, we shall consider the case 7i ^ 0. The difference equation and the matrix will be normalized as follows: cr n _i?/ n _i + yn + rnyn+i = 0 yo = a, In matrix notation it assumes the form
where
Copyright © 2002 Marcel Dekker, Inc.
yN+i = P.
(5.11)
CHAPTER 5. BANDED MATRICES
150
( 1
n
T=
y=
(5.12) 1
T/V-l
0-/V-1
1
6 =
V
V
/
First, we shall discuss the existence of the following factorization T^LDU,
(5.13)
where
D = / 1 L
=
1 ) \
U
=
1 = 1,
/ (5.14)
This is a nonlinear difference equation of first order of Riccati type. Obviously, the factorization (5.13) exists iff of, ^ 0, i = 1. . . . ,N — 1. Since both L and U are invertible, T"1 exists whenever D is invertible. The key step of the discussion is obviously the behaviour of -the solution of the equation (5.14). The study of the behavior of its solutions is not easy and needs more involved techniques even in the constant coefficients case. We shall start, however, with the latter case in order to introduce the new technique. We shall see that under suitable conditions the solutions of (5.14) are bounded away from the origin, and their dynamics can be enclosed in a box in the (di,d,+i) plane. The formal proof will be given in the next two theorems. In order to simplify the discussion and consider together the two possible cases, a special notation is needed. Let us define the following functions:
Copyright © 2002 Marcel Dekker, Inc.
151
5.2. BOUNDARY VALUES PROBLEMS sigma*tau=-0.2
sigma*tau=0.2
1.2
0.8
.±0.6
0.2
0
0.2
0.4
0.6
0.8
1
Figure 5.1: Dynamics of solutions of (5.14) in the cases ar > 0 and ar < 0.
x+ =
x 0
if if
x >0 x <0
x 0
if if
x <0 x > 0.
As noted above, we consider first the particular case of Toeplitz tridiagonal matrices, that is, the case in which 01 — cr, TI = r, i = 1, . . . , N — 1. Lemma 5.2.1 Let a{ = a, TI = r, i = 1, . . . , TV - 1, A = 1 - 4(crr) + > 0 7 and m > 0 be such that
m<
v/A
and d\ = 1 > m. Then 7?i < di < 1 — (crr)_/m,
i > 2.
Proof. From the hypothesis on A it follows that the equation x2 — x + (CTT) + = 0 has either one or two real non-negative roots. Since the chosen values of m are between such roots, we have m 2 — m + (<JT) + < 0, that is, 0 < m < 1 — (crr) + /m. Setting f(x] = 1 — (crr)/x, we have (aT}f'(x] > 0, and then the minimum of /(x) in the domain Dm = {x : x > m} is attained at the lower border m when ar < 0 or at infinity in the opposite case. It
Copyright © 2002 Marcel Dekker, Inc.
152
CHAPTER 5. BANDED
MATRICES
assumes the value 1 — (ar) + /m (See Figure 5.1). By considering equation (5.14) we have di+i - f ( d i ) > 1 - (-^± > m, m for all z and this completes the first part of the proof. The proof concludes by considering that f ( x ) < 1 — (crr)_/x < 1 — (ar}-/m. D Such solutions in the two representative cases are shown in Figure 5.1. Going back to the general case with <77; and TJ varying, the previous result can be generalized as follows:
= 1 — 4(cr?;T7)+ ,
(0"?")- = minKcTjTj)-},
m = min
one has (see also [110] for a more general version). Theorem 5.2.1 // A z > 0 for i = 1, . . . , N - 1, then m < di < 1 — (crr)_/ra, Proof.
i — 1, . . . , N.
Let fl(x] — 1 - (cr ? T;)/x, Dm = {x : x > m}. Since A7 > 0 and 1 - x/A, < 2m < 1 + v^,
by Lemma 5.2.1 it follows that for x > m, /7;(x) > m. Moreover
D
The condition Aj > 0 implies that in the (a, r)-plane, the points (CT^,TJ) must lie inside the region bounded by the hyperbola 1 — 4crr = 0 (see Figure 5.2). That is, fT7;T?; < ^
l = l,...,N-l.
(5.15)
Outside such a region it may happen that the matrix T is singular. By the way. we have observed that the same technique used in proving Lemma 5.2.1 can be employed in more general cases. For example, in factoring a /c-banded matrix the following non-linear difference equation arises (see [63], [110]): n
.
.
-
9] ~
r,
h
.
^^
] ~~
-*
_
~
_
^L__
yj-i y-j-k Such equation is a k-th order difference equation (some may prefer to call it a delay difference equation). The initial conditions y\, y^- • • • , yk are assigned. For completeness we merely report the generalizations of the above results to the present case. For the proof see [110]. The following lemma generalizes Lemma 5.2.1.
Copyright © 2002 Marcel Dekker, Inc.
153
5.2. BOUNDARY VALUES PROBLEMS Figure 5.2: Invertibility region.
0.8
0.6
0.4
0.2 a
0
-0.2
-0.4
-0.6
-0.8
-1 -4
0 sigma
-2
1
Lemma 5.2.2 Suppose that
a) Sj = s > 0, tj — £, u>j = w for all j ; b) A = s2 - 4(t+ + w+) > 0, m > 0 suc/i
s - v/A 2
<m< ~ ~ ~
~
s 2
c) Vj > m for j = 1,2, ...k. Then for all j > k,
m < y j < s — (t_ + w-)m~ . Theorem 5.2.2 Suppose that
a) Sj > 0;
d) yi > mi.
Copyright © 2002 Marcel Dekker, Inc.
154
CHAPTER 5. BANDED
MATRICES
Then for all j, m = min(yi,TOi) < yr Moreover, posing t- = max (tj)-, W- = max (u>j)_, s = maxsj, we have m < yj < s — (t- + w-)m~l.
5.2.2
Sufficient conditions for well-conditioning
We recall that the matrix T is well-conditioned if its condition number k(T) is bounded from above by a quantity independent of the dimension N of the matrix. In applications, however, this quantity can be allowed to grow linearly with N. In this case, we shall say that T is weakly well-conditioned. We shall now derive additional conditions on the entries of T in order to avoid ill-conditioning. For this, we need an estimate of (JT" 1 !). This can be done by considering (5.13). We have
\\T-I\<\\L-I\\-\\D-I\.\\U-I\\. From Theorem 5.2.1 we get / \-1 \\D'l\\ = (mincU < m"1. V * / The previous estimate is independent of the dimension of the matrix. The estimates for Hi/" 1 ]! and H^" 1 ]! need a long and tedious manipulation (see [23] for the complete treatment). The next theorem gives sufficient conditions in a case of particular interest in the applications.
Theorem 5.2.3 Suppose that a) <j? and TJ have constant sign, b) CTi+Ti-i < 1,
c) or 7 :_i +r, > -1 or b')
Vi+Ti^
> -1,
c') cr?;-i + Ti < 1. Then matrix T is invertible and well-conditioned. Proof. See [123]. If inequalities are substituted by equalities, then we have weak conditioning. A more easy set of sufficient conditions is provided by the following theorem.
Copyright © 2002 Marcel Dekker, Inc.
5.2. BOUNDARY VALUES PROBLEMS
155
-0.5 -
-2
-1.5
Figure 5.3: The region of well-conditioning for Problem 5.11. Theorem 5.2.4 Suppose that a) &i and r\ have constant sign,
d) \0i-i + Ti\ < 1. Then matrix T is invertible and is well-conditioned. Proof.
See [23].
This means that all the pairs (cr^^Tj), (af,r ? ), (a ? , TJ_I) should be inside the strip in the (<j, r)-plane shaded in Figure 5.3. The hypotheses of the theorem are very simple to test, and provide a practical tool to test both non-singularity and conditioning for the matrices examined in this section. We note that condition b) implies (5.15), i.e. conditions of invertibility of T (see Problem 5.6), and could be substituted by it. When the inequalities are substituted by equalities, then weak conditioning arises. It is interesting to observe that the usual conditions of diagonal dominance correspond to taking only a piece of the above strip, i.e. the square having vertices at the points (1, 0), (0,1), (-1,0), (0,-1).
Copyright © 2002 Marcel Dekker, Inc.
156
CHAPTER 5. BANDED
MATRICES
Figure 5.4: Condition numbers of matrices defined in Examples 16 and 17. Example 16
/1 1 2
A=
2 3
\
1
(5.16) 2 3
\ The value of extradiagonal elements satisfies the hypotheses of Theorem 5.2.4. and therefore the matrix is well-conditioned (see the left part of Figure 5.4) Example 17
/ 1 (5.17)
A=
V
i
The value of extradiagonal elements satisfies the hypotheses of Theorem 5.2.4 only with equalities and therefore the matrix is weakly well-conditioned (see the right part of Fig. 5.4). The following problem is related to the results presented above. Let yn = aijn-r + byn-s
(5.18)
with a, b, r, s real numbers. When is the null solution asymptotically stable for all couples (r, s)? The answer is given by the following theorem (see [126], [135]). Theorem 5.2.5 A necessary and sufficient condition for the null equilibrium of (5.18) to be asymptotically stable for every couple (r. s} of delays is that a\ + \b\ < 1.
Copyright © 2002 Marcel Dekker, Inc.
157
5.3. CYCLIC REDUCTION
5.3
Cyclic Reduction
Cyclic reduction belongs to the class of divide and conquer algorithms and is a powerful technique in treating tridiagonal matrices. It can be easily described in the framework of linear algebra, although it could be defined by considering the equivalent difference equation. Essentially it splits the solutions by considering separately the entries with even index and those with odd index (for this reason it is also called odd-even reduction). The original equation is then split into two equations with half unknowns. Under some conditions, the process can be iterated until eventually the number of unknowns reduces to one or two. The description of the method will also provide examples of interesting nonlinear difference equations. For simplicity, we shall assume that N = 2m - I . Consider the following problem, bnyn+i + anyn + Cnyn-i = gn,
n = 1, 2 , . . . . TV - 1
(5.19)
with 2/0 and y^ assigned. It can be represented in a matrix form as (5.20)
MQy = d,
where
aN
V
and y and d are the vectors whose entries are the yn and gn, (plus the boundary conditions), respectively. Moreover, let P be the matrix which performs the operation of grouping odd and even entries of a vector, i.e.
/
1 \
/
2 3
P
=
i
and let us define
Copyright © 2002 Marcel Dekker, Inc.
om
1
\
3
i
j
2m-l 2 \ 2m - 2 /
158
CHAPTER 5. BANDED MATRICES
The matrix P is a permutation matrix satisfying the relation PT P = I. It is obvious that y° and ye are vectors of dimension 2 m ~ 1 and 2m~l - 1 having as entries the odd-indexed and the even-indexed entries of y respectively. Equation (5.20) is first transformed into V
(5-21)
where the matrix PM§PT has the block form Al
Tl
S1
B,
The matrices A\ and B\ are diagonal and TI, S\ are lower and upper triangular of dimension 2 m ~ 1 x (2 m ~ 1 - 1) and (2m~1 - 1) x 2 m-1 , respectively. Note that in the case under consideration they are not square matrices. The matrix PM$PT is then factored as PM0PT = LiC/i, where LI and U\ are block triangular.
'1 = The square matrix M\ = B\ — SiA±lT\ has the dimension 2 m ~ 1 — 1 and is tridiagonal itself. In principle the process can be continued by substituting M\ to MQ. Before proceeding we will show how the cyclic reduction transforms the original difference equation (5.19). Equation (5.21) becomes
' d° The procedure to get the solution is now standard. It is done in two steps: first an intermediate vector x — (x°1xe)T is defined as solution of L\x — Pd, and then Uy = x is solved. It is immediate to verify that the first step is trivial because it amounts to setting ,£° — fj°
.£e — (-je
SiA~~^~d°
Considering that A\ is diagonal, the above step does not present difficulties, provided that al are nonzero. The second step, that is the solution of
yc if
Copyright © 2002 Marcel Dekker, Inc.
5.3. CYCLIC REDUCTION
159
leads to the solution of the system of dimension 2 m ~ 1 — 1 Miye = xe,
(5.22)
and then we obtain y° = A^l(d° — T\ye}. Namely, the odd components of the solution vector y can be obtained once the even components are known and the latter can be derived independently. Equation (5.22) can be written componentwise as,
&iyn+i + aiy£ + ciy5_i = o£,
(5.23)
i.e. the even components of the solution satisfy a second-order difference equation, having as coefficients the nonzero elements of the n-th row of MI . The process can be iterated provided that the odd terms of the sequence denning the diagonal of MI are not zero. Is there a way to say in advance when the process can be iterated? We shall discuss the question for the case of Toeplitz tridiagonal matrices.
5.3.1
The case of Toeplitz tridiagonal matrices
Let MQ be an N x JV, TV = 2m — 1 tridiagonal Toeplitz matrix, i.e.
CD
ao
60
\
CQ
NxN
As observed above, the cyclic reduction defines a sequence of tridiagonal matrices {Mi} of dimensions (2 m ~ r — 1}. Such matrices are still of Toeplitz type whose nonzero elements on the generic row will be denoted by {ci,a.j,&j}. It is an easy matter to derive the following equations related to the elements of matrices Mi to those of MJ_I :
^-^ ^
(5.24) (5.25)
2
^-.
(5.26)
This is a system of three nonlinear difference equations. They are defined provided that the sequence {a,;} does not contain the zero. In this case the process of cyclic reduction can be continued. In order to simplify the notation, we introduce the new variables
Copyright © 2002 Marcel Dekker, Inc.
160
CHAPTERS.
BANDED
MATRICES
We then have Ci
=
1 — 2<7jTi,
(5-27)
Such system completely describes the process. Many features can be derived almost at once. For example Fl:
din > 0 for i > 0;
F2:
if |^ < 1 then al tends quadratically to zero;
F3:
if |^ < 1 then r\ tends quadratically to zero.
F4:
If O-Q = ±TO then a, = r% for i > 0
(see Problem 5.8). The above properties provide useful information about the matrices Mi. For example, F2 and F3 imply that Mj rapidly tend to become bidiagonal when
(1 - x,)2
that is 2 2r ^xi This is a nonlinear difference equation which can be further simplified by setting 1
After a few manipulations, we get *+i - ,! *'2 , 2 = 4>te)(5-28) (2 + ' ) This equation has three critical points, namely 2 — 0, 2 = —3/2, arid z = 1/2. The first two arc unstable, while the third one is asymptotically stable
Copyright © 2002 Marcel Dekker, Inc.
5.3.
CYCLIC REDUCTION
161
(see Problem 5.9). Moreover, the semiaxes z > 0 and z < 0 are invariant. It follows that the positive critical point is asymptotically stable for all solutions starting from positive initial points. Since z; > 0 implies ^—20-^ > 0, that is a2 — 46^ > 0, it follows that if the characteristic polynomial of the original equation (5.19) (obviously with constant coefficients) has real roots, the same will happen for the characteristic polynomials of (5.23) and subsequents. Diagonally dominant matrices have ZQ > 0. In fact from a2 > (\b\ + c|)2
it follows that a2 — 46c > (|6| — |c|)2 > 0. This confirms the known result that for such important class of matrices the process is well defined. The asymptotic stability of z = 1/2 permits the assertion that lim
i—>oo
flj
In order to get the limit of the sequence a^, it will be useful to prove the existence of a constant of motion for the system (5.27). Theorem 5.3.1 The quantity a2(l — 4<7jT,-) is a constant of motion for the system (5.27). Proof. By multiplying the second and the third equations of the system, we get
ai+1Ti+1
= fM = (*' - ^ i
from which it results
Considering the definition of x?;, the conclusion follows. D In order to obtain the limit value of the sequence Q,J, we consider the sequence of products O{Ti. We have
= - i The product cr^r7; satisfies the equation
Copyright © 2002 Marcel Dekker, Inc.
(5 30)
-
162
CHAPTER 5. BANDED
MATRICES
(5.31)
(1 — 2(JiTi)2 '
which, when <JQTQ\ < 1/4 can be estimated explicitly, obtaining °iTi < 7( 4cr OTo) 2 ',
i.e.
lim <7jTj = 0 (see
Problem 5.6).
Note that \CTQTQ\ < 1/4 is equivalent to ZQ > 0, the hypotheses which guarantee the existence of the limit of z z . This permits us to obtain the limit of the sequence {a,-}. In fact, from the constant of motion, we have
namely - 4cr 0 ro) 1//2 . The dynamics of equation (5.28) are then very simple for ZQ > 0- For negative initial values the dynamics become very entangled (see [174]). More complete information about the process can be derived by considering the following sets: J° =
{(cr,r) | 1 -40-r = 0, a < 0,
r < 0}
{(a,r) | l - 4 o - r > 0, a < 0.
r < 0}
J
=
ft
- {(a,r) | cr + r = -1} a + T > -1,
cr < 0,
r < 0}
The following theorem can be proved. Theorem 5.3.2 Suppose that 1 — 4aoro > 0. Then the above sets are invariant. Proof. The first set is invariant since err = 1/4 is an equilibrium point for (5.31). The invariance'of the second set is a simple consequence of the above discussion. The invariance of (R) is verified by considering that al + r , 2 -
IffiTi
and by noting that if 2<7o + TO — — 1. the same will happen for all values of i. The proof of the remaining assertions follows similarly. D
Copyright © 2002 Marcel Dekker, Inc.
5.4. PROBLEMS AND REMARKS
163
In the applications the case &Q — TO — — \ has a special significance. In this case one has GI = TV = — ^. This is easily explained by considering that (-1/2,—1/2) is an equilibrium point for the system (5.31). In terms of matrices, we can say that in this particular case all the matrices Mi have the same non-zero entries.
5.4
Problems and Remarks
5.1 Show that orthogonal polynomials satisfy (5.3). 5.2 Show that (5.4) holds true. 5.3 Derive the tridiagonal matrix (5.6) when the polynomials are not normalised. 5.4 Derive ^i,(3l in Example 15. 5.5 Show that the vectors Ui are linearly independent in Example 3. 5.6 Show that \al + TV| < 1 implies (5.15). 5.7 In cubic splines approximation of a smooth function on a set of not equi-spaced knots, the value of the second derivative of the spline at knots satisfy a relations like (5.11) (see [166]). Denoting by hi the distance between the i-ih and the previous one, &{ and TV assume the following values ~
hi
r,
TV =
hi+i
Verify that the hypotheses of the theorem are satisfied. 5.8 Prove Fl, F2, F3. and F4 in Sec. 5.3. 5.9 Prove the stability of the critical points of Equation (5.28) are as indicated in the text.
5.5
Notes
Although tridiagonal matrices and three-terms recurrence relations are like the two faces of the same problem, for a long time their study has been carried out separately. The main reference concerning the results on the latter is the Golub and Van Loan book [78]. The transformation of a generic matrix in tridiagonal form is discussed in Geist [73] and Parlett [147]. There are many good books devoted to orthogonal polynomials (for example Chihara [39])and to quadrature formulas (Davis and Rabinowitz [54]).
Copyright © 2002 Marcel Dekker, Inc.
164
CHAPTER 5. BANDED
MATRICES
Often the main results are summarized in books on numerical analysis (see for example [51], [70]). The Euclidean algorithm is not very popular. It is usually confined to books on number theory. Its use for finding roots seems quite recent (Brugnano [22] and Brugnano and Trigiante [25]). The conjugate directions go back to the early fifties (Hestenes and Stiefel [91]). At that time it was derived from the theory of polarity. Nowadays one prefers to see it as a method which permits the transformation of a matrix in tridiagonal form. For presentation of the family deriving from such method, see Ashby, Mantueffel. and Saylor [15]. Concerning the existence of the inverse of a tridiagonal matrix, as well as its conditioning, see Lewis [111], Usmani [178, 177], Mattheij and Smooke [118], Brugnano and Trigiante [23], and Mazzia and Trigiante [123]. Cyclic reduction has been studied recently since it is considered a good algorithm suitable for parallel machines; see Buzbee, Golub, and Nielson [31], Amodio and Mazzia [14], and'Amodio [13]. Results on the relations between banded matrices and difference equations can be found also in Cheng and Hsieh [38] and Kratz [103].
Copyright © 2002 Marcel Dekker, Inc.
Chapter 6
Applications to Numerical Analysis 6.0
Introduction
Despite the fact that the two theories have been developed almost independently, there are several connections between numerical analysis and the theory of difference equations. In the previous chapters, some common linear problems have seldom been considered. In this chapter, we shall explore more deeply some of these connections. In Sections 6.1 to 6.3, iterative methods for solving nonlinear equations are discussed, and the importance of employing the theory of difference inequalities is emphasized. Sections 6.4 and 6.5 deal with certain classical algorithms, such as Clenshaw, Miller, etc., from the point of view of the theory of difference equations. Sections 6.6 and 6.7 are devoted to the study of monotone iterative techniques, which offer monotone sequences that converge to multiple solutions of nonlinear equations. This study also includes an extension of monotone iterative methods to nonlinear equations with a singular linear part as well as applications to numerical analysis. In Section 6.8 we provide related problems of interest.
6.1
Iterative Methods
By iterative methods one usually means methods by which one is able to approximate a root of a linear or nonlinear equation. Let us consider the problem of solving the algebraic equation of the form F(x) = 0,
(6.1)
where F : IRS —> IR'S, which is usually transformed into the iterative form: Xn+l = f(xn),
165 Copyright © 2002 Marcel Dekker, Inc.
XnQ - X0.
(6.2)
166
CHAPTER 6. APPLICATIONS TO NUMERICAL
ANALYSIS
The function /(x) is called the iterative function and is defined such that the fixed points of (6.2) are solutions of (6.1). There are countless ways to define the iterative function /. The essential criterion of choice is that the root x* of (6.1), which we are interested in, is asymptotically stable for (6.2), although it may not be sufficient to ensure rapidity of convergence. What is ensured is the existence of a region of asymptotic stability for x*. In other words, if XQ is near enough to x*, the sequence of iterates will converge. If / is linear, we know that asymptotic stability can be recognized by looking at the eigenvalues of A (the iterative matrix), and indeed this is what is done in the study of linear iterative methods such as Jacobi, GaussSeidel, SOR, etc. Moreover, in the linear case, asymptotic stability implies global asymptotic stability. For nonlinear equations the situation becomes more difficult. In this case there are essentially three kinds of results that one can discuss: A. LOCAL RESULTS. These results ensure asymptotic stability of x*, but very often they have nothing to say on the region of asymptotic stability. Usually theorems in this category start with the unpleasant expression "If XQ is sufficiently near to x* . . . " . B. SEMILOCAL RESULTS. The results in this category verify that an auxiliary positive function (usually the norm of the difference of two consecutive terms of the sequence) is decreasing along the sequence itself. The sequence is supposed to (or be proven to) lie in a closed set D c IRS. Then one can infer that the auxiliary function has a minimum in D and this minimum is located at x*. We shall see that this usually requires that x* is exponentially stable. Connected to these types of results are those requiring the stronger condition of contractivity of / in a closed set D C IR S . Contractivity implies exponential stability of the fixed point x*. C. GLOBAL RESULTS. These results say that the sequence xn, given by (6.2) is almost globally convergent (except on a set of measure zero). Of course, results are very few in the third class of the nonlinear case. One of such result is due to Barna and Smale [18], who say that for a polynomial with only real roots. Newton's method is almost globally convergent (we shall not present their results here). We shall, however, discuss in the next two sections some of the most important results in the classes A and B.
Copyright © 2002 Marcel Dekker, Inc.
6.2. LOCAL RESULTS
6.2
167
Local Results
Let us begin with the following main result. Here / : IRS —•> IRS and x* satisfies x* = /(x*). Theorem 6.2.1 Suppose that f is differentiable with Lipschitz derivative, and the spectral radius of f(x*) is less than one. Then x* is asymptotically stable. Proof. equation
en+i
By letting en = xn — x*, from (6.2) we obtain the difference
= f(xn} - /(x*) - f(x*)en + A/V 4- s(xn - x*)) - f(x*)}dsen Jo
=
f'(^)en + g(en)
(6-3)
where \\en\\
(see Problem 6.2). The equation (6.3) is in the form required by Corollary 4.7.2, which assures the asymptotic stability of x*. D Note that the asymptotic stability does not imply in general that ||xi — x*|| < |xo — x*|| because asymptotic stability will only imply \\en\\ —> 0. The result established in Chapter 4 can be used to prove the next theorem on the convergence of nonstationary iterative methods defined by xn+i - q(n,xn],
(6.4)
where q : N+Q x D C IRS —* D, XQ G D. It is obvious that non stationary iterative methods are, in our terminology, nonautonomous difference equations. Theorem 6.2.2 Consider the difference yn+i = /(*/n),
equation yno = XQ,
(6.5)
where f : D —> D, and f is locally Lipschitzian. Suppose that y* e D is an asymptotic stable solution for (6.5) and that the solutions y(n,no,xo) of (6.5) and x(n, no, XQ) of (6.4) are in D for n e A/"^. Assume further that \\q(n,x)-f(x)\\
(6.6)
where Ln —>• 0 for n —> oo. Then the solution x(n, no, XQ) —->• y* , when XQ is suitably chosen.
Copyright © 2002 Marcel Dekker, Inc.
168
CHAPTER 6. APPLICATIONS
Proof.
TO NUMERICAL
ANALYSIS
Rewrite (6.4) as xn+i — f(xn) + R(n,xn)
(6.7)
and consider (6.7) as the perturbed equation relative to (6.5) with R(n, xn} = q(n, xn) - f(xn)
(6.8)
as the perturbation, which tends to zero as xn —>• y* . Then apply Corollary 4.11.1 D Other local-type results can be obtained by using the comparison principle given in Section 1.6. For example: Theorem 6.2.3 Let D be an open subset o/Hs and f : D —> D continuous, g : 1R+ —> IR+ . Suppose that (1) there exists x* € D such that for all x € B(x*,8) C D, 8 > 0, \\f(x)-x*\\<\\x-x*\\+g(\x-x*\\);
(6.9)
(2) G(u) = u + g(u] is nondecreasing with respect to u and c/(0) — 0;
(3) the trivial solution of G(un},
UQ = \\X0 - X
is asymptotically stable, with UQ in the domain of asymptotic stability, and Ui < UQ for all i. Then x* is an asymptotically stable fixed point for the equation = f(xn).
(6.10)
Proof. Let XQ e B(x*,8}. Then by Theorem 1.6.1, we have ||x n — x*j| < un and, because un is decreasing to zero, all xn 6 B(x*, 6) and the sequence xn — x* will tend to zero. D The next two examples show how the applications of LaSalle's Theorem 4.8.6 can be useful in studying iterative processes. Example 18 Secant method On the real line the secant method is defined
by Zn+1 ~
., f(Zn+l)
( \ ~ f(Zn)
If, for all n. f ( z n ) ^ f(zn+i), the difference equation (6.11) is well-defined. Suppose that / is differentiate and that a is the simple root of f(z] = 0. Using the transformation en = zn - a
Copyright © 2002 Marcel Dekker, Inc.
(6.12)
6.2.
LOCAL RESULTS
169
and the mean value theorem f(a + e) = f(a)e + g(a,e}e2,
(6.13)
we write the new difference equation corresponding to (6.11) having the fixed point at the origin in the form en+2 = M(a,e n ,e n + i)e n e n +i,
(6-14)
where MI \ > n n -g(a,en}en M(a,e n ,e n +i) = —T,—;-^-77~--x~-
I6-15)
If a is a simple root and g(a, e) is continuous and bounded, then M(a, e n , e ra _i) is continuous and bounded if e n ,e n _i are small enough. The second-order difference equation (6.14) can be transformed into a first-order system by introducing the variables xn = en and yn = en+\. Then the system is
=
yn+i
=
Urn
M(a,xn,yn)xnyn.
By using the Liapunov function V(x,y) — \x\q + \y\q,q > 1. one obtains &V(xn,yn} = -(I - \M(a,xn,yn}yn q}\xn\q , which is negative if W(x,y) = (1 — \M(ot,x,y)y\) > 0. Let
Since y = 0 implies W(x, y) = 1 > 0, there exists some 77 > 0 such that W(x,y) > 0 in D(q). If the initial values ZQ and z\ are such that (xo,yo) and (xi,yi) G D(rj), then (x n , ?/n) will remain in Z) (see Corollary 4.8.4) and will converge to the maximum invariant set contained in the set E — { ( x , y ] e D(rj)\x = 0}. The only invariant set with x — 0 is the origin, so we obtain (x n , yn) —>• (0, 0) for n -^ co. Example 19 Newton method Corollary 4.8.4 can be used to study the convergence of the Newton method defined by zn+i = zn - f ( z n ) ~ l f ( z n ) .
(6.16)
Since there are no additional difficulties involved, we shall consider the case where (6.16) is defined in H s .'Let the root a. which we arc interested in, be simple. The change of variable similar to (6.12) allows us to write the method in the form
Copyright © 2002 Marcel Dekker, Inc.
170
CHAPTER 6. APPLICATIONS TO NUMERICAL
ANALYSIS
en+l = Mi(e n )[M 2 (e n )e n - f(a + e n )]
< n)\ = --i(e ox J
, ^ A M f ( 2 (e and n n) = L
ox
It can be shown that for ||en|| < 77, where 77 is positive and suitably chosen. M 1 (e n )[M 2 (e n )e n - /(a
e
2
Then, taking V(c} = \\e\\, one can obtain ) < -(1 -
showing that AV(e n ) < 0 for /c(?7)||e n || < 1. Using Corollary 4.8.4. we get a region of convergence D(T)Q) = {x\\\x — a\\ < J]Q} with r/o = min(?7. l / k ( r j ) ) . The parameter ?/ can be chosen to maximize 770 and consequently the region of convergence.
6.3
Semilocal Results
Results of semilocal type are obtained either by requiring contractivity or some of its generalizations on a compact D C IR'S or by requiring the weaker condition that some auxiliary function decreases over a sequence. We shall consider some cases of the latter type. We begin discussing the simplest of such results in some detail to clearly bring out the arguments used. Theorem 6.3.1 Suppose that allxn obtained by (6.2) lie in a compact DQ C IRS and that xn+2 - xn+i\\ < a\\xn+i - x n ||, (6.17) with a < 1. Then the sequence converges to x* G DQ. Proof.
Setting yn = \\xn+i - xn\\
(6.18)
2/n+i < ayn.
(6.19)
we have, because of (6.17), We can now apply Theorem 1.6.1 to obtain yn < un, where un is the solution of un+i = aun (6.20) provided that y$ < UQ.
Copyright © 2002 Marcel Dekker, Inc.
6.3. SEMILOCAL RESULTS
171
Now the equation (6.20) has the solution un = 0 exponentially asymptotically stable, which means that Y^=i uj ls bounded (see Theorem 4.1.1). In fact from (6.20) we get CO
-1
It then follows that \\Xj+i - Xj\\ < UjJ = 1 , 2 , . . . , if we choose UQ = \\x\ — XQ\\. Moreover, for p > 1. we obtain + xn+p-l ~~ %n-p-2 + • • • p-l
p-l ]C \\xn+j+l 3=0
<
X
n+J\\ ^ Yl Un+jj=0
Because the right-hand side can be made less than an arbitrary positive quantity when n is chosen properly, this inequality shows that xn is a Cauchy sequence and thus must converge to x* in DO- From this result we can also give a bound for the distance between XQ and x*. In fact, from n— 1
n— 1 1
X
\\Xn - Zo|| < £] I^-H- ~ J'II - S j=0 3=0
U
^
it follows that 00
-,
\\x* -xo\\<^Uj = -—uo.
D
(6.21)
j=0
Using this property one can avoid the a priori assumption that all xn lie in DO- This theorem is a prototype of a large family obtained by assuming additional information on the iterative function /. For example, if we assume that / is continuous, then /(x*) = x* . Furthermore, if we assume more than (6.17), namely, for x,y e D,
then the solution x* is unique in DQ. Using similar arguments, one can also prove the following result. Theorem 6.3.2 Suppose that X Q , X I lies in DQ and B(XQ, Y^^o uj] ^ A)Then the sequence converges to x* G B(x$, X^?^o uj}-
Copyright © 2002 Marcel Dekker, Inc.
172
6.3.1
CHAPTER 6. APPLICATIONS
TO NUMERICAL
ANALYSIS
Newton-Kantorovich-like theorems
The Newton method is the principal tool for solving nonlinear problems. It falls in the class of iterative methods with a very judicious choice of the iterative function /. i.e..
The choice is made such that its derivative in each root of F is zero, as one realizes without difficulty. As a consequence, each root of F is asymptotically stable for the difference equation (6.2). The iterative process becomes then xn+l =xn- F'(xn)-lF(xn)
(6.22)
which for convenience will also be written as F'(x n )Az n + F(xn) = 0. Since the process is of paramount importance in the applications and in its study interesting questions related to the difference equations arise, we shall discuss it in detail. We start with the following preliminary result, on which all the subsequent theorems are based. Lemma 6.3.1 Let F : IR'S —> IRS have a Lipschitz derivative, i.e. \\F>(x}-F>(y}\\<7\\x-yl then \\F(x) - F(y] - F'(x}(y - x}\\ < \^\\x - y\\2.
Proof.
(6.23)
From the mean value theorem one has
F(x) - F ( y ) - F'(x)(y - x} = f\Ff(x + s ( y - x } } - F ' ( x ) ] d s ( y - x);
./o
and then ||F(x) - F(y) - F'(x}(y - x)|| < 7 /' sds\\y - x|| 2 , Jo from which (6.23) easily follows.
D
The main result on the convergence of the Newton method was first proved by Kantorovitch and is usually called the Newton-Kantorovitch theorem.
Copyright © 2002 Marcel Dekker, Inc.
6.3. SEMILOCAL RESULTS
173
Theorem 6.3.3 Assume that, for XQ,X, y G D, where D is a convex subset of1Rs, (1) F is differentiable (2)
in D;
\\F'(x)-F'(y)\\
(3) UF'^o)- 1 !! < 0,
\\xi - .ro|| = \\F'(xQ}^F(x0}\\ = 77;
(4) (5) BXO,
CD.
Then the sequence defined by (6.22) is well-defined and converges to a root x* of F(x) = 0 contained in B(XQ, l//?7). Proof.
For every x G B(XQ, l//?7) we have
A bound of H-F^x)" 1 !) can then be obtained by using the Banach lemma (sec Corollary A. 6. 2), which yields
< From this estimate, it follows that for x G B(XQ, 1 / fij) , Ff (x) is nonsingular and the Newton's iterative function is defined. Moreover,
\\xn+2 - xn + i|| = \\F'(xn+l}-F(xn+l}\\ <
n
1 - ^\\xn+i - x o l l
(6.24)
and (see Lemma 6.3.1) =
||F(x n+ i) - F(xn) - F'(xn)(xn+1 - xn)\\ 1
"
-xn\\2.
(6.25)
Substituting for F(xn+i] in (6.24), we then find \Xn+2 — ^'n+l|| S
ll^n+l ~" xn\\
2 1-
Suppose for a moment that the sequence xn lies in B(XQ,\/j3~y}. then apply Theorem 4.5.4 with n
Copyright © 2002 Marcel Dekker, Inc.
n—l
We can
174
CHAPTER 6. APPLICATIONS
TO NUMERICAL
ANALYSIS
The comparison equation is therefore (6.26)
which can easily be solved if one takes u$ = \(j3^} \ obtaining un — (^7)- 1 2~ n - 1 . Since by (4) ||xi-x 0 || < \ (Pi) -1, the hypotheses of Theorem 4.5.4 arc satisfied, and we can conclude that n.
n
The origin is thus exponentially stable for (6.26). All xn lie in B(XQ, 1//37) because n-l
i
j =l
We may apply the same arguments used before to get,
P-I
p-i ~
X
n+j\
which shows that xn —* x* and also gives the error estimate (6.27) The proof that x* is a root of F(x] follows from 7-1 /
\
j—i/ /
\ /
\
/ T~I/ /
\
,
in/ /
\
F(x n ) = F (x n )(x n + i - x n ) = (F (x0) + F (xn) and then i^(^)||
<
Taking the limit as n —>• oo one has, by the continuity of F, D
Note that if the formula (6.27) were an equality, it would only imply that
lim which states a linear convergence for the method. It is however known that the method has a quadratic convergence. A better estimate can be obtained by assuming that UQ < \(fil}~1 • In fact, the conclusion of Theorem 6.3.3 can easily be obtained because the solution of (6.26) can be given in the
Copyright © 2002 Marcel Dekker, Inc.
6.3. SEMILOCAL RESULTS
175
closed form. When w0 < \(Pl}~1 •> the solution of (6.26) is more difficult to obtain but it can be done. In fact, let us define the new sequence zn by Az n _i — j3jun, z-\ = 0. Then equation (6.26) is transformed into 1 (zn - Z n -l) 2 Zn+l - Zn + ~ , ^
1
Zn
which possesses a first integral, that is, the sequence {zn} satisfies the firstorder equation
where ZQ = /3jUQ. The solution of this equation is zn = 1 -(l-22 0 ) 1 / 2 cothA;2 n , (see Problem 1.29) where A: is a constant depending on the initial condition, given by
(see Problem 6.8 for its determination and for a more refined bound). The solution of the previous equation exists if ZQ < |, and then UQ < ^(f3^}~1 . The case of equality in the above expression has already been examined. Suppose now that UQ < (j3^}~1 . Then it follows that lim zn - 1 - (1 - 2z 0 ) 1/2 - 2*
n—*oo
and since for large n, coth/c2 n ~ 1 + 2e~/c2" , we obtain zn-z*^ 2(1 - 2 Consequently, we get x z±£o
-, - Z 7*^)2 7 —1 °° ( V^n )
o /i — 0 ? n U '/ 2 ' Z(l Z2QJ
which shows that the order of convergence of the sequence zn to the limit point is quadratic. From the comparison principle it follows that II
and hence
Copyright © 2002 Marcel Dekker, Inc.
n+l
n||
Q
( n
n-1)
176
CHAPTER 6. APPLICATIONS TO NUMERICAL
ANALYSIS
which exhibits a much more rapid convergence with respect to (6.27). Moreover, for n > 0,
II n \\X
-
<^ 0IK | | _V 2_^_
n
1 -< ^Z
*
because of the increasing behavior of zn. Summing up these considerations, we have proved a variant of Theorem 6.3.3, which we state below. Theorem 6.3.3 (a) Let z* = 1 — (1 - 220)2 with ZQ e (0, \\ Assume that the hypotheses (1). (2), and (3) and (4) of Theorem 6.3.3 hold. Suppose further that (5) B(XO,±Z*) CD. Then the sequence xn is well-defined and converges quadratically to x contained in B (XQ, ~~ Newton's method is invariant under affine transformations. That is, it is invariant under the transformation G = AF where A is an invertible matrix, whereas the hypotheses in the Newton-Kantorovitch theorem are not. In fact, one has G~1G = F~-1A~1AF = F~1F, while the hypothesis (3) is not invariant under the same transformation. The following two modifications of Theorem 6.3.3 take care of this situation. Theorem 6.3.3 (b) Assume that the hypothesis (1) of Theorem 6.3.3 holds, and suppose that
(2) \\F'(x0)-1F(xQ)\\
\\F'(x0)^(F/(y)-F>(x))\\<w\\x~y\\;
(4) aw < \ •
(5) B(z 0 ,w~V) c D, where z* = 1 - (1 - 2wa)1/2 < I . Then the sequence xn remains in B(XQ,W~IZ*) cally to a root x* of F(x). Proof.
and converges quadrati-
We have =
-F'(xn+l
)~lF(xn+i) l
- -F'(xn+l}- [F(xn) =
+ F(xn+l) - F ( x n ) ]
l
-F'(xn+1r {F(xn} + F'(xn)(xn+l - xn)
fl + /
Jo
•} [F'(Xn + t(xn+l - Xn}} - F'(xn}}dt(xn+l
- Xn] }
)
/' F'(x0r[[F'(xn + t(xn+1 - xn)} o F'(xn}}dt(xn+l~xn}},
Copyright © 2002 Marcel Dekker, Inc.
6.3.
SEMILOCAL RESULTS
177
from which it follows that ||xn+2 - a:n+i|| < \\F'(xn+1}-lF'(xQ)\\\\Xn+l
- Xn\\2.
Considering that by hypothesis (3) we get \\F'(x0rl(F'(xn+1} ~ F'(xQ)}\\ < w\\xn+l - xo|| < z snce - F'(x0rlF'(x0) + F' '(xo)'1 F1P (x 0 ) , if x n +i ^ -B(xo, if^z*), by the Banach lemma it follows that F'(xn+i) is invertible and \\F'(xn+lrlFf(x0)\\
< ::
1-
Finally, we arrive at \\Xn+l
- 2 1-L which is the same as before with j3^ replaced by w. All the previous results then apply, and we are done. D The next theorem supposes the existence of the solution in the subset of 1RS, and therefore it is simpler with respect to the previous one. Theorem 6.3.4 Assume that (1) F : D C }RS —>• IR5, with D convex, is continuously
differentiate;
(2) F'(x) is invertible in D\ (3) \\F'(x)-l(F'(x + tv) ~ F'(x))v\\ < HHI 2 , for all t e [0,1], w > 0 and v £ IRS so that x + v <E D; (4) In D there exist a solution x* and a starting point XQ such that, letting p = \\XQ — x*||, it happens that 2 p< — w
and
B(x*,p) C D.
Then the sequence of iterates remains inside B(x*,p) and converges to x*. Proof.
The proof proceeds by induction. Is is an easy matter to obtain
£n+i - £* = F'(xn)~l (F(x*} - F(xn] - F'(xn}(x* ~ xn)) .
Copyright © 2002 Marcel Dekker, Inc.
178
CHAPTER 6. APPLICATIONS TO NUMERICAL ANALYSIS
Using an argument very similar to that employed in Lemma 6.3.1 and by hypothesis (3) we get l l ^ n + l ~ X || S T ^ H ^ n ~~ X ||
(see Problem 6.2). Assuming that xn G B(x*,p), we have by (4) ii * i ^ w ii n *IMI *ii ^ ii *n x , Y>
/f
.,
IKn+1 ~
I
I T1
l — o II
~~
-T"
I i I 'y
'V
11
^
I 'V
'~f
11
llll^n ~ X II <\ \\Xn — X ||,
which shows that all the iterates are in B(x*,p) and converge to x*. D The next result also shows the familiar quadratic decay of the errors in the Newton method, but it requires more assumptions on the inverse of the Jacobian. Theorem 6.3.5 Assume that F : DQ C IRS —•» IRS and DQ is a convex set. Moreover, suppose that (1) for x G DQ.
F is
(2) for x G DO,
F'(x) is invertible and \\F'(x}-l\\ < ft,
(3) for x,ye Do(4) for XQ G Do.
differentiate,
\\F'(x) - F'(y)\\ <>y\\x-y\\, ||F / (a;o)~ 1 ^(a:o)|| < V, and
(5) a = kfi'yr/ < 1. Then the Newton iterates remain in B(XQ,TQ), where TQ = ?/X!j*Lo a27 and converge to a solution x* of F(x) = 0. Proof.
From \\Xn\-2 — Xn
Jl = I I F'(xn+i )~lF(xn+i ) II <
fl\\F(x n + i
) II
and the estimate of ||F(x n -fi)|| obtained in the previous theorem, we get lkn+2 - xn+i\\ < ~\\xn+i - x- n || 2 .
(6.28)
The comparison equation is therefore Un+l = -^-U2n,
UQ - T],
(6.29)
whose solution is un = na2"~l. Now applying comparison Theorem 1.6.1 we arrive (see Problem 6.4) at \\xn - xn+i | < un = r/a2""1.
(6.30)
Since a < 1, the trivial solution of (6.29) is exponentially stable, and hence it follows that xn is a Cauchy sequence. Furthermore. wre also have \\xn — XQ\\ < X]j^oWj = i\), which means that all the iterates remain in S(xo,r'o) and converge to x*. d
Copyright © 2002 Marcel Dekker, Inc.
6.3. SEMILOCAL RESULTS
179
Corollary 6.3.1 The error \\x* — xn\\ satisfies the inequality \\x* - xn\\ < en\\xn - x n _i|| 2 ,
/or n = 1,2,3, . . .
(6.31)
As an application of Theorem 1.6.6, let us consider the two-step iterative method
xk+i = G(xk,Xk-i}. Theorem 6.3.6 Suppose that G : D x D c I R s x I R s —> D and on some closed set DQ C D, the following inequality holds true \\G(xiy)-G(y,z)\\ 1. Then the iterates converge to the unique fixed point of G(x] — G(x, x). Proof.
Setting yk = \\%k+i ~ xk\\i
we
obtain
yk+i < By Theorem 1.6.6, we then get yk < uk, where u^ is the solution of auk + fluk-i, which is Uk — CQZ^ + c\z%, where z\ and 22 axe the roots of z2 — az — j3 — 0. Because of the assumption on a and ,3, the two roots z\ and z<2 are less than 1 in modulus, which means that the zero solution of the comparison equation is exponentially stable. Using similar arguments as in the previous theorems, we can conclude that the sequence x^ is a Cauchy sequence. Since
+ \\G(x*,xk.l}-G(xk,xk-l < @\\x* -xk_i\\ +a||x* the convergence of xk shows that xk+i —* G(x*}. The uniqueness of x* now follows very easily, and the proof is complete. D
6.3.2
Effect of perturbations
The effect of perturbations on the iterative methods are of two different kinds. If the perturbations are small and tend to zero as n tends to oo, then the theorems on total stability will ensure that the fixed points continue to have asymptotic stability. In numerical analysis, however, perturbations may remain bounded for all n (roundoff errors, for example). In this case, a more convenient concept is practical stability. If an iterative procedure is practically stable, the sequence of iterates will not tend to the solution x* but to a ball B(x*,5] surrounding x* . Inside 5, the solution may oscillate,
Copyright © 2002 Marcel Dekker, Inc.
180
CHAPTER 6. APPLICATIONS
TO NUMERICAL
ANALYSIS
but what is important is that it never leaves B. Of course it would be nice to have B as small as possible. Let us consider for example the iterative method •-^n+1 — J\'E"n);
"^no
^'O-
^D.oZJ
WTe assume that for x, y e DQ C IRS,
\\f(x)-f(y}\\
(6.33)
with a < I . Suppose that the errors in the computations perturb (6.32) by a bounded perturbation Rn, that is xri+} - f ( x n ) + Rn,
xno = x0,
(6.34)
with xn,xn G DQ for all n. Then Theorem 4.11.2 can be applied. The conclusion is that the difference of the two solutions \\xn — xn\\ will not exceed R/(l — a), wThere R = supRn. The condition that all the balls B (xn. j™;) must be contained in DQ may be very restrictive. In fact if a is close to one, it follows that these balls are very large and the method is considered a bad one.
6.4
Miller's, Olver's, and Clenshaw's Algorithms
The situation that we shall analyze in this section and in the next is in a certain sense the opposite of that treated in the previous ones. In fact, there the limit point was important (and indeed in the theory of iterative processes the "solution1' is the limit point), where the intermediate values zn are considered an unavoidable noise. Here the limit point will riot be important (generally it will not exist) and the important part (the solution) will be a finite subset of the sequence itself. These problems are very typical in that part of numerical analysis which concerns the study of special functions, orthogonal polynomials, quadrature formulas, and numerical methods for ordinary differential equations. Consider the homogeneous scalar linear equation (see (2.12)) of order k Lyn = 0.
(6.35)
It has k linearly independent solutions /i(n), f2(^)1 • • • •> f k ( n ) - Suppose that the solution yn — 0 (n > 0) is unstable, and 11111^7-7=0^
i
= 2,3, . . . , f c .
(6.36)
When this happens, the solution f i ( n ) is said to be minimal. The problem is how to find f\ or a multiple of it. Even if we know the exact initial
Copyright © 2002 Marcel Dekker, Inc.
6.4.
MILLER'S, OLVER'S, AND CLENSHAW'S ALGORITHMS
181
condition that generates the minimal solution, small errors in the calculations will, as usual, lead us to solve a perturbed equation Lyn = e n , whose solution (see (2.18)) will contain all the fi(n). This will destroy the process (see for example [69, 180]) because the fi(n) (i > 2) grow faster than \f\(n}\. The same problem appears, of course, in the nonhomogeneous case, where it is riot excluded a priori that there exists a bounded solution even if all the fi(ri) are bounded. One is often interested in following this solution. Consider, for example, the following first-order equation yn+i -(n + l}yn = -1,
y0 - e.
(6.37)
The solution of the homogeneous part is yn = en\, while the complete solution is
which is bounded for n —> oo. If one tries to follow this solution, a small error in the initial data e (always existing because it has infinite digits), will be amplified by the factor n\. A simple method is to find the way to look for the solution of the problem backwards (for n —> oo). because in this case the origin becomes asymptotically stable for the homogeneous equation and the errors will be damped. This idea indeed works, as we will show in a simple case, which is the original problem to which it was applied (see [69] for references). Consider Lyn = anyn+i + bnyn 4- c n 2/ n -i, (6.38) where a n ,c n 7^ 0 for all n, and the nonhomogeneous problem Lyn = 9n-
(6.39)
Suppose that two linearly independent solutions 0n and 4>n of the homogeneous equation are such that 0n is minimal, that is lim — = 0.
n-oo^ n
(6.40)
Of course all solutions that are multiple of 4>n are minimal and vice versa. That means that only one appropriate initial condition is needed to determine the minimal solution we are interested in. The general solution of the nonhomogeneous problem is then yn — Cl4>n + C2$n + 2/n,
(6-41)
where yn is a particular solution that will be supposed minimal too; that is, lim ^ = 0
n-oo -
Copyright © 2002 Marcel Dekker, Inc.
(6.42)
182
CHAPTER 6. APPLICATIONS
TO NUMERICAL
ANALYSIS
and £o = 0. We are interested in the solution
yn = ~n + yn.
(6.43)
9o
A class of methods are based on the following result. Theorem 6.4.1 Consider the boundary value problem
where N is a positive integer, and suppose that conditions (6.40) and (6.42) are verified. Then the problem (6-44) has a solution and moreover for fixed n, y(n -> yn as N -> oo. Proof. Since yn is a particular solution of the nonhomogeneous equation, any other solution of the same equation will be (6.45) The boundary conditions give for c\ , . 4>N
It follows that C} n
and c2
and c2 Co
the values
__ —
tend to zero as N —> oo and the claim follows.
Note that the condition yN — 0 can be replaced by y^- = k where k is any fixed value (for example an approximation of 7/yv, if available). In the homogeneous case yn can be obtained starting from yN = 0, yjv-i
=
1>
anc
^ then using the difference equation backwards
iin-i — ~~r~yn+i ~ ~yn i obtaining a value y^
different from y^. Multiplying the sequence by the one
scale factor 1/0/2/0 ' o^t^118 yn • This is Miller's algorithm. A disadvantage of this algorithm is that one does not know a priori which value of TV must be used. A modified version of it which works in the nonhomogeneous case is also obtained by considering that the boundary value problem (6.41) is equivalent to the system of equations Ay(N] = b,
Copyright © 2002 Marcel Dekker, Inc.
(6.46)
6.4.
MILLER'S, OLVER'S, AND CLENSHAW'S ALGORITHMS
183
where 0
\
A= CN-I
a/v-2 6/v-i J
and y(N)
y
, (N) (N) —_ \y\ • • • UN-I) ' ]T
Some clever methods to solve this system can be used to avoid the growth of the errors (see [33, 34]. By solving the system in an appropriate way, it is also possible to determine the best value of ./V (Olver's algorithm). Similar in motivation to the previous algorithm and indeed related to it is the following, known as Clenshaw's algorithm. Here the problem is to evaluate the sum N v—«.
bnUr
where bn is a given sequence and un is supposed to satisfy a linear difference equation Lun = 0. The algorithm uses the result stated in Theorem 2.2.6, that is, one considers the transposed equation (j = 1,2, . . . ,
= 0
L yn, obtaining /v
jf
fc-i
]T bnun = Yl yj ^Pi(j - k)uj-i-
n=0
j^O
i=0
Suppose, for example, we want to compute f^ = Y^n=o bnTn(z), where Tn(z) are Chebyshev polynomials satisfying the equation Tn+\ — 2zTn + T n _i — 0. The transpose equation is yn - 2zyn+i + yn+2 = bn,
n = N, N - 1 , . . . , 0
2//V+1 = 2//V+2 = 0, which can be solved recursively obtaining y\ and y^. From the quoted result it then follows /N ~ yo + y\\T\(z} — 2zTo(z)} — yo — zy\. It is interesting to note that the sum can be obtained without the knowledge of the polynomials Tn(z) except for TQ(Z) and T\(z} and also the reduction of the operations involved.
Copyright © 2002 Marcel Dekker, Inc.
184
6.5
CHAPTER 6. APPLICATIONS
TO NUMERICAL
ANALYSIS
Boundary Value Problems
The topic discussed in this section arises typically in that part of numerical analysis that concerns the approximation of solutions of ODEs and PDEs. We shall consider, for simplicity, the autonomous case. The more general case is not, in principle, more complicated. Let us consider yn+i - 25yn + yn-i = gn,
yo = a,
yN = /?,
(6.47)
with 5 > 1, TV » 1. This problem can be solved in many different ways. It can be rewritten as a system of N — 1 equations similar to (6.46), and then solved by a suitable factorization of the coefficient matrix. This procedure has already been discussed in Chapter 5. Here we prefer to remain in the difference equation framework. We only outline that the LU'factorization would lead to the same kind of equations derived below. The characteristic polynomial of (6.47) has two positive roots, one less than one and the other greater than one. This means that the origin is not asymptotically stable for the homogeneous part and this, as usual, creates difficulties because the errors may be amplified. One way to avoid this is to transform the linear second-order equation (6.47) into nonlinear first-order equation (see Problem 6.12). xn+i = ^—, zo — xn
(6.48)
*»' = w^-
(6 49)
and
-
Furthermore, one verifies that 2/n+i = xnyn + zn.
(6.50)
The solution is obtained by computing recursively the sequences {xn}, {zn} utilizing (6.48) and (6.49) with x\ = 0 and z\ — a up to n — N — 1 and then computing {yn} using (6.50) from n — N to n = 1 with y^ — (3. The advantage is that (6.48) has a limit point asymptotically stable and the sequence xn converges rnonotonically to this point, while the sequences (6.49) and (6.50) remain bounded. Theorem 6.5.1 If XQ = 0,5 > 1, the sequence {xn} converges rnonotonically to the first root p of the characteristic polynomial of (6-47) and, moreover, 0 < xn < p.
Copyright © 2002 Marcel Dekker, Inc.
6.6.
MONOTONE ITERATIVE METHODS
185
Proof. The critical points of (6.48) are the roots of the equation xn — 26x + 1 = 0, which is the characteristic polynomial of (6.47). Let p be the root less than one. Changing the variable hn = p(p - xn), equation (6.48) becomes 2
One shows at once (for example using the Liapunov function Vn = h^ that the half line h > 0 is contained in the asymptotic stability region of the origin, and moreover 0 < hn < p2, from which it follows that -p < xn ~ P < 0.
O
(6.52)
The method just described is known under many names, for example Sweep method (see [41]), recessive method, discrete Riccati method, and so on. It is also related to Bellman's method of invariant imbedding.
6.6
Monotone Iterative Methods
In this section, we shall develop a general theory for a broad class of monotone iterations. This class of iterations includes Newton's method as well as a family of methods, which are called Newton-Gauss-Seidel processes that are obtained by using the Gauss-Seidel iteration on the linear systems of Newton's method. As before, we are interested in finding the solutions of 0 = f(u),
(6.53)
where / e C[IR n ;IR n ]. Let us first split the system (6.53) as 0 = /»(ui,M w ,[4J,
(6-54)
where, for each z, 1 < i < n, pi + qi = n — 1 and u — (w^, [u]pi, [u]qi). Let v, w G IRn be such that v < w. Then v, w are said to be coupled quasi lower and upper solutions of (6.53) if 0 < fi(vi,[v]pi,[w]qi), 0 > fi(wi,[w]pi,[v]Qi). Coupled quasi-extremal solutions of (6.53) can be easily defined. Vectorial inequalities mean the same inequalities between the components of the vectors. We arc now in a position to prove the following result.
Copyright © 2002 Marcel Dekker, Inc.
186
CHAPTER 6. APPLICATIONS TO NUMERICAL
ANALYSIS
Theorem 6.6.1 Assume that f G C[IR n ,IR n ] and possesses a mixed quasimonotone property. That is, for each z, fi(ui, [w] pi , Hg,;) is nondecreasing in [u]p., and nonincreasing in [u]qi. Suppose further v,w are coupled quasi lower and upper solutions of (6.53) and
whenever v < u < u < w and Mi > 0. Then there exist monotone sequences {un},{wn} such that vn —* p, wn —» r as n —> oo and p.r are coupled minimal and maximal solutions of (6.53) such that v
For any 771,772 G [v.w] = [u G IRn : v < u < w}, consider the
Ui = M^fifai,
[r)i]jri, \rn\qi) + mi,
i = 1, 2, . . . , n.
Clearly u can be uniquely defined, given 771,772 G [v,w]. Therefore, we can define a mapping A such that .A [771, 772] — u. It is easy to show that A satisfies the properties: (i) 0 < A[v,w],
0 > A[w.v}]
(ii) A is mixed monotone on [TJ,U']. Then the sequences {vn}. {wn} with VQ — v. WQ — w can be defined as follows
w = Furthermore, it is clear that {vn}, {wn} are monotone sequences such that v < Vn < wn < w and consequently limT^-^oo vn = p, limn^oo wn = r exist and satisfy the relations
By induction, it is also easy to show that if ( u i , U 2 ) is a coupled quasi solution of (6.53) such that v < u\,u^ < w and consequently (p, r) are coupled quasi solutions, the conclusion of the theorem follows and the proof is complete. D If / does not possess the mixed quasi monotone property, we need a different set of assumptions to generate monotone sequences that converge to extremal solutions. This is the content of the following results. Theorem 6.6.2 Assume that (i) there exist v, w G IRn with v < w such that 0 < f(v).. 0 > f ( w ) :
Copyright © 2002 Marcel Dekker, Inc.
6.6. MONOTONE ITERATIVE METHODS
187
(ii) there is a n x n matrix M such that f(y)-f(x)>-M(y)(y-x), whenever v < x < y < w. Then the sequence {wn}, given by wn + Bnf(wn),
(6.55)
where Bn is any nonnegative subinverse of M(wn), is well defined, and {wn} is monotone nonincreasing such that limn_+00 wn — r. If there exists a nonsingular matrix B > 0 such that lim inf n_>oo Bn > B, then r is a maximal solution of (6.53). Proof. Set v — VQ and w — WQ. From BO > 0 and /(WQ) < 0 it follows that w\ < WQ. Using the fact that BQ is a subinverse of M(WQ), we find for any u e [v,w], u + B0f(u)
= wi - (WQ-U)- B0[f(w0) - f(u)] < wi - [I - BOM(WQ)}(WQ -u}<wi.
Hence, in particular. VQ < VQ + 5o/(vo) < w\- Similarly, we obtain f ( w i ) < f ( w Q ) + M(w0 - wi) = [I- M(wQ}B0}f(w0)
< 0.
Proceeding similarly we see by induction that wn-i>wn>vo,
f(wn) > 0,n = 1,2,
----
(6.56)
Consequently, as a monotone nonincreasing sequence that is bounded below, {w} has a limit r > VQ. If u is any solution of (6.53) such that u 6 [v, w}, then u = u + BQ/(U) < wi, then by induction u < wn for all n. Hence u < r. Finally, the continuity of / and the fact Iiminf n 5 n > B, where B > 0 is nonsingular, (6.56) yields 0 = limmi[wn+i - wn] = ]immi(Bnf(wn)) n—»oo
=
n—KX>
-(lim inf B n ) f ( r ] > -Bf(r) > 0,
which implies /(r) = 0 completing the proof. D By following the argument of Theorem 6.6.2, one can prove the next corollary. Corollary 6.6.1 Let the assumptions of Theorem 6.6.2 hold. Suppose that M(y] is monotone nonincreasing in y. Then the sequence {vn} with VQ = v, given by vn+i = vn + Bnf(vn), is well-defined, m.onotone nondecreasing such that limr^oo vn = p, and is the minimal solution of (6.53).
Copyright © 2002 Marcel Dekker, Inc.
188
CHAPTER 6. APPLICATIONS TO NUMERICAL
ANALYSIS
The case of most interest is when p — r because then the sequences {vn},{wn} constitute lower and upper bounds for the unique solution of (6.53). The following uniqueness result is of interest.
Theorem 6.6.3 Suppose that f(y)-f(x) 0. Then if (6.53) has either maximal or minimal solution in \v,w\, then there are no other solutions in
Proof. Suppose r G \v, w\ is the maximal solution of (6.53) and u e [v.r] is any other solution of (6.53). Then 0=
f(r)-f(u)
Since N(u)~l > 0, it follows that r > u and hence r = u. A similar proof holds if the minimal solution exists. The proof is complete. D
6.7
Monotone Approximations
Consider the problem of finding the solutions of
Ax = /(z),
(6.57)
where A is an n x n matrix and / € C[IRn, IRTO], which arises as finite difference approximation to nonlinear differential equations. If A is nonsingulai, writing F(x) — f ( x ) — Ax = 0, one can study existence of multiple solutions by employing the method of upper and lower solutions and the monotone iterative technique described in Section 6.6. In this section we extend such existence results to equation (6.57) when A is singular. For convenience let us split the system (6.57) and write it in the form (>hOi = / i ( u » , M P i , M 9 i ) ,
(6.58)
where for each z, 1 < i < n, u — (uj.. [u]pi, [u]qi} with pi + ql = n— 1 and (Au)i represents the ith component of the vector Au. As before, a function / 6 C[]R'S, IR'S] is said to be mixed quasi-monotone if, for each z, /,• is monotone nondecreasing relative to u Pi components and monotone nonincreasing with respect to [u]qi components. Let v,w G IRS be such that v < w. Then v,w are said to be coupled quasi lower and upper solutions of (6.57) if
Coupled quasi-extremal solutions and solutions can be denned with equality holding. We are now in a position to prove the following result.
Copyright © 2002 Marcel Dekker, Inc.
6.7. MONOTONE APPROXIMATIONS
189
Theorem 6.7.1 Assume that (i) / G C[IRS,IRS] and f is mixed quasi-monotone; (ii) v.w are coupled quasi lower and upper solutions of (6.57); (iii) fi(v.i, [u]pi, [u]qi) - f i f a , [u] pi , [u]qi) > -Mi(ui - Uj) whenever v 0, for each i; (iv) A is an n x n singular matrix such that A + M = C is nonsingular where M is a diagonal matrix with Mi > 0 and C"1 > 0. Then there exist monotone sequences {vn}, {wn} such that vn ^> p,wn -* r as n —> oc and p:r are coupled quasi- extremal solutions of (6.57) such that if u is any solution of (6.57), then v] since the matrix C is nonsingular. Furthermore, we have Av < F(v,w),Aw > F(w.v], and F is mixed monotone. Consequently we can define a mapping T such that T[?/, \ J L - U and easily show, using the fact that C~l > 0, that (a) v < T[v,w],w > T[w,v]] (b) T is mixed monotone on [v.w]. Then the sequences {vn},{u}n} with VQ = V,WQ = w can be defined as follows: = T[vnj wn], wn+i = T[wnj vn]. Furthermore, it is evident from the properties (a) and (b) that {vn}, {wn} are monotone such that
^o < ^i < • • • < vn < wn < . . . < w\ < WQ. Consequently, lim vn = p, lim wn = r
n—>oo
n—>oo
exist and satisfy the relations (6.60) By induction, it is also easy to prove that if (111,1*2) is a coupled quasisolution of (6.57) such that v < 111,112 < w, then vn < u\,u<2 < wn for all n and hence (p, r] are coupled quasi-extremal solutions of (6.57). Since
Copyright © 2002 Marcel Dekker, Inc.
190
CHAPTER 6. APPLICATIONS
TO NUMERICAL
ANALYSIS
any solution u of (6.57) is a coupled quasi-solution, the conclusion of the theorem follows and the proof is complete. D The case of most interest is when p — r, because then the sequences {vn},{wn} constitute lower and upper bounds for the unique solution of (6.57). The following uniqueness result is thus of interest. Corollary 6.7.1 If in addition to the hypotheses of Theorem 6. 7.1, we assume for each i, that f i ( X i , [x] pi , [y]qi) - f i ( y i , [y] p ., [*]J < [B(x - y}}^
(6.61)
where B is an nxn matrix such that (A — B] is nonsingular and (A — B)~l > 0. Then u = p = r is the unique solution of (6.57) such that v < u < w. Proof.
Since p < r. it is enough to prove that p > r. We have by (6.61)
\A(r ~ p}]r < /t(r?, [r] p?; , \p]qi) - ft(Pi, [p] pl , [r]9J < (B(r - p}^ Consequently, (A-B)(r- p) < 0. which implies r < p. D If qi = 0 for each i, then Theorem 6.7.1 shows that r, p are maximal and minimal solutions of (6.57) and Corollary 6.7.1 gives the unique solution. If / does not possess the mixed quasi-monotone property, we need a different set of assumptions to generate monotone sequences that converge to a solution. This is the content of the next result. Theorem 6.7.2 Assume that (iv) of Theorem, 6.7.1 holds. Further, suppose that v. w 6 IRn such that v < w, Av< f(v)-B(w-v), -B(x
Aw> f ( w ) + B(w-v),
~y}< f(x] - f(y) < B(x - y]
(6.62) (6.63)
whenever v < y < x < w, B being an n x n matrix of nonnegative elements. Then there exist monotone sequences {vn}, {wn} that converge to a solution u of (6.57) such that
provided that (A — B) is a nonsingular matrix. Proof.
We define (6.64)
Copyright © 2002 Marcel Dekker, Inc.
6.7. MONOTONE APPROXIMATIONS
191
It is easy to see that F(y, z) is mixed monotone and -B(z ~z}< F(y, z) - F(y, z} < B(y - y}
(6.65)
whenever z, z,y,y £ [v, w] and z < z, y < y. In particular, we have F(y,z)-F(z,y)
= B(y-z).
(6.66)
From (6.64) we obtain -B(w-v)
Similarly Aw > F(w,v). Finally, it follows from (6.64) that F(x,x) = f ( x ) . Consider now for any 77, /i 6 v,w],
the linear system given by Cu = G(rj,n) where C — A -f M, M being the diagonal matrix with Mi > 0 and for each i, G,(r),Li) = Fi(Ti,fj.) + Mi7ii. Proceeding as in proof of Theorem 6.7.1, we arrive at Ap = F(p, r),
Ar = F(r, p ) .
Using (6.65), we see that A(r -p} = F(r, p] - F(p, r) = B(r - p) and this implies u = r — p is a solution of (6.57). The proof is complete. D
Corollary 6.7.2 If in addition to the hypotheses of Theorem 6.7.2 we also have f(x)-f(y)
then u is the unique solution of (6.57).
Copyright © 2002 Marcel Dekker, Inc.
192
CHAPTER 6. APPLICATIONS Proof.
TO NUMERICAL
ANALYSIS
If u is another solution of (6.57), we get A(u - u] = f ( u ) - /(u) < C(u)(u - u),
which yields u — u. Similarly, u < u and hence u is the unique solution of (6.57). D Equations of the form (6.57) arise as finite difference approximations to nonlinear partial differential equations as well as problems at resonance where the matrix is usually a singular, irreducible M-matrix. For such matrices the following result is known. Theorem 6.7.3 Let A be an n x n singular irreducible, M-matrix. Then (i) A has rank (n — I}; (ii)
there exists a vector u > 0 such that Au = 0;
(iii) Av > 0 implies Av = 0; (iv) for any nonnegative diagonal matrix D, (A + D] is an M-matrix and (A + D}~1 is a nonsingular M-matrix, if da > 0 for some i.l < i < n. As a consequence of Theorem 6.7.3, if we suppose that A in (6.57) is n x n singular irreducible, M-matrix, then (A + M) is a nonsingular A/matrix. Therefore assumption (iv) of Theorem 6.7.1 holds since a nonsingular M-matrix has the property that its inverse exists and is greater than or equal to zero. Furthermore, in some applications to partial differential equations, the function / in (6.57) is of a special type, namely /(«) = (/i (HI), 72(^2)5 • • • ) fn(un))- In this special case lower and upper solutions such that assumption (ii) of Theorem 6.7.1 holds whenever we have limsup f i ( u i ) S i g ( u i ) < 0 for each i.
I-U/HOO
In fact, by Theorem 6.7.3, there exists a £ > 0 such that KerA — Span(£), and hence we can choose a A > 0 so large that /(AC) < 0 and /(-A£) > 0. Letting v = —\£,w = A£ we see that v < w.Av < f ( v ) and Aw > f ( w ) . Assumption (iii) is simply a one-sided Lipschitz condition, and in assumption (6.61) B is a diagonal matrix.
6.8
Problems
6.1 Show that p(G'(x*)) = 0 where G(x] = x - (F1 ( x ) ) ~ l F ( x ) and a:* is a simple root of F ( x ) — 0.
Copyright © 2002 Marcel Dekker, Inc.
6.8. PROBLEMS
193
6.2 Show that, under the hypotheses of Theorem 6.3.4, one has HF'(x)- 1 (F(y) - F(x) - F'(x}(y - x)) || < |||x - y\\2. 6.3 Show that if in the Newton-Kantorovitch theorem one supposes llF'(x)"" 1 1 bounded for all x 6 H n , then ||xn — x*|| < c||x n _i — x*||2. 6.4 Show that in the hypothesis of Theorem 6.3.5, the error satisfies the inequalities (6.30) and (6.31). 6.5 Suppose that xn e IR s ,n > 0 and ||Axn|| < £*""_ ||Axn-i||, where £ n , is a positive converging sequence with to = 0 and limn^oo tn = t*. Show that xn converges to a limit x*. 6.6 As in the previous exercise suppose that ||Axn|| < A*n (||Axn-i||)7 71, — 1
with ^J
< 1. ZQ-1Z2
6.7 Show that the solution of zn+\ = l_2z n is given by zn = 1 — (1 — 2;io) 1//2 coth(A;2 n ), with k appropriately chosen. 6.8 Find the constant k in the previous problem and deduce that for Newton's method one has:
where
6.9 Obtain the result of Theorem 6.3.3 by considering the first integral vvith 20 = \- (Hint: in this case the first-order equation becomes zn+\ = 6.10 Consider the iterative method xk+i = fJ.[XG(xk) + (1 -
X)G(xk,xk-i)]
where A, /u e [0, 1] and G, G are defined as in Theorem 6.3.6. Show that // and A can be chosen such that the convergence becomes faster. 6.11 Solve equation (6.47) directly and show the growth of the errors. 6.12 Obtain the relations (6.48), (6.49), and (6.50). 6.13 Show that the sweep method is equivalent to the Gaussian elimination of the problem Ay — g when the problem 6.47 is stated in vector form (see (6.46)).
Copyright © 2002 Marcel Dekker, Inc.
194
6.9
CHAPTER 6. APPLICATIONS TO NUMERICAL
ANALYSIS
Notes
The discussions of Sections 6.1 to 6.3 have been introduced only to show examples of application of difference equations to numerical analysis. More details can be found in the excellent books by Ortega and Rheinboldt [140] and Ostrowsky [142]. Theorem 6.3.3 is a simplified version of the original one, see [137, 140], while the estimates given before 6.3.5(b) are new. For other estimates of the error bounds for the Newton's method and Newtonlike methods, see also Meil [124, 125], Yamamoto [184], and Potra [154]. For the effect of errors in the iterative methods, see Urabe [176], A very large source of material on the problem discussed in Section 6.4 can be found in Gautschi's review paper [69]. Theorem 6.4.1 has been taken from Olver [136]. A detailed analysis, in a more general setting, of the Miller's algorithm is given in Zahar [185]. For an extension to three terms matrix equations, see Ahlbrandt and Patula [11]. Applications to numerical methods for ODE's can be found in Cash's book [33], while a large exposition of applications as well as theoretical results are in Wimp's book [180] (see also Mattheji [120]). The sweep method can be found in the Godunov and Ryabenki's book [76], where it is also presented for differential equations. An improvement of the Olver's algorithm can be found in Van der Cruyssen [55]. An application of the "sweep" method to the problem of Section 6.4 can be found in Trigiante and Sivasundaram [175]. The contents of Section 6.6 are adapted from Ladde and Vatsala [66] and Ortega and Rheinboldt[139, 140]. The material of Section 6.7 is taken from [106], but see also [98, 101, 66].
Copyright © 2002 Marcel Dekker, Inc.
Chapter 7
Numerical Methods for Differential Equations
7.0
Introduction
Numerical methods for differential equations are one of the very rich fields of application of the theory of difference equations where the concepts of stability play a prominent role. Because of the successful use of computers to solve difficult problems arising in applications such as multiscale problems, the connection between the two areas has become increasingly important. Furthermore, in a fundamental work Dahlquist emphasized the importance of stability theory in the study of numerical methods of differential equations. In this chapter we shall consider some of the most relevant applications. In Section 7.1 we discuss linear multistep methods again and show that the problem can be reduced to the study of total or practical stability when the roundoff errors are taken into account. In Section 7.2 we deal with the case of a finite interval where we shall find a different form of Theorem 2.8.3. Section 7.3 considers the situation when the interval is infinite restricting in the linear case. The nonlinear case when the nonlinearity is of monotone type is investigated in Section 7.4, while in Section 7.5 we show how one can utilize nonlinear variation of constants formulae for evaluating global error. Sections 7.6 and 7.7 are devoted to the extension of previous results to partial differential equations via the method of lines, which leads to the consideration of the spectrum of a family of matrices in order to obtain the right stability conditions. The problems given in Section 7.8 complete the picture. 195 Copyright © 2002 Marcel Dekker, Inc.
196
7.1
CHAPTER
7. NUMERICAL
METHODS
Linear Multistep Methods
We have already seen that a linear multistep method (LMM), which approximates the solution of the differential equation
y' = f(t,y)>
y(*o) = i/o,
(7.1)
is defined by p(E)zn-h
= 0,
(7.2)
where p(E) and cr(E) are defined in Section 2.8. For convenience we shall suppose that / is defined on [0, T] x H and it is continuously differentiable. The general case can be treated similarly except for some notational difficulties. The values of the solution y(t) on the knots tj = to + jh satisfy the difference equation p ( E ) y ( t n ) - ha(E)f(tn, y ( t n } } = rn
(7.3)
where rn is the local truncation error. The global error
en = y(tn) - zn
(7.4)
then satisfies the difference equation p(E)en - ha(E] ( f ( t n , zn + en] - f(tn, zn)) = rn.
(7.5)
In practice, equation (7.2) is solved on the computer and instead of (7.2) one really solves the perturbed equation p(E}zn - ha(E)f(tn, zn) = e n ,
(7.6)
where en represent the roundoff errors. The equation for the global error then becomes p(E}en - ha(E] ( f ( t n , zn + en] - /(t n , *„)) = wn,
(7.7)
wn = rn - en-
(7.8)
where It is worth noting the different nature of the two kinds of errors. If the method is consistent, rn depends on some power of h, greater than one, while en is independent on such a parameter (it depends on the machine's precision). For simplicity, we assume that en is bounded. The equation (7.7) can be considered as the perturbation of the equation p(E)en - ha(E] ( f ( t n , zn + en] - f ( t n , zn}} = 0, which has the trivial solution en = 0.
Copyright © 2002 Marcel Dekker, Inc.
(7.9)
7.1. LINEAR MULTISTEP METHODS
197
The problem is then a problem of total stability or of practical stability if the roundoff errors en are taken into account. One needs to study equation (7.9) and verify whether the zero solution is uniformly asymptotically stable and then apply, for example, a theorem similar to 4.11.1 in order to get total stability. This has been done indirectly in the linear case. Recently some efforts have been devoted to certain nonlinear cases. The techniques used to approach the problem are essentially of two types. One approach uses the Z-transform (with arguments similar to those used in the proof of Theorem 2.8.3 (see for example [132]) while the other transforms the difference equation of order A: to a first-order system in R n . We shall follow the latter approach. Let us put f(tn, zn + en) - f(tnj Zn) - cnen.
(7.10)
Equation (7.7) can be written (we take ak = 1) as fc-i en+k - hfikcn+ken+k + ^(oij - h(3jCn+j}en+j
- wn.
j=0
or if we suppose that 1 — h/3kcn+k ^ 0 for all values of indices, as
en+k
_ -
- >
-
——
w
-en+j
— h/3kcn+k
k-l _ v~^ j n+j - 2_^ j=0 j=0 fc-1 k-l . , », V" ^(")
1 — h(3kcn+k
k-l
E
E
w
a e
i
aj
j
n
1
_
j=0
"^
where , n _
1 - h(3kCn+k
By introducing the fc-dimensional vectors
En
—
Copyright © 2002 Marcel Dekker, Inc.
e
ri K>
^
198
CHAPTER 7. NUMERICAL
METHODS
and the matrices /
0
0
\
0
(7.12)
1 -ttfc-1 I
v/ o
0
\
(7.13) (n)
\ \
equation (7.7) reduces to the form pi
/ A
•t-^n+l — I71
hBn}En + W.
(7.14)
The problem is now to study total stability for the zero solution of this equation, which depends crucially on the nonlinearity of / contained in the matrix Bn. Historically this problem has been studied under some simplifying hypothesis such as infinite precision arithmetic, single-scale problems, multiscale problems around an asymptotic stable critical point, etc. Each of the above assumptions leads us to important simplification of equation (7.14). For example the infinite precision hypothesis leads to assume that \\Wn\\ —> 0 as h —> 0. Moreover, when h is kept fixed, single-scale problems have N — T/h not very large and the opposite is true in the case of multiscale (stiff) problems. Since the latter circumstance is very similar to the case where T is infinite, for sake of simplicity wre shall group the above assumption under the following categories: finite interval, infinite interval with / linear, and infinite interval with / of monotone type.
7.2
Finite Interval
If the interval of integration (to, to + T) is bounded and the precision is infinite, we take h = ^ and n = 0 . 1 , . . . ,7V. This implies that h becomes smaller arid smaller as N increases and the quantity hBnEn. like the term W n , can be considered as small perturbation of the equation En+l = AEr
(7.15)
The eigenvalues of the matrix A are roots of the characteristic polynomial p(z) and the requirement of the stability of the null solution of (7.15) is equivalent to the notion of 0-stability. We cannot assume asymptotic stability of the zero solution and apply Theorem 4.11.1 because one of the
Copyright © 2002 Marcel Dekker, Inc.
7.2. FINITE
INTERVAL
199
conditions of consistency imposes p(l] = 0, which means that at least one of the eigenvalues of A is on the boundary of the unit disk in the complex plane. The analysis of the stability of the zero solution of (7.14) needs to be done directly. This is not difficult if we use (4.38). In fact n-l n
(7.16)
En = A E0 3=0
Taking the norms, we get
\\En\\ < 3=0
Now suppose that the zero solution of (7.15) is stable (that is demanding that the method is 0-stable). This means that the spectral radius of A is one and moreover the eigenvalues on the boundary of the unit circle are simple. It follows then that the powers of A are bounded. Let us take for simplicity that || A7' || < I . j = 1 , 2 , . . . , TV (see also Theorem A.6.2). The inequality (7.16) becomes
\\EN\\ < Poll + £ j=o
h\\BJ\\\\EJ\\).
By Corollary 1.6.2 we obtain <
\\E0\\
<
\\E0\\e
,|| exp h
\\BT\
(7.18) 3=0
where L — maxi< T
(7.19)
Then (7.18) becomes \\EN\\ <\\E0\\eLT+eehL~li Usually the initial points are chosen such that lim U-Ebll = 0.
N—>oo
Copyright © 2002 Marcel Dekker, Inc.
(7.21)
200
CHAPTER 7. NUMERICAL
METHODS
Hence, taking the limit as h —» 0, we have lim \\EN\\ < v(eLT - 1) lim € *L T ^ . /i-o " "~ /i-+o e^ - 1 from which it is clear that if e 7^ 0 (i.e. when the roundoff errors are considered), the right-hand side is unbounded. It is possible to derive for H-EWII a lower bound with a similar behavior. This means that the zero solution is not practically stable. If the roundoff errors are not considered, and if the method is consistent (sec Definition 2.8.1) then e = 0 and r(h) — O(hp+l),p > I and therefore lim||£ n ||=0.
/i—>0
(7.22)
This implies that the null solution is totally stable. It is possible to give a more refined analysis of (7.16) by considering in the limit only the components of the matrix A (see Appendix A), corresponding to the eigenvalues on the unit circle, that have nonzero limit for n —•> oo (see Problem 7.1). This analysis allows us to weaken the conditions that we impose on the local error. If, as usually happens in the modern program packages designed to solve the problem, one allows the stepsize to vary at each step, the difference equation for the global error is no longer autonomous even for autonomous differential equations. Gear and Tu [72], for example, obtain the difference equation En+l = S(n}En + Wn,
(7.23)
where the matrix S(n) takes into account many other factors that define the method. It is shown that S(n) = S(n) + hnS(n).
(7.24)
Suppose that these two conditions hold: (1) (2)
where $(n, no) is the fundamental matrix of the homogeneous equation derived from (7.23), namely En+i = S(n}En. The first condition (see Theorem 4.2.1) is equivalent to asking that the zero solution of En+l = S(n}En
(7.25)
is uniformly stable. The second condition is related to the bouridness of the Lipschitz constant of f ( t . y ) .
Copyright © 2002 Marcel Dekker, Inc.
7.3. INFINITE
INTERVAL
201
From (4.38) we have N-l
EN =
and hence it follows that \\EN\\ < k0\\E0\\ + k0
- l l + \\Wj\\).
By Corollary 1.6.2 we obtain N-I \\EN\\
< fco||£o
N-I / N-I + k0 ^ ||Wa||exp I k0ki ^ s=n 0
y
j=s+l
<
from which follows that if ||£^o|| and ZlsLno II ^5 II tend to zero as h —>• 0, the method is converent.
7.3
Infinite Interval
In practice one uses methods with h bounded away from zero. This means that it is no longer true that h —> 0 as n tends to infinity. The term hBnEn cannot be considered as a perturbation, especially when the norm \\Bn\\ is very large (in stiff equations). For such problems it is necessary to consider the quantity hBnEn as a principal component of equation (7.14). Because h is taken either fixed or bounded away from zero, there are problems (stiff or multiscale) where n may assume very large values. When the term hBnEn is taken into account, the equation is no longer linear because the elements of the matrix Bn depend on the solution. The problem becomes a difficult one. It has been studied extensively in the linear case f(y] = Xy when Re\ < 0. In this case, the matrix Bn becomes independent on n, namely
(7 26)
-
where bT =
(7.27)
and the equation (7.14) reduces to En+i = (A + hB)En
Copyright © 2002 Marcel Dekker, Inc.
(7.28)
202
CHAPTER 7. NUMERICAL
METHODS
whose solution is, taking no — 0, n-l
En = CnE0 + ]T Cn-j+lWj.
(7.29)
3=0
The behavior of En will be dictated by the eigenvalues of the matrix C, which are the roots of the polynomial iv(z,q) defined in Section 2.8. The set D of values of q = hX for which the zero solution of the unperturbed equation En+l = CEn (7.30) is asymptotically stable is said to be the absolute-stability region of the method. As in the previous case, since the roundoff errors EJ are only bounded, it is more desirable, according to Theorem 4.11.1, to require the asymptotic stability of En = 0 in (7.30). This implies that q must be strictly inside the absolute-stability region. A very large effort has been made to study the absolute stability for different a.t and /3j, that is, for different classes of methods. Special importance is given to methods for which the absolute stability region is unbounded and contains the complex left-half plane (/i-stable methods) because in this case no restrictions are to be imposed on the parameter h in order to restrict hX to be inside that region. We will not go into the details of this work. We are only interested in the techniques used relative to difference equations. In recent years, the case of nonautonomous linear equations has also been discussed. This is very important because in the modern software packages to solve differential equations, one allows the step-size to vary at each step. In this case, the difference equation is no longer autonomous and the considerations based on the eigenvalues of the matrices C(n) are no longer enough to ensure the asymptotic stability (compare the counterexample given in Section 4.4). One needs to impose some conditions on the norms of matrices ||C'(n) in order to get information on the fundamental matrix <$(n,no) of the equation En+i = C(n}En.
(7.31)
Once this has been achieved, we can use the formula (4.41) and obtain n-l
En = $(n, no)E0 +
$(n,j + \)Wj-
( 7 - 32 )
Suppose that the zero solution of (7.31) is uniformly asymptotically stable, then by Theorem 4.2.2, there exist a, 77 > 0 and 77 < 1 such that
Copyright © 2002 Marcel Dekker, Inc.
7.4. NONLINEAR CASE
203
It then follows that ll^nll < ^n-nol|£o|| +a ]T rjn->-l\\W3l j=no
from which one derives at once the stability of En = 0 for the perturbed equation. So far we have treated the case of a single equation. The case of autonomous systems y' = Ay, y(tQ) = yo, (7.34) can be treated similarly. In fact, suppose that the matrix A can be reduced to the diagonal form by means of a similarity transformation A — T~1KT where A — diag(\i,\2, . . . ,\k)- Then the system (7.34) reduces to uncoupled scalar equations and the condition becomes that each h\i must be inside the absolute stability region.
7.4
Nonlinear Case
The study in the nonlinear case of equation (7.14) is difficult and has been made only for nonlinearities of a special kind, namely, monotone. The techniques used require essentially to bound norms of the matrix A -f hBn. We shall be content to present only the case of the implicit Euler method and the one-leg methods, since they are good examples of the use of the equations employed here. Let / : [0, oo] x IRS —> IRS and suppose that for w, v e R, (u - v } T ( f ( t , u) - f ( t , v}) < fi\\u - v\\2,
(7.35)
with fj, < 0. The difference equation (7.14) becomes in this case en+i - en - h (f(tn+i,zn+i + e n+ i) - f(tn+i,zn+\)) = wn.
(7.36)
By multiplying both sides by e^+l we have \\en+i II 2
< <
||en||||en+1|| + Mien-fill 2 + ||en+i|||K||,
(7-37)
where the Cauchy-Schwarz inequality \aTb\< \\a\\\\b\\ has been used. From (7.37) one gets
Copyright © 2002 Marcel Dekker, Inc.
(7.38)
204
CHAPTER 7. NUMERICAL METHODS
By using Corollary 1.6.1, we then obtain n-n 0
n
-!
/
1
\n-j-l
(7 40)
'
Since the quantity 77 = ^^ is less than one by hypothesis, the uniform stability of the solution en — 0 follows. The previous arguments can be generalized to the class of methods defined by p(E)yn - hf(a(E}tn, a(E}yn) = 0.
(7.41)
This class of methods was proposed by Dahlquist who named one-leg methods. There is a connection between the solutions of the difference equation (7.2) and the solutions of (7.41) (see Problems 7.6, 7.7). For the sake of simplicity, we shall suppose / : [0,T] x IR —>• 1R. In this case the quantities in (7.41) are scalars. The error equation for (7.41) is p(E)en - h ( f ( a ( E ) t n , a(E}yn + cr(E}en] - f(a(E)tn, a(E}yn}} = wn. (7.42) Multiplying by <j(E)en one obtains cr(E)en • p(E}en < h/j,\a(E}en\2 + wn • cr(E)en. Let us switch now to the vectorial notation and suppose that there exists a symmetric positive definite matrix G such that the Liapunov function
Vn = En GEn satisfies the inequality Vn+l -Vn< 2a(E)en • p(E)en.
(7.43)
It then follows that Vn+i-Vn
< 1a(E)en • p(E)en en\ + 2\a(E}en\\wn\.
By considering that (here || • | 2 is used)
i=0
we have: by setting (Eto A2 Vn+l - Vn < &(\\En+l\\ + ||^n||)kn +
Copyright © 2002 Marcel Dekker, Inc.
2
^(\\En+l\\ + \\En\\)2 .
(7.44)
7.5.
OTHER TECHNIQUES
205
Since \\En\\ < HC'^IIHG 1 / 2 ^!! - \\G-l/2\\Vn/2 the relation becomes
Vn+1 -Vn< We let (5||G- 12 || = a. Consequently
from which follows the estimate T/V2 /
a
\'
n—no
/ a
*\~^ I
n
2\
~J~l
~*~ ^^f~ |
i-/*Y j=o \ i - f c V / i
7 LLCL
L. id
l - i
7 Lid"
I
which gives a bound for ([EViHcThe existence of the matrix G can be established if the method is Bistable. See Dahlquist [48].
7.5
Other Techniques
The formula (4.52) can be used to evaluate the global error. In fact if x(n, no, XQ) is the solution of the unperturbed problem corresponding to (7.2) and 7/(n,no,xo) ig the solution of the perturbed problem corresponding to (7.3), then the difference En = y(n, no, XQ) — x(n, no, XQ) derived from (4.52) gives the global error. We have already used this result in the linear case (see (7.16)). In applications one uses a similar formula obtained by using the method of variation of parameters of the original differential equation.
Theorem 7.5.1 Let (1) y(t] be the solution of (7.1); (2) 7/0 — 20; zii • • • , zn be approximations of y(i] at the points £Q, £1,. • • , tnj (3) p ( t ) be a sufficiently smooth function that interpolates the points zi so that p(t7j) = Zi i = 0 , 1 , . . . , n. Then En = y(tn) — zn satisfies the relation En = - I" $(*n, s,P(s)}\p'(s} - f ( s , p ( s ) ) ] d s , Jto
Copyright © 2002 Marcel Dekker, Inc.
(7.45)
CHAPTER 7. NUMERICAL METHODS
206
Proof. Let s G { t o , t n } and y(to,s,p(s}} be the solution of (7.1) with initial value (s,p(s)). It is known in the theory of differential equations that dy(tn,s,p(s)) ds and
dy(tn,s,p(s)) dp Consider the integral /
= -En.
y(tn,s,p(s))ds = y(tn,tn,p(tn))
This integral also is equal to ,
dp
'to
which completes the proof. D The formula (7.45) can be used to obtain methods with higher order, to find the order of known methods, and to study new methods. See [150, 179].
7.6
The Method of Lines
Consider the following problems du(x.t) di u(0,1}
d2u(x,t)
du(x,t) dt
du(x.i]
(7.46)
dx2 -0,
u(x,Q) = g(x]
and
u(0,t)
dx = 0,
(7.47)
' u(x,0) = g(x},x > 0.
Let us discretize the interval (0,1) by taking Xi = ih. i — 0 . 1 , . . . , N + 1 and consider the vectors U(t} = (u(xi,t),..., U(XN, t))T, G = ( g ( x \ } . . . . , g(xN))J where u(x^t) are approximations of the solution along the lines x = x,. We approximate the operators -j^ and ^ by central differences and backward differences respectively. By introducing the matrices / -2 1 AN =
Copyright © 2002 Marcel Dekker, Inc.
0
1 0 ... -21 0
...
0 \ 0 (7.48)
207
7.6. THE METHOD OF LINES and / 1 -1
1
(7.49) -1 1 / NxN
the two problems can be approximated by
= G,
(7.50)
and
(7.51) We have approximated the two problems of partial differential equations with two problems of ordinary differential equations. If we discretize the time, using, for example, the explicit Euler method and try the results on absolute stability, we get two different results, namely, one correct result for problem (7.46) and one incorrect result for problem (7.47). In fact, the eigenvalues of AN and BN are k = 1 , 2 , . . . , TV,
Xk = -2 + 2cosTV + 1' // fc = 1,
k = 1,2, . . . , 7 V ,
(7.52) (7.53)
and the region of absolute stability for the explicit Euler method is D = {z£C:\z + l\
(7.54)
In order to have stability of the method we must have (7.55) for the first case and (7.56)
for the second case. From (7.55) and (7.52) it follows that the condition of stability for the first problem is 2'
and for the second one is
Copyright © 2002 Marcel Dekker, Inc.
At
< 2.
(7.57)
(7.58)
208
CHAPTER 7. NUMERICAL METHODS
It happens that (7.57) agrees with the Courant Friedrichs and Lewy condition, while (7.58) does not agree with the correct condition, which is ~ < 1(7-59) ' Ax The reason for the discrepancy lies in the fact that the ordinary differential equations (7.50) and (7.51) depend on JV, which tends to infinity when Ax tends to zero. This implies that when we study the error equation, for Ax and At tending to zero, the dimension of the space IR^ and therefore of the matrices AN and BN increases. We shall see in the next section how to obtain the right conditions.
7.7
Spectrum of a Family of Matrices
Consider a family of matrices {Cn}n£N+, where Cn are n x n real matrices. Definition 7.7.1 The spectrum of a family of matrices {Cn} is the set of complex numbers A such that for any e > 0, there exists n E N^ and x E C n , x ^ 0 such that
\\Cnx - Xx\\ < e\\x\\.
(7.60)
The spectrum of [Cn] will be denoted by 5({Cn}), or S when no confusion will arise. It is obvious that the set S of all eigenvalues of all matrices of the family is contained in S. The elements of the set 5 \ S are called quasi-eigenvalues. We now list some properties of S, without proof. PI S is a closed set. P2 If the matrices are normal, then S = S where E is the closure of S. P3 If P is a compact set of the complex plane and if P Pi S — 0, then sup \(Iz-Cnrl\\ < oo.
(7.61)
P4 Let S be compact and 5 C Q where il is an open, simply connected set of C. Moreover let / be an analytic function in O. Then
S({f(Cn)}) = f ( S ( { C n } ) )
(7.62)
q = sup|A|.
(7.63)
P5 Let A65
If q < 1, one has for every n, m E N^ imi < where k is independent on m and n.
Copyright © 2002 Marcel Dekker, Inc.
fc,
(7.64)
7.7. SPECTRUM OF A FAMILY OF MATRICES
209
Let us now find the spectrum of the family {Cn}: a
7
(7.65) -. 7 (3 a /
where a, /3,7 are real and we shall suppose for simplicity that |/3/7J < 1. Let be Q = S\ S. The elements of Q are the quasi-eigenvalues. Theorem 7.7.1 The set of complex numbers A, such that the zero solution of the difference equation -yxi+i + (a - \}xi +
0
(7.66)
is asymptotically stable, is contained in S({Cn}). Proof.
The characteristic polynomial of (7.66) is 7r2 + (a - A)r + /3 = 0
(7.67)
and to get asymptotic stability it must be a Schur polynomial (see Theorem 2.7.1). Suppose it is. Then taking as vector x the vector made up with the first n components of the solution of problem (7.66), one has Cnx - \x — (0, . . . , 0,
It follows that \\Cnx - \x\\ =
where r\ and r^ are the roots of (7.67) which, for simplicity, are supposed to be simple and ci,C2 are deduced from the initial conditions. That is 1 1 C2 = n -r 2 I — r\ We then get \\CnX -
n+l
In -
r' 1
—
where we have considered that ||x|| > 1. The quantity multiplying ||x can be made arbitrary small if \r\ and r^ arc less than 1 . Similar considerations can be made if \r\\ — \r^\. One can show that if \r\ or \r^\ are greater than 1, the corresponding A is no part of the spectrum and A ^ E (see Problem 7.9). n The next theorem explicitly furnishes the boundary of S.
Copyright © 2002 Marcel Dekker, Inc.
210
CHAPTER 7. NUMERICAL METHODS
Theorem 7.7.2 The boundary of S is given by x = Q• + (0 + 7) cos 0,
y
= (7 - /3) sin 0
(7.68)
where X = x + iy, and 0 < $ < 27r. Proof. The proof reduces to finding for which values of A the polynomial (7.67) is a Schur polynomial. This can be done easily by using Theorem B.I.I of Appendix B, obtaining the conditions (7.68). d The set S has then the ellipses with center (a, 0) and semiaxes (j3 + 7) and (7 — j3) as boundary. By considering that the eigenvalues AJ,, °f the matrices Cn, are given by X^} = a + 2(3 7 ) 1 / 2 cos ^. itfollowsthat AJ; n) 6 S. Particular cases are: (a) a = —2,3 = 7 = 1; in this case the family {Cn} coincides with the family {An} defined by (7.48). The spectrum S({An}) is then the segment -2 4-2 cos 0, O<#
y = - sin 0.
(7.70)
Once the spectrum of the family has been obtained, the correct stability conditions can be derived. In the case where the ordinary differential equations are discrctized by using the Euler method, for example, instead of (7.55) and (7.56). one imposes At
~" ' }}CD
(7.71)
^S({Bn}) c D
(7.72)
for the first problem and
for the second problem. In fact we have: Theorem 7.7.3 Suppose that the problems (7.50) and (7.51) are solved with the explicit Euler method. If (7.71) and (7.'72) hold, then the resulting difference equations are stable for all values of n. Proof. Let Us. s = 0. 1 . . . . be an approximation of U(ts}. The Euler method will produce the difference equations
Us+i = U, +
Copyright © 2002 Marcel Dekker, Inc.
AnUs =(l+ - A
n
U
7.8. PROBLEMS
211
and
respectively. When (7.71) and (7.72) hold, by P4, it follows that the spectrum of the families of matrices {/ + ^z^-n} and < / — ^.Bn\ are contained in the unit circle. This implies, by P5, that all the powers of the matrices of the families are uniformly bounded, and this implies the stability. D One should consider, however, that the condition of consistency for the space discretization implies a + /? -f 7 = 0 and the origin is a point of the boundary of both S and 3D. This implies that (7.71) and (7.72) cannot be strictly satisfied. For some class of matrices, for example normal matrices, this does not create any problem. Note that condition (7.71) is not more restrictive than the one obtained by using only the eigenvalues because in this case, the matrices are normal and the spectrum is only the closure of the set of all the eigenvalues. Condition (7.72) is more restrictive because the spectrum is a disk in the complex plane, while the set of eigenvalues is made up of only one point. More generally, a region of absolute stability D defined by the discretization of the variable t and a region 5, which is a function of the spectrum of a family of matrices defined by the discretization of the space variables, will be associated with each method. For the stability the second region must be contained in the first. This approach is also useful because it permits the choice of appropriate discretization of the time as a consequence of the discretization of the space variables.
7.8
Problems
7.1 Suppose that the matrix A has the eigenvalues 1 = \\ > \2 > • • - > XkUsing the decomposition (A. 13) of the matrix, give a bound for \\En\\ in (7.16). 7.2 Show that for the Euler method yn+\ = yn + hfn the region of absolute stability is the circle with center at —1 and radius 1. 7.3 Show that the region of absolute stability of the trapezoidal method zn+i = zn + |(/n + fn+i) is the left-hand plane. 7.4 Find the one-leg correspondent of the trapezoidal method. It is known as the implicit midpoint method. 7.5 Show that the region of absolute stability of the midpoint method zn+2 = zn + 2hfn+i is the segment [—i, i] of the imaginary axis.
Copyright © 2002 Marcel Dekker, Inc.
212
CHAPTER 7. NUMERICAL METHODS
7.6 Suppose that yn satisfies the one-leg equation p(E)yn - hf(a(E}yn} = 0. Show that yn — a(E}yn, satisfies the equation p(E)yn — ha(E)f(yn) = 0. (Hint: the operators p(E) and cr(E) commute.) 7.7 Suppose that {yn} satisfies the equation p(E)yn — ha(E}f(yn) — 0 and P ( z ) and Q(z] are polynomials satisfying P ( z } a ( z ) — Q(z}p(z) = 1. Show that yn = P(E)yn — hQ(E)f(yn) satisfies the one-leg equation p(E}yn —
7.8 Suppose that c.n < -p, (fj, > 0) for all n.3n / 0 in (7.10) and the roots of cr(z) are inside the limit disk. Show that the error en tends to zero. 7.9 Show that in Theorem 7.7.1 if one of the roots of (7.66) has modulus greater than 1, then A does not lie in Q. 7.10 Show that for the matrix (7.65) we have \\Cnx - \x\\ = f3pn-lUn(q) , where p = (/3/7) 1 / 2 and q = (A — a ) / 2 ( l d j } 1 ^ 2 and Un(q) is the Chebyshev polynomial of second species. (Hint: use the results of Section 2.4.) 7.11 Show that the midpoint method cannot be used for \trie time discretization of 7.50. 7.12 Consider the second-order hyperbolic equation Utt - Uxx = 0, U(x, 0) - f ( x ) , (7(0. i) = U(TT, t) = 0 and discretize the space variable obtaining a system of first-order equations,
1 dt
= -1^D2N'W,
Aa
™(0) = '0,
where w = (U.V)T , ;/' = (^-0) T . C7, V, $ G IR^, discrete points in (O.Tr), and 0 A
N
N is the number of
Ax 2 / n 0
7.13 Show that D\^ is normal and find its spectrum. Using this result, show that the midpoint method can be used in this case.
7.9
Notes
The problem of numerical methods for differential equations has been extensively studied in the last 40 years. For important works in this field see Henrici [88, 89] and Dahlquist [45]. The most important reference books on the subject arc by Lambert [107], Butcher [30]. and Hairer ct al. [83].
Copyright © 2002 Marcel Dekker, Inc.
7.9. NOTES
213
The recent book by Brugnano and Trigiante [26] is more oriented toward difference equations. For example, it contains a wide treatment of the notion of spectrum of a family of matrices. The study of nonlinear case was also initiated systematically by Dahlquist [52, 46, 47, 48, 49, 50]. Important contributions have also been made by Odeh and Liniger [134], Nevanlinna and Liniger [131], and Nevanlinna and Odeh [132]. For the class of RungeKutta methods, not treated in this book, see Butcher [29] and Burrage and Butcher [27], [28]. For applications of the theory of difference equations to methods for boundary value problems see Mattheij [119]. Material on the spectrum of family of matrices can be found in Bakhvalov [17], Godunov and Ryabencki [76], and Di Lena and Trigiante [109]. A general treatment in Hilbert space setting of the spectrum of infinite Toeplitz matrices can be found in Hartman and Wintner [87], as well as in the Toeplitz original papers [171]. A related question (the pseudo-spectrum) has been principally studied by Reddy and Trefethen [156] and F. Chaitin-Chatelin et al. [37]. Applications of the enlarged spectrum of symmetric tridiagonal matrices (Jacobi matrices) are found in the theory of orthogonal polynomials (see for example Mate and and Nevai [116] and the references therein).
Copyright © 2002 Marcel Dekker, Inc.
Chapter 8
Models of Real World Phenomena 8.0
Introduction
In this chapter we offer several examples of real world models to illustrate the theory developed, to show how versatile difference equations are and to demonstrate how often the results of discrete models can explain the observed phenomena better when compared to continuous models. Furthermore, the contents of this chapter are also intended to help the practitioners who can look first at the examples and then proceed to read the necessary theory of difference equations. Section 8.1 deals with linear models of population dynamics, sketches the development, and discusses the appearance of population waves. We devote Section 8.2 to the study of nonlinear models of population dynamics including bifurcation and chaotic behavior. In Section 8.3, we consider models arising in the distillation of a binary ideal mixture of two liquids, while in Section 8.4 we discuss models from theory of economics. Models from queueing theory and traffic flow are investigated in Section 8.5. In Section 8.6 we collect some interesting examples dealing with various models.
8.1
Linear Models for Population Dynamics
The dynamics of populations have been extensively studied since Malthus. The object of study is the evolution in time of a group of individuals who follow the simplest and essential life events: they are born, mature, reproduce and die. The group can be of bacteria, animals, or humans. For simplicity, in all the models one usually only considers the female part of the population. Both continuous and discrete models have been proposed. More than the law of mechanics it does nor always seem justified to use in this field continuous models. In fact the usual assumption that the variation 215 Copyright © 2002 Marcel Dekker, Inc.
216
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
per unit time of the number of individuals is small compared to the total number is no longer valid as in mechanics. Often the results of continuous and discrete models are qualitatively similar, but sometimes, especially in nonlinear models, they are not and the continuous models are unable to explain observed phenomena (see Section 8.2). Let yn be the number of females of a population at time tn. The simplest model, called after Malt 1ms, the Malthusian model is the following 2/n+i = aVn-
yno = yoi
a > 0,
(8.1)
which simply states that the number of newborn individuals, as well as the number of individuals that are deceased in the time interval At n , is proportional to the number of individuals at time tn. The parameter a is called the intrinsic growth rate and is simply the difference between the birth and death rates. The solution of (8.1) is yn = an~n°yo and has been found to be very effective to fit exponential data for populations that are not large. One of the weaknesses of the model is that for n —> oo, yn —> oc, which is impossible due to limitation of the resources. The foregoing simple model does not take into account the age structure of the population, which is important because all the essential life events depend on the age of individuals. A more refined model is the following (often called Leslie model). Let us divide the lifetime L into M parts and then population in groups (or classes) yi(n], y<2 ( n ) , . . . , yi\i(n) such that the group yi(n) is made of individuals whose age a,- satisfies
Considering the unit time equal to the time spanned by a class, one has that yi+i(n 4- 1) is made up of all the individuals in the class i at time n except of those who are deceased (or removed in some way). That is, yi+i(n+l) =/3iyi(n),
i = 1. 2. . . . , M - 1,
where /?7 > 0 is the survival rate of the ith age and M
which states the newborns at time n + 1 as sum of the contribution of the different groups each of which is considered to have an homogeneous behavior in the reproduction. Having the human population in mind, one would expect a, to be zero in the extreme groups and the maximum in some group in the middle. In nature, however, there is a wide range of
Copyright © 2002 Marcel Dekker, Inc.
8.1. LINEAR MODELS FOR POPULATION
DYNAMICS
217
possibilities; for example on = 0 for all groups except one, giving rise to different kinds of behavior. By introducing the vector y(n) — ( y i ( n ) , . . . , ?/A/(n)) T and the matrix
0
x
0
(8.2)
...
o J MxM
0
the model can be written as y(n + 1) = Ay(n),
y(n0) =
(8.3)
where yo is supposed to be known. The solution is y(n) = Anyo (we take no = 0 for simplicity). The behavior of the solution will then follow from the spectral properties of the matrix A. Consider first the case on > 0 and j3i > 0 for all indices. In this case it is easy to verify that AM > 0 (see Sec. A.7). By the Perron-Frobenius theorem (see Theorem A.7.2), it follows that A has a positive simple eigenvalue AI associated with a positive eigenvector u\. The remaining eigenvalues are inside the circle B^ in the complex plane. By using the expression (A.26) for the powers of A and considering that in the present case m\ — 1, one has
An = \\
\n—i+l
_u + k=2i=l
from which it follows that the double sum in brackets tends to zero as n —* oo. Considering that Z\\ is the projection to the eigenspace corresponding to A], one has Any$ « cX^ui where c is a constant. The population tends to grow as A^ and is close to u\. that means that the distribution between the age groups tends to remain fixed. In fact. (8.4)
where u^ is the i — th entry of the vector u\. The vector u\ is called the stable age vector and AI the natural growth rate. Of course the overall population will grow for AI > 1 and will extinguish for \\ < 1. Special cases of interest are those where some c^ can be zero. To make the study simpler, it is useful to change variables. Letting l^ = 1, lk = nto1 A (k = 2 , . . . , M), a, = Ii0ti (i = l,2,...,M),D = diag^h,..., / A / ),
Copyright © 2002 Marcel Dekker, Inc.
218
CHAPTER 8. MODELS OF REAL WORLD
x(n) = D l y ( n ] and B = D
B=
l
PHENOMENA
AD, one verifies that
0*1
Cif2
1 0
0
V 0
\
0
1
which is a companion matrix (see Section 3.4). The model is now x(n+ 1) = Bx(n),
(8.5)
and the characteristic equation of B is then (see (3.20)): \ A/
A
„ \ A/-1
— CL\A
„
\ A/-2
— a2-A
\
n
. . . — &M — 1-^ ~ &M — U-
(a (;\
V*'W
The following theorem from Cauchy is very useful in this case. Theorem 8.1.1 If a, (i = 1.2....M) are nonnegative and the indices of the positive a, have the greatest common divisor 1, then there is a unique simple positive root A I , of the greatest modulus. For a proof, see Ostrowski [141]. Since the eigenvector u\ of B is u\ = (A^" 1 , A f 7 , . . . , 1)T, it follows that til is a positive vector and all the previous results are still valid. When the hypothesis on the indices of Theorem 8.1.1 are not satisfied, then some eigenvalues may have the same modulus of AI giving rise to the interesting phenomenon of population waves. In this case in fact there are oscillating solutions to the model. To see this in a simpler way, let us consider the scalar equation for the population of the first group. This can be seen from (8.5). The first equation of the system gives A/
and from the following equations, choosing the appropriate index n, we obtain xl(n] — x\(n — i + 1). By substitutions one has xi(n + 1) =
which is a scalar difference equation of order M. The characteristic polynomial coincides with (8.6). The solution of (8.8) in terms of the roots of (8.6) is given by (2.37) with A, instead of Z{. If there are two distinct roots with the same modulus, sa AI = el°, \2 = e~ld, one has + other terms.
Copyright © 2002 Marcel Dekker, Inc.
8.2. THE LOGISTIC EQUATION
219
It is possible to find two new constants a and i/j such that c\ — ae1^, 02 = ae~1^, and then x\(n) — 2pnacos(n6 + i/') + other terms, from which follows that x\(n) is an oscillating function. In the general case more than one period can coexist. In the extreme case a^ — 0, i = 1, 2 , . . . , M — 1, OM 7^ 0 a number proportional to M of periods can coexist and for large M a phenomena similar to chaos (see next section) may appear. The population waves have been observed in insect populations.
8.2
The Logistic Equation
The discrete nonlinear model, which we are going to present for the dynamics of populations and which takes into account the limitation of resources, has been used successfully in many areas such as meteorology, fluid dynamics, biology, and so on. Its interest consists of the fact that, in spite of its simplicity, it presents a very rich behavior of the solutions that permits to explain and predict a lot of experimental phenomena. Let N(n) be the number of individuals at time n and suppose that N(n + 1) is a function of 7V(n), that is, N(n+l) = F(N(n)). (8.9) This hypothesis is acceptable if two generations do not overlap. For small N, (8.9) must recover the Malthusian model, that is, F ( N ( n ) ) — aN(n] + nonlinear terms. The nonlinear terms must take into account the fact that, when the resources are bounded, there must be a competition among the individuals which is proportional to the number of encounters among them. The number of encounters is proportional to N2(n). The model becomes N(n + 1) = aN(n) - WV 2 (n),
(8.10)
with a and 6 positive. The parameter a represents the growth rate, and b is a parameter depending on the environment (resources). The quantity a/b is said to be the carrying capacity of the environment. For a > 1, this equation has a positive critical point N = ^^ which, as shown below, is asymptotically stable when a is in a suitable interval. This means that limn_^oo N(n) — N and N is a limit size of the population. N depends on both a and b. A more realistic model would assume both such parameters to vary with time. In fact b can diminish because the population learns to better use the existing resources or to discover new ones; a can grow because the population learns how to live longer or how to accelerate the birth ratio. Anyway, taking
Copyright © 2002 Marcel Dekker, Inc.
220
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
into account the corresponding variation of TV (which happens with a longer time scale), one sees the evolution of the population can be described as a sequence of equilibrium points with increasing values of TV. Almost all the previous results are similar to those obtained in the continuous case. A further analysis of the discrete logistic equation shows solutions that are unexpected if one thinks the discrete equation merely as an approximation of the continuous one. To see this, let us change the variable and simplify the equation. Let yn — ^TV(n). The model becomes yn+i = ayn(l - yn] = f ( y n } -
(8.11)
The new variable yn represents the population size expressed in unit of carrying capacity a/6. The equation (8.11) has two equilibrium points, the origin and y ~ (a — l)/a. which has physical meaning only for a > 1. For a < 1, by using the theorems of stability by first approximation (see Section 4.7), one sees at once that the origin is asymptotically stable. For a > 1, the second critical point becomes positive and the origin becomes unstable. The stability of y can be studied locally by the linearization methods again using one of the theorems of Section 4.7. In fact 2/n+i - y - f'(y)(yn -y)-\
—(yn - y}2 = ( 2 - a)(yn - y) - a(yn - y ) 2 .
If the coefficient of the linear term is less than one in absolute value, then y is asymptotically stable. As a consequence, one has that for 1 < a < 3 the point y is asymptotically stable. For a = 2, the solution can be found explicitly (see (1.15)). For a > 3, a new phenomenon appears. There exists a couple of points x\ and a:2 such that x2 = f ( x i ) and xi - /(x 2 ).
(8.12)
The couple x\ and X2 is called a cycle of period two. From (8.12) it follows that L), (8-13) that is, x\ and x-2 are fixed points of the difference equation
)-
(8.14)
One can determine the fixed points by using the Equation (8.13). which is a fourth degree equation (it contains as roots the origin arid y ) . The following theorem permits us to simplify the problem. Let f [ x . y] be the usual divided difference, that is. f,[rx , y \ ! = /(*) - /(y)
Copyright © 2002 Marcel Dekker, Inc.
8.3. DISTILLATION OF A BINARY LIQUID
221
Theorem 8.2.1 The solutions of period two for the difference equation — f(yn] exist iff there exist values of x such that /[x, f(x}} = — 1. Proof.
By definition
If /( 2 )(x) = x then (8.15) is equal to -1. On the other hand, if (8.15) is equal to — 1, then it follows at once that x = f^(x). D Applying the previous results one obtains in the present case /[x, f ( x } \ = a(l — x — ax + ax2} = — 1. This equation has real roots for a > 3. It can be shown, for example, considering the linear part of /( 2 )(x), that the cycle is asymptotically stable for values of a in a certain range, while y becomes unstable. One says in this case that the solution y bifurcates in the cycle of order two. As a result, it follows that if y is greater than | then the system tends to oscillate between two values. We cannot go into details on what happens for a > 3 (see [122] for details), but we shall qualitatively sketch the picture. For higher values of a, a cycle of order four appears and then a cycle of order eight and so on. All the cycles of even period 2 r . r > 0 will appear. All this happens as a goes from 3 to 3.57, where the last point is an accumulation point of cycles of 2r periods. What is the solution like to the left of this point? For a near 3.57 one can find a very large number of points that are parts of some even period solutions. In this region even if two similar populations evolve starting from two very close initial points, their history will be completely different. After a long time every subiriterval of (0, 1) will contain a point of the trajectory and if one maps the density of the number of occurrences of yn, in subintervals of (0, 1) the pictures are very similar to sample functions of stochastic processes. This behavior has been called chaos. After the value 3.57 of a, the solution of period three appears. There is a result by Li and Yorkc [112] that states if there is a cycle of period three, then there are solutions of any integer periods and. for the same reasons discussed above, the term chaos is appropriate in this case as well. When a = 4, again the solution can be done in closed form (see Example 3). Almost all of the previous results can be extended to more general functions / leading to the conclusion that the qualitative results are widely independent from the particular function chosen to describe the discrete model.
8.3
Distillation of a Binary Liquid
The distillation of a binary ideal mixture of two liquids is often realized by a set of AT-platcs (column of plates) at the top of which there is a condenser
Copyright © 2002 Marcel Dekker, Inc.
222
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
and at the bottom there is a heater. At the base of the column there is a feeder of new liquid to still. A stream of vapor, whose composition becomes richer from the more volatile component, proceeds from one plate to the next one until it reaches the condenser from which part of the liquid is removed and part returned to the last plate. On each plate, which is at different temperature, the vapor phase will be in equilibrium with a liquid phase. Because of this, a liquid stream will proceed from the top to the bottom. We suppose that the liquids are ideal (Raoult's law applies) as well as the vapors (DaltoiVs law applies). Let yl (i — 1. 2) be the mole fraction of the z-th component in the vapor phase and :r? the mole fraction of the same component in the liquid phase. Of course y\ + y2 — 1 and x\ + x^ — 1 (the sum of the mole fraction in each phase is 1 by definition). Under this hypothesis, the quantity called relative volatility,
- y^'^ 2/2 Xi '
can be considered constant in a moderate range of the temperature (see [173]). If a > 1. one says that the first component is more volatile. For simplicity, we shall consider as reference only the more volatile component. Setting y\ — y and x\ — x. one has
(i-s) -
, 8(8.16) irM
which will be considered valid every time the two phases are in equilibrium. Let us see what happens on the n-th plate. Here the two components are in equilibrium in the two phases. Let xn be the mole fraction of the more volatile component in the liquid phase, y^ the mole fraction in vapor phase of the same component, and yn the mole fraction of the same component leaving the plate n. If we assume that the plate efficiency is 100 percent, then y* = yn. Moreover, part of the liquid will fall down with rnole rate d and the vapor will go up with mole rate V. Let D be the mole rate of the product, which is withdrawn from the condenser. Consider now the system starting from the nih plate (above the point where new liquid enters into the apparatus) and the condenser. We can write the balance equation Vyn-i = dxn + DxD,
(8.17)
where XD is the mole fraction of the liquid withdrawn from the condenser. To this equation one must add the definition of relative volatility that will hold for the equilibrium of the two phases at each plate (8.18)
Copyright © 2002 Marcel Dekker, Inc.
8.3. DISTILLATION
OF A BINARY LIQUID
223
From the last relation we obtain n
(8-19)
, ++ ,(a- l)x ,n 1
and, after substitution in (8.17) we get xn DxD(a - 1) - Va DxD xnxn-i + -- -f -—.-IT- xn-i + -T( -TT = 0, a —1
d(oi — 1)
d(a — 1)
from which, by letting _
1 ~, Q-l'
CL —
^ _ DxD(a- 1) - Va ~ ~ , d(a-l) '
0 —
_ C—
DxD d(a-l)
one obtains xnxn-i + axn + bxn^i + c = 0.
(8.21)
This is a difference equation of Riccati type (see Example 11). Let us consider the boundary conditions. The initial condition depends on how the apparatus is fed from the bottom, and we will leave this undetermined. The other boundary condition will depend on the condition that we impose on the composition of the fluid on some plate. The final condition (which can be deduced from Fig. 16 of [110]) requires that y^ = XD, XN+I = XD- It is consistent with the condition V = d + D, necessary for other physical considerations. The Riccati equation can be transformed to a linear one by setting ^--a
(8.22)
to obtain zn+l + (6 - a)zn + (c - ab}zn^ = 0.
(8.23)
If AI and A2 are two solutions of the polynomial associated with the previous equation, one has (if AI ^ A2)
and therefore
By letting XN+\ — XD we get D
~
One obtains an equation whose unknowns are ^ (the Riccati equation is of first order and its general solution must contain only an arbitrary constant) and N (the number of plates needed to complete the process).
Copyright © 2002 Marcel Dekker, Inc.
(8.20)
224
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
Let us set, for simplicity, K — c\/c<2. After some simple algebraic manipulations. one obtains .--
=
(
}
i<*£ The value of /C is determined from the condition of the feed plate (which can be assumed as the zero-th plate).
8.4
Models from Economics
A model called the cobweb model concerns the interaction of supply and demand for a single good. It assumes that the demand is a linear function of the price at the same time, while the supply depends linearly on the price at the previous period of time. The last assumption is based on the fact that the production is not instantaneous, but requires a fixed period of time. Let the above period of time be unitary. We shall then have
do
(8.27)
where d n , sn are respectively the demand and supply functions and a, 6, d$. SQ are positive constants. This model is based on the following assumptions: (1) The producer hopes that the price will be the same in the next period and produces according to this. (2) The market determines at each time the price such that dn = sn.
(8.28)
From (8.27) and (8.28) one obtains Pn-l
+d^.
(8.29)
The equilibrium price pe is obtained by setting pn = pn-\ = pe, which gives
and the solution of (8.29) is given by Pn=--)"po+l --a which is deduced from (1.14).
Copyright © 2002 Marcel Dekker, Inc.
Pe,
(8-30)
8.4. MODELS FROM ECONOMICS
225
If b/a < 1, it follows that the equilibrium price pe is asymptotically stable, that is limn-^oopn = pe. Since —a/6 is negative, we see that pn will oscillate approaching pe. For b/a > 1 the equilibrium price is unstable, and the process will never reach this value (unless p$ = pe). As a more realistic model, take sn as a linear function of a predicted price at time n. This means that the producer tries to predict the new price using information on the past evolution of the price. For example, he can assume as the new price pn = pn-\ + p(pn-\ — Pn-i] obtaining Sn = b(pn-i + p(pn-l ~ Pn-l}} + «0,
while the first equation remains unchanged. By equating the demand and supply, one obtains -apn + d0 = bpn-i + bppn-i - bppn-2 + «0
from which />„ + -(! + p)pn-l - -pPn-2 + ^—^- = 0.
a The equilibrium price is again Pe
a
a
(8.31)
^° ~ so
a —o
The homogeneous equation is pn + -(l + p}pn-i - -ppn-2 = 0 a a and the characteristic polynomial reduces to z2 + -(I + p)z-^- =0. a a
(8.32)
(8.33)
Let z\ and z^ be the roots of this equation. The general solution of the homogeneous equation is Pn =
while the general solution of (8.31) is Pn = CiZ? + C2z2 + Pe,
(8.34)
where c\ arid c^ are determined by using the initial conditions. From (8.34) it follows that if both \z\\ and [22! are less than one. one has lim n _^oop n = pe. If at least one of the two roots has modulus greater than one, then \imn-Kx, pn — oo (for generic initial conditions). The problem is now reduced
Copyright © 2002 Marcel Dekker, Inc.
226
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
to derive the conditions on the coefficients of the equation in order to have \zi\ and Z2 both less than one. A necessary condition is
bp The other condition is
a 1 + P < T -P0
A positive value of p means that the producer expects the price will continue to have the same tendency, while a negative value means that the producer expects to have an inversion in the tendency. To an equation similar to (8.31), one arrives in the Samuelson's model of national income. Here the economy of a nation is modeled by considering four discrete functions: the national income yn, the consumer expenditures Cn, used to purchase goods, the investments In and the government expenditure Gn. The definition of yn is (8.35)
Gn-
The other relations among the four variables arc based on the following assumptions, which are self-explanatory even from the economics point of view 0 < a < I,
ayn,
p(Cn-Cn_i),
p>0.
The parameter a is usually called marginal propensity to consume and p is the acceleration coefficient or "the relation"'. For simplicity we assume Gn = G constant. By substituting in (8.35) we get yn - f*(l + p)yn~i + apyn-2 ~ G = 0, which is similar to (8.31). The constant solution (or equilibrium point) is now G
i
Copyright © 2002 Marcel Dekker, Inc.
8.5. MODELS OF TRAFFIC IN CHANNELS
227
If this point is asymptotically stable, then lin^^oo yn = ye, which means that with this hypothesis the national income, under the effect of the government expenditure G, will tend toward the income ye, which is greater than G. The factor j^ is said to be a multiplier. To see when this happens, just repeat what is said for the previous model remembering that in this case p cannot be negative.
8.5
Models of Traffic in Channels
The models presented here concern the traffic in channels (for example telephone lines) and are from information and queueing theory. In the first simple model, the number of messages of given duration is derived. In the other two models the probability of requests for services (for example telephone calls) is derived as well as the probability of loss of the request in a system of limited channels. Consider a channel and suppose that two elementary informations Si and 82 of duration t\ and £2 respectively (for example the point and line in the Morse alphabet) can be combined in order to obtain a message. Let t be a time interval greater than both t\ and £2- One is interested in the number of messages Nt of length t. These messages can be divided in two classes: those ending with Si and those ending with 82- The number of messages in the first class is Nt-ti while the number of messages in the second class is Nt-ti- Then we have Nt = Nt.tl + Nt.tz.
(8.36)
Suppose, for simplicity, that t\ = 1 and £2 — 2. The previous formula becomes Nt = Nt-i + Nt-2.
(8-37)
The initial conditions are N\ = I and 7V2 — 2. The equation (8.37) is the same as that which defines the Fibonacci sequence (see (2.45)). The solution is
where p\ = 1 ^ ' P2 — *~2 • Because \p2\ < 1. for large t one has Nt — c\p\. Shannon defines the capacity of a channel as c = rhm —2-— = Iog9 pi . t—»oc
Copyright © 2002 Marcel Dekker, Inc.
£
228
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
Difference equations arise very often in queuing theory. We shall give two examples. Consider a queue of individuals (or telephone calls in a channel). Let Pn(t] be the probability of n items arrived at time t and we have /L??Lo Pn(t) ~ 1-
Let AAt be the probability that a single arrival during the small time interval At and suppose that the probability of more than one arrival in the same interval is negligible (Poisson process). Let fj,At be the probability of completing the service in the interval At. We shall assume that the service is a Poisson process, that is, the probability of no arrivals in At is 1 — AAt and the probability of the service not completed in At (no departures) is 1 — /j,At. The following model is due to Erlang and describes the situation where at the beginning there are no items in the single channel queue and the service is made on a first come, first served basis,
P n (t P0(t + At) =
P0(t)(l-
The meaning of the equations is the following: the probability that at time t + At there are n items in line is equal to the sum of three terms: (1) The probability of already having n items at time t multiplied by the probability of no arrivals during At and the probability of no departure in the same interval; (2) the probability of having n — 1 items at time t multiplied by the probability of a new arrival in At; (3) the probability of having n + I items at time t multiplied by the probability of a departure in At. Taking the limit as At —-> 0 one has
dt
-XP0(t)+lJ,Pi(t).
It is important to know how the system behaves for large t. That is, to know if the limit Pn = Hindoo Pn(t] exists. The probability Pn. will describe the steady state of the problem. In order to get this information one observes that in the steady state the derivatives will be zero.
Copyright © 2002 Marcel Dekker, Inc.
8.5. MODELS OF TRAFFIC IN CHANNELS
229
It follows that the Pn will satisfy the difference equations M P n +i
- (A + v)Pn + AP n _i -XPo + ^Pi
= 0, = 0.
(8.38) (8.39)
To this one must add the condition X^n^=o-^n — 1> which simply states that it is certain that in the system we must have either no items, or more items. Let p = -. The solution of (8.38) is given in terms of the roots of the polynomial equation Z 2 - ( l + p ) 2 + p = 0,
which are z\ = 1, 22 = p. If p ^ 1, we get
Considering that from (8.39) one has P\ = pPo, it follows that c\ = 0. The integral condition can be satisfied if p < 1, obtaining
r
n=0
which gives c% = 1 — p and then Pn = (1 — p)pn, which is called geometric distribution. Important statistical parameters are: (1) the expected number in the system
n=0
n=0
p
n=0
(2) the variance r\Jp _ \ " "2p
n=0
r-2
— 1^) *n — / 'i - » n — -^ 5 n=0
one shows that E^o n2Pn = L + 2L2 and F = L + L 2 ; (3) the expected number in the line
n=0
In the following generalization the case of ./V identical channels is considered. The hypothesis on the distribution of arrivals and departures (Poisson distribution) is maintained. Moreover, we shall assume that both the state with zero items EQ and the state with N items EN arc reflecting, that is,
Copyright © 2002 Marcel Dekker, Inc.
230
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
from these states transactions are possible only in one direction. From the state EQ a possible transaction will be to the state E\, and from the state EN a possible transaction will be to the state E^_l. The resulting model is eft
= -(A + n^Pn(f) + AP n _i(t) + (n + l) M P n+1 (t). 1 < n < N - 1,
dt
The steady state solution satisfies the difference equations (n + l)Pn+i ~ ( p + n)Pn + pPn_! = 0. - pPo,
Pi
where, as before p — -. Let us look for solutions of the form Pn — znpr from which one has ZQ — PO, zi = P(h ^v = ^isT1- The equation is then p n [(-^ 4- (n + l)zn+l)p - (-2 Tl -i + nz n )] - 0. Let On = — zn + (n + l}zn+\. The equation becomes f)0n ~ On.! = 0.
whose solution is n
x>—"/a
"n — P
"()•
For n — N—l, we must have ^v-i — —ZN-\ +Nzj\ It follows then OQ — 0. The equation for zn is then ^2 n + (n + l)zn+1 = 0, which has the solution zn = —.ZQ — —PQ. n\ n\
Going back to Pn we have Pn — ^pnP(). Imposing the condition 1, one finds P(j = I/ X^jLo ^T- The solution is then
Copyright © 2002 Marcel Dekker, Inc.
8.6. PROBLEMS
231
which is called Erlang's loss distribution. For n — N, one obtains the probability that all the channels are busy, which also gives the probability that a call is lost. As N —» oo one obtains the Poisson distribution on
Pn -~~ —i ce~p
1
n\
The previous model also describes the parking lot problem. For this problem, N is the number of places or slots in the parking lot. The reflecting condition at the end just describes the policy that the parking lot closes when it is full. The probability that a car cannot be parked is P/V. If the management follows the policy of allowing a queue waiting for a free place, then the model is modified as follows: Pi
(n + l)Pn+\ -(p + n}Pn + pPn-i NPN+l -(p + n}Pn + pPn,i
= pPo,
= 0, = 0,
n < TV, n > N.
The solution is
= —PQ,
n < TV,
as before, and
Using the relation ]C^Lo Pn — 1 one obtains PQ.
8.6
Problems
8.1 In the case a7; > 0, derive the existence of eigenvalue of greatest absolute value from Theorem 8.1.1 instead of the Perron-Frobenius theorem. 8.2 Write the generating function for the solution of (8.8) and derive the behavior of the solution (Renewal Theorem). 8.3 Show that for a < I the origin is asymptotically stable for T/Q £ (0,1) for the discrete logistic equation. (Hint: use the Lyapunov function Vn =
vl) 8.4 Discuss the stability of y for the logistic equation using Vn — (yn — y)2. 8.5 Determine the solution of (8.5) for M = 4 and a\ = 0,2 = 03 = 0, 04 = 16. 8.6 Discuss the modified cobweb model with p — — 1.
Copyright © 2002 Marcel Dekker, Inc.
232
CHAPTER 8. MODELS OF REAL WORLD PHENOMENA
8.7 Derive the national income model assuming that In — p(yn-\ ~ 2/71-2)8.8 Suppose that the input for the traffic in N channels has Poisson distribution and depends on the numbers of free channels, that is. pi is not constant. Derive the model and find the solution.
8.T
Notes
Age-dependent population models have many names associated with them such as Lotka, McKendrick, Bernardelli, and Leslie. An extensive treatment of the subject may be found in Hoppensteadt [93]and Svirezhev and Logofet [169]. In [169] we also have several references to the subject. Theorem 8.1.1, which is a generalization of a Cauchy theorem, may be found in Ostrowski [141], but see also Pollard [151]. The discrete logistic equation has been discussed by many authors, for example, Lorenz [113], May [122, 121], Hoppensteadt [93]. and Hoppensteadt and Hyman [94]. The result of Li and Yorke [112] on the existence of periodic solutions is contained in a more general result due to Sharkovsky (see [163], [165], [94], [167] and [1] for more recent proofs and results). When a difference equation is derived from an approximation of a continuous differential equation, the question arises to what extent the behaviors of the respective solutions are similar. This problem has been outlined and treated by Yamaguti et al. [182, 183, 181] and Potts [155]. The distillation model has been adapted from [80], see also [2]. The discrete models in economics may be found in Gandolfo [67], Goldberg [77], and Luenberger [114]. Queueing theory is a large source of difference equations (see for example Jagermann [96] and Saaty [158]) where one may find very interesting traffic models more general than those presented in Section 8.4. For the parking lot problem, as well as other models concerning traffic flow, see Haight [82].
Copyright © 2002 Marcel Dekker, Inc.
Chapter 9
Historically Important Equations 9.0
Introduction
It is well-known that in mathematics many unsolved problems or problems solved after centuries of efforts have a very easy formulation. This is also true in the present field. All the examples provided below are very easily formulated, but, in most cases, their solutions have been obtained after many years of effort. In fact they have been studied for almost two centuries and still continue to attract the interest of many mathematicians. As we shall see, most of them share the difficulty that the set of critical points is not a discrete set. but the entire line y = x. To each initial point there corresponds a critical point on such line. The second example, derived from the Weierstrass method for proving the fundamental theorem of Algebra, has also a similar problem, although of a different nature. In Section 9.4, few examples relating difference equations to prime numbers are presented, along with the celebrated Euler theorem.
9.1
Combinations of Means
It is well-known that, given for example two positive values x and y, one can define countless types of averaging values, such as A G H
^rr^ (xy}* - ( x ~^y } r ,
Arithmetic mean; Geometric mean; Harmonic mean; r ^ 0 Power mean.
Both A and H are P-means corresponding to the values r = 1 and r = —I respectively. What happens if any couple of such operations is 233 Copyright © 2002 Marcel Dekker, Inc.
234
CHAPTER 9. HISTORICALLY IMPORTANT
EQUATIONS
iterated? In all cases they give rise to a system of two difference equations of the form
Xn+l
=
/(^n,2/n)
2/n+l
=
g(Xn,yn)
I9-1)
with the common feature that all the points of the line y — x = 0 are solutions of the system defining the critical points, given by (9.2)
Many common properties of the system (9.1) can be derived from the properties of the means. We list some of them here. For simplicity we denote a generic mean by M(x.y). PI If x > Q.y > 0, then m.m(x.y) < M(x,y] < max(x.?/); the equality occurs only when x = y< P2 two different means Mi(x,y] and M^x.y) give two different values when x ^ y: P3 M(x,y) is an homogeneous function of degree one, i.e. M(\x,\y) = XM(x.y). Such properties permit us to prove two important results about the location of the limit points. The first one, almost trivial, says that the motion is entirely in the first quadrant. The second is the following theorem. Theorem 9.1.1 If the sequences xn andyn converge, then their limit points need to coincide. Proof.
In fact, if lim yn =
n—>oc
then such limit points must satisfy (9.2). If x ^ y. P2 would be violated for both functions f ( x . y ) and g(x.y}. D Unlike the cases considered in the previous chapters, where the set of critical points was a discrete set, here the limit points depend on the initial values (XQ. yo}. The study of this type of iterative process can be traced back to Gauss and even earlier. In the following subsections we shall present a few of them not in their historical sequence, but starting from the easiest case.
Copyright © 2002 Marcel Dekker, Inc.
9.2. ARITHMETIC-GEOMETRIC
9.1.1
(BORCHARD)
235
Arithmetic-harmonic mean
In this first case the arithmetic and the harmonic means are combined and iterated giving
(9.4)
with XQ ^ 7/0 both positive. By multiplying both equations, we obtain at once that xn+iyn+i = xnyn, i.e. that xy is constant along the motion, or equivalently the motion stays on the hyperbola xy = XQJ/O (and, as mentioned above, in the first quadrant). Moreover, one has _
x
n+l ~ TJn+l — 7^ ; r> 2(x n + J/n)
i.e. for n > 1 we have xn > yn. Apart perhaps for the first point, the motion stays on the hyperbola below the bisectrix of the first quadrant. The direction of the motion can be also easil deduced. In fact we have X
_ —
2
which, as indicated above, is negative. The direction on the hyperbola is towards the bisectrix, and then the two sequences converge. By Theorem 9.1.1, they converge to the point (Y/XO^/O? ^/XQyo}• Since such convergence is quadratic (see Problem 9.1 ), the system (9.1) can be used to numerically find the square roots of any positive number.
9.2
Arithmetic-Geometric (Borchard)
Here the couple of arithmetic and geometric means is considered, although not simultaneously. First we perform the arithmetic mean and then the geometric mean. The technique of using the updated variable in the following equations of a system, well-known in numerical analysis as the Gauss-Seidel process, usually leads to an improvement in the rate of convergence, while leaving unchanged the limit point. Here, since the limit point is sensitive to the initial point, such a process changes such points as well. The system is now described by
n+l
_ —
x
x
yn+l
=
(Xn+lVn}*-
It is not difficult to check that
Copyright © 2002 Marcel Dekker, Inc.
n i 7/
(9-6)
236
CHAPTER 9. HISTORICALLY IMPORTANT
EQUATIONS
- Vn+l = o 2 ( Z n + yn)2 + (2yn)2
from which it follows that the sign of xn — yn is constant. The motion will then always be below or above the bisectrix of the first quadrant, according to the position of the initial point (xo>2/o)- Suppose that xn < 2/o> it will always be xn < yn. We can then define X7
cosa = — getting
cosan+i = -2 —^ (Xn +, \2^
We then have a
_ &n
n+l — — ,
i.e. a n — 2~ n ao- From this result one easily derives that ihm • — ^n — 11.
n^co yn
It also possible to obtain explicitly both xn and yn as functions of n. For this we need to prove the following identity:
n
J
cos2-Ja 0 =
^
n . °n . 2 n s m 2 n ao
, N 9.7)
The proof is almost immediate by considering that sin2-J'+1Q!o cos 2 Jar) — ———-—:— and substituting in (9.7). We are now in the position to obtain the result. Since x n +i = yn cos" a n +i, we have
from which we get
n Conccrnine: xn. we have
Copyright © 2002 Marcel Dekker, Inc.
, cos2 Ja0 = yo
sina 0
9.2. ARITHMETIC-GEOMETRIC
xn = yy ncos2
(BORCHARD)
an u =
1
237
sin a0
-VQ. 2ntan2-na0y
It is now an easy matter to show that smc r v *0 (yg-xg) 1 / 2 hm xn - hm yn = - y0 = —--^—-.
n->oo
n-+oo
Q;O
(9.8)
Iii the case XQ > yo the limit is x o) 1//2 —— hm xn = hm yn = - y0 (2/0 =~-*oc n-+oc QJQu arccosh^
(9.9)
(see Problem 9.2).
9.2.1
Arithmetic-geometric mean II
Contrary to the system considered in the previous section, the present system does not use the updated value in the second equation. The resulting limit is completely different and cannot be expressed by means of elementary functions, but it is expressible in terms of integrals. The system is
yn+i
=
•Era i" yn
/,-,-, n \
(x n y n )2.
(9.11)
Since 2-n+l ~ Un+\
==
/
"2
*? \ 2
7^ (2-n ~ J/n j i
it follows that xn > y n , n > 0 independently of the sign of XQ — yo- Moreover, one has
and
i
I
-
yn+i -yn = y2(xn -yn) > 0, i.e. for n > 0, the sequences xn and yn arc decreasing and increasing, respectively. Apart from the initial point (XQ, yo], the successive points (x n , yn] will be contained in the triangle having vertices at the points (XI,T/I), ( x i , x i ) and (3/1,2/1). The convergence of the process is then evident. In the case %Q > 2/0) the limit point is (see [162]) lim xn — lim xn —
n—>oo
n—>oo
2
As result, the successive iterates can be used to approximate the above integral.
Copyright © 2002 Marcel Dekker, Inc.
238
9.3
CHAPTER 9. HISTORICALLY IMPORTANT
EQUATIONS
The Weierstrass Method
In this section we give an overview of another historically important system of difference equations. It was proposed by Weierstrass, he used it to prove the fundamental theorem of algebra. It has been rediscovered several times and used as an algorithm to simultaneously approximate all the roots of a complex polynomial. The starting point is the Vieta's relations among the roots z\, 2 2 , . . . , zs and the coefficients ps,ps-\,. ..po of a polynomial, i.e.
z\ + Z2 + . . . + zs
—
=
0
Ps
. .. + zs-izs +-^=—
— 0
f_iy'PQ ps
_ o
Ps
Let be z — ( 2 1 , 2 2 , . . . , 2S)T and F(z) the vector whose entries are the left sides of the above equations, which is then written as F(z)=0.
(9.12)
It turns out that the entries of any solution of (9.12) are roots of the polynomial ,s
p(, \ — \^ .,r' n J JJ-j JL
J ^ JLr J —
.
(a i "*} ^ a. L O j
?:=0
The Jacobian J(z] of (9.12) is a Vandermonde-like matrix whose determinant is not equal to zero iff 2; ^ Zj, for i =^ j. When this happens, the Newton method applied to (9.12) leads to -J~i(zk}F(zk}.
(9.14)
When the above sequence converges, it provides the roots of the polynomial. Of course a necessary condition for the convergence is that in each step Zi ^ Zj. which leads, to be conservative, to the assumption that the roots are simple. It has been conjectured that in the latter case the method converges for almost all initial points. The proof has been provided only in the case s — 2. We shall report this case for brevity and also because it provides an example of dynamics similar to those presented in the previous sections. Let f(x] = x~ — ax+b. whose roots are supposed real and z — (x, y)T. For simplicity, we shall also suppose that the roots are positive (a > 0. b > 0). Equation (9.14) reduces to
Copyright © 2002 Marcel Dekker, Inc.
9.4. DIFFERENCE EQUATIONS AND PRIME NUMBERS
xn+i \
( xn \
1
xn
-1
239
xn + yn - a
i.e. Xn+1
n
n
(9.15)
xn-yn
As expected, the vector (xn,yn)T needs to stay away from the bisectrix x — y = 0. It is easy to check that xn + yn — a, i.e. after the first iteration, the motion will stay on the line x + y — a — 0. This result is also valid for the general case, where the motion stays on the hyperplane
i=0
n Ps
Going back to the bidimentional case, it is easy to check directly, or by using the fact that the successive iterations stay on the straight line (see Problem 9.4), that f ( x n ] = f ( y n ) . This permits us to show that ,
0~
i.e. the motion stays always on the same side with respect to the hyperbola xy — b = 0 on which stays the solution (x* , y*). Suppose now that xn > yn, by subtracting the two entries in (9.15) we get (xn - y )2 - (f(x ] + f(y }}
b-x y
n n O n n Xn+i - yn+i = -—n:= 2-—-— > 0, Xn yn) X y n
n
i.e. all the successive iterates remain below both the bisectrix and the hyperbola xy — 6 = 0. The latter property implies that xn is greater than the largest root and then f ( x n ] > 0. As a consequence, for n > 0, xn is decreasing and yn is increasing such that xn — yn > 0. The convergence then easily follows.
9.4
Difference Equations and Prime Numbers
In Section 1.3 a difference equation was presented whose solution yn assumes prime values for all n. There are also many examples of difference equations whose solutions relatively prime values for all n. Here are two of them. Example 20 Let us consider the discrete problem 2/n+i -yn(yn - 2) - 2 = 0,
2/0 = 3.
The following facts can be deduced almost immediately:
Copyright © 2002 Marcel Dekker, Inc.
240
CHAPTER 9. HISTORICALLY IMPORTANT
EQUATIONS
1. At n-th step yn is the product of all the previous values plus 2. In fact
yn+i -2 = yn(yn - 2) = ynyn-\(yn-\ - 2) = . . . = ynyn-\ • • • 2/12/02. All the successive values are odd numbers; 3. All the successive values are relatively prime. In fact, if yn and yj had a common factor, such a factor would be divisible by 2 and thus be even. 4. The explicit form of the solution is
yn - 22" + 1, i.e. the set of Fermat numbers. The proof simply follows by setting yn = zn + 1. which reduces the above equation to zn+\ — z% (see Problem 1.28). Similar considerations apply to the following example. Example 21 yn+i - yn(yn - 1) - 1 = 0,
2/0 = 2.
Both example are consequences of Euler theorem, which in complete form states: Theorem 9.4.1 I f p o , p \ , . . . .pn is any set of relatively prim,e numbers, then P = POPl . . . p n + l
is a new number which is relatively prime to the previous one and of course different. Proof. When p is divided by each pt. it gives 1 as remainder, and then it is relatively prime to all of them. The following consequence is well-known. Corollary 9.4.1 // the above set contains all the primes less or equal to pn, then a new prime p is found and consequently the number of primes is infinite.
Copyright © 2002 Marcel Dekker, Inc.
9.5. PROBLEMS
9.5
241
Problems
9.1 Prove that the convergence of (9.1) is quadratic. 9.2 Prove the relation (9.9). 9.3 Show that the algorithm •Z-n+l —
n
2/n+l —
— -4Xn
J/
is equivalent to + — ],
^ = Vxoyo
already considered in (1.27). 9.4 Show that f ( x n ) = f(yn] in (9.15). 9.5 Show that for a — 0, the Weierstrass method becomes the method described in (1.27) 9.6 Show that the equation in Example (21) is a consequence of Theorem 9.4.1.
9.6
Notes
The few difference systems presented in Section 9.1 are part of a large family based on the use of couples of means and studied in the last two centuries. Such studies indeed can be traced back to Gauss and even earlier, and they continue to attract the attention of many mathematicians. The interested reader may consult Tricomi [173], Carlson [32], Schoenberg [162], Gatteschi [68], and Wimp [180]. There is a historical overview along with interesting generalizations in Carlson. The Weierstrass method has attracted the attention of many people because of its interesting theoretical properties (see Smale [164]) and also in connection to the search of polynomial roots (see for example Kerner [100], Durand [59], Ehrlich [60], and Aberth [3]). A similar method, although more general since it can also be used for searching roots of analytical functions, has been proposed by Pasquini and Trigiante [148]. In the latter paper they present the extension to the case of multiple roots, and a proof of the convergence in the case when all roots are real is also provided. The Euclid theorem, as well as the material of Section 9.4, can be found in Sylvester [170].
Copyright © 2002 Marcel Dekker, Inc.
Appendix A
Function of Matrices A.I
Introduction
Let A be a fh x m matrix with real or complex elements, and let us consider expressions such as
p(A) = 5>^,
(A.I)
2=0
where k G N+ and pi E C. Such expressions are said to be matrix polynomials. To each of them we associate the polynomials
i=0
sharing the same coefficients. There are, however, some differences of algebraic nature among polynomials defined in the complex field and those defined on a commutative ring (the set of powers of a matrix). For example, it can happen that An — AAn~l — 0 but neither A or An~l is the null matrix. More generally, it can happen that p(A) — Y[i=i(A — Z{I] with Zi G C is the null matrix and A / Zil, which cannot happen for complex polynomials. The Cayley theorem, in fact, states that if z\, 22, • • • , zs are the eigenvalues of A with multiplicity ra?, letting P(A)
= f[(A-ZiI)**
(A.2)
z=l
one has p(A) = 0. The polynomial p(z) is called the characteristic polynomial associated to the matrix A. Let ip(z] be the monic polynomial of minimal degree such that ij>(A) = 0, and let m be its degree. 243 Copyright © 2002 Marcel Dekker, Inc.
(A.3)
244
APPENDIX A. FUNCTION OF MATRICES
Theorem A. 1.1 Every root of the minimal polynomial is also a root of the characteristic polynomial and vice versa. Proof. such that
Dividing p ( z ) by 0(z) we find two polynomials q(z) and r ( z ) p ( z ) = q(z)^(z) + r ( z )
(A.4)
where the degree of r(z] is less than m. For the corresponding matrix polynomials we have 0 = p(A] = g(A)'0(A) + r(A). Since ?/>(A) = 0 and it is minimal, it follows that r(A) is identically zero. Thus (A.4) becomes p(z] = q(z)'il'(z) proving that the roots of ijj(z) are necessarily roots of p ( z ) . Now suppose that A G C is a root of p(z] and therefore also an eigenvalue of A. For every j G N+, it follows that AJx — XJx. where x is the corresponding eigenvector. Then we have fd>(A}x — i/'(A)x from which it follows il'(X) = 0, proving that A is also a root of y ( z } . n Let h(z] be any other polynomial of degree greater than m. We can write h ( z ) = q(z)u'(z) + r ( z ) where the degree of r ( z ) is less than m. Consequently
h(A) = r(A).
(A.5)
The last result shows that a polynomial of degree greater than m and a polynomial of degree less than m may represent the same matrix. Let h ( z ) and y ( z ) be two polynomials such that h(A) — g(A). Then the polynomial d(z) = h(z) - g(z] (A.6) annihilates A; that is, d(A] = 0. It follows that it is possible to find q(z) such that d(z) — q(z}'il}(z). Let 777,1,77^ • • • , ^.s, be the multiplicities of the roots of 0(2). Then we have V(*i) = v'&) = ... =^(mi^(zl] = 0,i - 1, 2 , . . . , s, from which d ' ( z i ) = d ' ( z i ) = ... = d^mi~l\Zi) = Q,i = l , 2 , . . . , s . This result shows that
(A.7)
for i — 1.2..... s. Two polynomials satisfying the above condition are said to assume the same values on the spectrum of A. Now. suppose that the two polynomials assume the same values on the spectrum of A. It follows that d(z] has all the Zi as roots of respective multiplicity ?n,? and will be divided exactly by ty(z): that is. d(z] = q(z)'il>(z). Then d ( A ) = g(A) — h(A] — 0 and the two polynomials represent the same matrix. The foregoing arguments prove the following result.
Copyright © 2002 Marcel Dekker, Inc.
A.I. INTRODUCTION
245
Theorem A. 1.2 Let g(z] and h(z) be two polynomials on C. Then g(A) — h(A) iff they assume the same values on the spectrum of A. Definition A. 1.1 Let f ( z ) be a complex valued function defined on the spectrum of A, and let g(z) be the polynomial assuming the same values on the spectrum of A. The matrix function f ( A ) is defined by
The problem of defining f(A) is then solved once we find the polynomial g(z) such that g(l](zk}'= }(i\zk] for k = 1, 2, . . . , s, i = 0, 1, . . . ,mk-i. The later is solved by constructing the interpolating polynomial (LagrangeHermite polynomial) s
mk
1
^)^)'
(A-8)
where $ki(z] are polynomials of degree m — 1 such that ki
' •"
/', k = 1 , 2 , . . . ,s, ' i, r = 1 , 2 , . . . ,mjfe.
"J "
For example, for m7- = 1, z — 1, 2, . . . . n,
It can be proved that the functions 4>ki(z) are linearly independent. The matrix polynomial is then s
mk (l l) z
~ ( ^kr(A}.
(A.9)
The matrices 4>ki(A) are called component matrices of A. They are independent of the function f ( z ) . As usual in this field we shall put
ki(A) = Zki.
(A.10)
Such matrices, being polynomials of A, commute with A. With this notation (A. 10) becomes s mk 1 f ( A ) = g(A) = V Z—/ V Z—, /^- )(z fc )^-.
(A-ll)
The set of matrices Zki are linearly independent. In fact if there exist constants cki not all zero such that -0,
Copyright © 2002 Marcel Dekker, Inc.
246
APPENDIX A. FUNCTION OF MATRICES
the associated polynomial s
mk
would annihilate A and be of degree less then the degree of the minimal polynomial.
A. 2
Properties of Component Matrices
Let us look for some properties of the component matrices Zkl. Taking f ( z ) — 1, we get from (A. 11)
Also, taking /(z) = z. we see that s
A = ]T (zkZkl + Zk2),
(A.13)
from which s
s
k=l
3=1
k=lj=l Starting directly from f ( z ) = z1, one has Zkl + 2zkZk2
fc=i Comparing the two results, it follows that
Zk\Zj\
=
Proceeding in a similar way, it can be proved in general that ZkpZir ZkpZkl ZkpZk2
Copyright © 2002 Marcel Dekker, Inc.
= 0 = Zkp — pZk,p+i
k ^ i, p>l. p > 2.
(A. 14)
A.2. PROPERTIES
OF COMPONENT MATRICES
247
From the last relation it easily follows that
1
j
It is worth noting that from the second relation of (A. 14) we get Z2ki = Z fcl ,
(A. 16)
showing that the matrices Zki are projections. Multiplying the expression s
s
A — zT — VVz — z-\Z
-\- V^ Z
k=l
k=l
by Zn, and considering (A. 14), one gets, s
s Zk
(A — ZiI)Zn = 2—<( fc=l
z
~ i}ZklZn + / ^ Zfc2^il = ^z2fc=l
(A-17)
Because of (A. 15) and (A.17), we obtain Zkp =
(p- l)\Z^2
=
(p- 1}\^A ~ Zkl^
Zkl
'
(A- 18)
From this result, it follows that (A. 11) can be also written as 5
=
mk
f(i-l)(
\
E E V*r - nL ) -. (A ~ t*1?-1^-
fc=li=l
( A - 19 )
Let us now consider the function f ( z ) = -^^ where A 7^ zk. The function f(A) = (A — A/)"1 is expressed by
(A - A/)- 1 = - £ E(* - 1)!(A - z*)-1^
(A.20)
fc=l i=l
from which, considering (A. 15), we get
(A - A / ) -1
fc-i + (A - zk)-3Z2k2 + . . . + (A Because of (A — XI) — — Y^k=i[(^ ~ zk)Zki - ^2], we then obtain
k=l
Copyright © 2002 Marcel Dekker, Inc.
APPENDIX A. FUNCTION OF MATRICES
248 (A
-2 ^ 2
ZkiZk2 - (A
J
k1
k=l
from which results
= o,
(A.21)
showing that the matrices Z^2 are nilpotent. This result allows us to extend the internal sum in formula (A. 19) up to ra/c, the multiplicity that the fc-th eigenvalue assumes in the characteristic polynomial. In fact, since m^ < m^, Z™2k+J — 0 for j = 0 . 1 , . . . , m^ — m^, and hence (A. 19) can be written as (A.22) Definition A.2.1 An eigenvalue z^ is called simple if rrik = 1. Definition A.2.2 An eigenvalue z\. is called semisimple if nik — 1 (Zk2 — 0). A semisimple eigenvalue is not simple (or degenerate) if nik — 1 and In both cases the terms containing (A — Z k I ) p ~ l Z k i with p > 2 are not present in the expression of
A.3
Particular Matrices
We discuss in this section matrices which frequently appear in the applications considered in this book. Example 22 An important class of matrices are the so-called companion matrices (or Frobenius matrices), defined by
1
° 0
1
0
0
1
...
A=
0
\
0 0
\
...
0
1
—an-i
For this class of matrices m7; = mi (for all values of i): that is, the characteristic polynomial and the minimal polynomial coincide. In fact, let EI be the z'-th unit vector in IRn. It is easily checked that E^A° — E\,
Copyright © 2002 Marcel Dekker, Inc.
A.3. PARTICULAR MATRICES
249
EI A = EI and, in general, E\A1 = E^+i, for i — 0 , 1 , . . . , n — 1. If there were a polynomial ^>(A) = Y^=Q aiAl of degree m less than n such that tl'(A) = 0, it would imply
Such a result is not possible because the unit vectors are linearly independent. As a consequence, companion matrices do not degenerate semisimple eigenvalues. Example 23 Let f ( z ) = ezi and A be an n x n matrix. Then s
cAt — \//
rhk
^ x/ ^
^ (
J
,i-\
A ~ Zjc±rV"! 7 • ~(^ ;^_ r ]\\ ~ _r/ o~fc£/' e \ i\ ) ^/rl V ft/ ft-l
/l-rx.^O/ A OQ\
V
/
If the eigenvalues are all simple, the previous relation becomes _
.
(A.24)
k=\
If Zj is a semisimple eigenvalue, (A.22) becomes s
mk
,i-\
l Zkt i l Z]t k &At _ y^ y^ Z^- Z^ (« _ -\\\ c (A ^ — zit} ' ~ Z^ fcl+ e Z -J^l '
\(A• 25) i
Theorem A.3.1 // Rezi < 0 for i = 1, 2 , . . . , s, J/ien
lim e^ = 0.
t^oo
Proof.
From (A.23) it follows easily.
D
Theorem A.3.2 If for i = 1 , 2 , . . . . s, Rezi < 0 and those for which 0 are semisimple eigenvalues, then eAi remains bounded as t —» oo. Proof.
It follows easily from (A.24).
D
Example 24 Let f ( z ] = zn and
(A-26) fc=l 4=0
Copyright © 2002 Marcel Dekker, Inc.
APPENDIX
250
A. FUNCTION OF MATRICES
If the eigenvalues Zj of A are all simple, then (A. 26) becomes (A.27) k=\
If Zj is semisimple, (A. 25) reduces to
/_^ />=0> EE
(A.28)
k=l
Theorem A.3.3 // \zr < 1 for i = 1, 2 , . . . , s, then lim An = 0.
n—>oc
Proof.
The proof easily follows from (A.26).
D
Theorem A.3.4 If for i = 1, 2 , . . . , s, (z^l < 1 and t/ie eigenvalues for which Zi\ = 1 are semisimple, then An remains bounded as n -^ oo. Proof.
The proof is easy to see from (A.28).
D
It may happen, however, that for a multiplicity of a higher order and consequently higher dimension of the matrix, the terms in (A.28) may become large. For example, let us take the matrix / A J3 0 A
0 j3
...
0 \ 0
A
0H 0 / 0 1 0 0 0 1
(A.29)
0 A/ 0 \ 0
H =
[A.30) 1 0 )
s
and H = 0. Then (A.28) becomes (A.31) Multiplying by E = (1,1.. . . , 1)T and taking n ~ s — 1, we have \n-lQllilE. ?'=!
Copyright © 2002 Marcel Dekker, Inc.
(A.32)
A.4. SEQUENCE OF MATRICES
251
The first entrv of this vector is = (A + /3)n,
(A.33)
i=0 W
from which it is seen that \\AnE\\ > |A + /3|n.
(A.34)
This implies that the component of AnE will grow even if |A| < 1, but |A + f3\ > 1. Eventually they will tend to zero for n > s — 1 because from (A.31) the exponents of A become larger and larger while those of (3 remain bounded by s — 1. But in the cases (as in the matrix arising in the discretization of P.D.E.), where 5 grows itself, the previous example shows that the eigenvalues are not enough to describe the behavior of An. This will lead to the introduction of the concept of spectrum of a family of matrices (see Section 7.7).
A.4
Sequence of Matrices
Often functions of matrices are defined using the Taylor series of the respective complex valued functions f ( z ) . We will show now that such definitions are compatible with the definitions given above. Let us consider a sequence of complex valued functions /i(z), / 2 ( z ) , . . . defined on the spectrum of A. Definition A.4.1 The sequence /i, /2, • • • converges to a function / on the spectrum of A if, for k — 1, 2 , . . . , s, lim fi(zk]
=
T f ' f \ hm JjAzki % \ r^ /
=
I—»00
f(zk}, -C> / J \
\ i (Zk) l\ /
The following theorem is almost evident (see Lancaster). Theorem A. 4.1 A sequence of matrices f i ( A ) converges if the sequence fi(z] converges on the spectrum of A. Corollary A. 4.1 Let A be an s x s complex matrix having all the eigenvalues inside the complex unit disk. Then
i=0
Copyright © 2002 Marcel Dekker, Inc.
252
APPENDIX A. FUNCTION OF MATRICES
Proof. Consider the sequence fi(z) = H}=o z^ f°r ^ — 0- This sequence (and the sequences obtained by differentiation) converges if \z\ < 1 to (1 — z}~1. It follows that the limit of f i ( A ) exists and the limit is (/ — A)" 1 . D Corollary A. 4. 2 Let A be an s x s complex matrix. Then
Proof.
Consider for i > 0 the sequence fi(z) = 5Z}=o ^r- This sequence
converges for all z e C. Likewise for /,- (z), follows that //(A) converges to eA. D
A. 5
Jordan Canonical Form
From (A. 13), it results that s
A = ]T>fcZfcl + Z fc2 ), fc=i
(A.35)
and from (A. 12) it follows that the space IRm can be decomposed into s subspaces M\ , . . . , Ms , defined by Mj^ZjiIR™,
j = l,2,...,s.
(A.36)
Similarly, from (A. 14) it follows that for i ^ j Mi n MJ = 0.
(A. 37)
Let inl — dim A/7. Then Y^l=\ ™li — ™- and it is seen that ffi% is the multiplicity of z% as root of the characteristic polynomial. We want to appropriately choose a base for the subspace MJ. Lemma A. 5.1 Let x ( j ) G Mj,ZJ2x(^ / 0 for 0 < i < p and Z^x^ = 0. Then the vectors Z'- 2 z' , i = 0. 1, . . . . p — I are linearly independent. Proof.
From
P-I ^c,Z]2x^^O,
(A.38)
7= 0
multiplying successively by Z^ •, ^j2^-. • • • , it follows that c?; = 0. D We shall call the set of vectors Zj2x^^ for i — 1. 2, . . . .p — I the chain starting at x^ . Such vectors are also called generalized eigenvectors associated with x^ .
Copyright © 2002 Marcel Dekker, Inc.
A.5.
JORDAN CANONICAL FORM
253
Lemma A.5.2 Let x^ G M,-,ZJ2x^ ^ 0 for 0 < i < p and Zj2z(j) = 0. Suppose that there exists y^ £ Mj linearly independent from the previous defined set of vectors {Zj2z^^}. Then the vectors {Z^2y^} are linearly independent from the previous set. Proof.
Similar to the previous proof.
D
Going back to our problem we shall distinguish the following cases: (1) If rrii = 1 = mi, for i = 1 , 2 . . . . , n, it follows from (A.35), x7; G M 7 , that Ax, = zlxl.
(A.39)
Defining the matrix X = ( x i , . . . , x n ), we obtain
0
•••
0 \
0
AX = X
(A.40)
0
\ Considering that the matrix X is nonsingular, we define the diagonal matrix B similar to A by
(2) ra? = l,m ? < ra?;. In this case we can choose m 7 linearly independent vectors in M^, such that (it is Z12 = 0): AXJ = ZiXj,
j = 1, 2, . . . , rrii
and the matrix A can be transformed in diagonal form, as in the previous case. (3) At least an rrij is different from 1, rrij = frij and ZJ2 ^ 0 for t < ni-j — 1. Consider the choice of vectors xj
= Z^XQ . i = 0. 1. . . . , rrij — 1, where
XQ is a vector in Mj. From Lemma A. 5.1, these vectors are linearly independent and can be chosen as a base for Mj. From (A. 35) we have
Defining the matrix X = (XQ , . .. , x'rr? _ j . . . . . x0 have from the above relations:
Copyright © 2002 Marcel Dekker, Inc.
x
m s -i )>
we
APPENDIX A. FUNCTION OF MATRICES
254
AX = X
V Since X is rionsingular. a matrix J similar to A can be defined as J = X~1AX = diag(Ji,J2,...,Js). The structure of B is block diagonal; each block has the form 0 1
\
'-.
(A.42)
(4) At least an rrij is different from f arid ZJ2
0 for 0 < t < rrij. Choos-
ing x{ we define the set of vectors x- — for 0 , 1 . . . . , t < t. These t vectors are linearly independent (see Lemma A.5.1). but they are not enough to form a base of Mj. Choose another vector xp independent of the previous ones and define x^ = Zj2X z and so on until a set of rrij linearly independent vectors has been found. Then we proceed as in the previous case.
The matrix J is block diagonal as in the previous case, but the block corresponding to the subspaces Mj can be decomposed in different subblocks with each one corresponding to one chain of vectors. Each subblock is of the type (A.42). The matrix J is said to be the Jordan canonical form of the matrix A. Each chain associated to Mj contains an eigenvector associated to Zj (the last vector of the chain). The number of eigenvectors associated to Zj is the geometric multiplicity of the chain. The dimension of Mj is the algebraic multiplicity of zr
A.6
Norms of Matrices and Related Topics
The definitions of norms for vectors and matrices can be found on almost every book on matrix theory or numerical analysis. We recall that the most used norms in a finite dimensional space IRS are:
Copyright © 2002 Marcel Dekker, Inc.
A. 6. NORMS OF MATRICES AND RELATED TOPICS
255
(1) Ni = E?=iM, (2) IM|2 = (E?=it;?) 1/2 , (3) \\v\\oo = maxi
(2')
(3') H^Hoo = maxi<j< s where A is any s x s complex matrix, AH is the transpose conjugate of A, and p(A) is the spectral radius of A. Given a non-singular matrix T, one can define other norms starting from the previous ones, namely, \\V\\T = \\Tv\\ and the related consistent matrix norms ||^4||T — HT^-ATII. If T is a unitary matrix and the norm chosen is || • ||2, it follows that \\A\\ = \\A\\T(A.44) For all the previously defined norms the consistency relations P*II<MIIINI
(A.45)
and
(A.46) hold, where A,B,x are arbitrary. Theorem A. 6.1 For every consistent matrix norm one has p(A] < \\A\\.
(A.47)
Proof. Let A be an eigenvalue of A and v the associated eigenvector. Then we have
from which (A.45) follows.
Copyright © 2002 Marcel Dekker, Inc.
D
256
APPENDIX A. FUNCTION OF MATRICES
Corollary A. 6.1 If \\A\\ < 1, then the series f>'
(A.48)
i=0
converges to (I — A)" 1 , and
Proof. From Theorem A. 6.1 it follows that p(A) < 1. and the results follow from Theorem A. 4.1 (see also Exercise A.I). D Another simple consequence is the following (known as the Banach lemma). Corollary A. 6. 2 Let A, B be s x s complex matrices such that HA" 1 )] < a. | A — B < 8, with a/3 < 1. Then B~} exists and
Proof. Let be C = A"1 (A - B}. By hypothesis \\C\\ < a/3 < 1. Then (/ - C)-1 exists and |(7 - C)~l < 1/(1 - a/3). But B~} = (I - C)~lA~l, and then \B~l < a/(l - a/3). D Theorem A. 6. 2 Given a matrix A and e > 0, there exists a norm \\ • \\ such that \\A\\ < p(A) + e. Proof. Consider the matrix A' — ~A and let X be the matrix that transforms A' into the Jordan form (see section A. 5). Then -X~1AX = Jf = -D + H. c ( where D is a diagonal matrix having on the diagonal the eigenvalues of A and H is defined by (A.30)'. For any one of the norms (I'), (2'), (3'), ||#|| = 1, and p| < p ( A ) . It follows that \X~1AX\\ = \\A\ x < \ D\\ +e\H\\ < p(A) + e. which completes the proof. D
A. 7
Nonnegative Matrices
In many applications one has to consider special classes of matrices. We define some of the necessary notions below. Definition A. 7.1 An s x s matrix A is said to be (1) positive (A > 0) if avj- > 0 for all indices. (2) nonnegative (A > 0) if a/j > 0 for all indices.
Copyright © 2002 Marcel Dekker, Inc.
A.7. NONNEGATIVE MATRICES
257
A similar definition holds for vectors x e Rs considered as matrices s x 1. Definition A.7.2 An s x s nonnegative matrix A is said to be reducible if there exists a permutation matrix P such that
PApT
=(c D
where B is an r x r matrix (r < s) and D an (s — r) x (s — r) matrix. Since PT = P~l, it follows that the eigenvalues of B form a subset of the eigenvalues of A. Definition A.7.3 A nonnegative matrix A which is not reducible is said to be irreducible. Of course if A > 0 it is irreducible. Theorem A.7.1 If A is reducible, then all its powers are reducible. Proof.
It is enough to consider that
PA*PT = PAPTPAP = ( B"
° )
u2
V °
and proceed by induction.
)
D
Definition A.7.4 An irreducible matrix is said to be primitive if there exists an m > 0 such that Am > 0. Definition A.7.5 An irreducible matrix A is said to be cyclic if it is not primitive. It is possible to show that the cyclic matrices can be transformed by means of a permutation matrix P to the form
•-•
0 T
PAP =
Gr\
i
0
0
0
G2
!
0
••• Gr-i
0
(A.50)
J
In case when A is irreducible we have the following result (due to PerronFrobenius). Theorem A.7.2 // A > 0 (or A > 0 and primitive) then there exist AQ positive and XQ > 0 such that
Copyright © 2002 Marcel Dekker, Inc.
258
APPENDIX A. FUNCTION OF MATRICES
(a) AXQ =
(b) if X 7^ XQ is an eigenvalue of A then \X\ < XQ, (c) AQ is simple. If A > 0 but is not primitive, then (a) is still valid but \X\ < XQ in (b}. Theorem A. 7. 3 Let A > 0 be irreducible with XQ its spectral radius. Then the matrix (XI — A)~l exists and is positive if \X\ > AQ. Proof. Suppose |A| > AQ. The matrix A* = X~1A has eigenvalues inside the unit circle. Hence it follows (see Corollary A. 4.1) that (/ — A*)~l is given by oo
(/-A*)" 1 -^ A*1 T= 0
from which we get 00
(XI - A)" 1 = A- ] (7 - A*)- 1 - A'1 ^(A^ i=0
The second part of the proof is left as an exercise.
Copyright © 2002 Marcel Dekker, Inc.
D
Appendix B
The Schur Criteria B.I
The Schur Criteria
We have seen in Chapters 2 and 3 that the asymptotic stability problem for autonomous linear difference equations is reduced to the problem of establishing when a polynomial has all the roots inside the unit disk in the complex plane. This problem can be solved by using the Routh method (see for example [160] and [19]). In this Appendix we shall present the Schur criteria, the one most commonly used for such a problem. Let
be a polynomial of degree k. The coefficients pi can be complex numbers. Let
(pi are the conjugates of pi) be the reciprocal complex polynomial q(z] = zkp(z-1}. Let S be the set of all the Schur polynomials (see Definition 2.7.4). Consider the polynomial of degree k — 1:
p(V(z)
= vwW ~z P°Q(Z)
It is easy to see that p^l\z] = Y!i=i(pkPi - PoPk-i)zl~l. Theorem B.I.I p(z) e S iff (a) |po| < \Pk\ and (b) pW(z) eS. 259
Copyright © 2002 Marcel Dekker, Inc.
APPENDIX B. THE SCHUR CRITERIA
260
Proof.
Suppose p ( z ) G S and let z\, 22, • • • , Zk be its roots. Then \Pk\
and condition (a) is verified. On the unit circle \z\ = I one has k
D >;=o
- \p(z}\.
Since condition (a) is verified, we get \Pkp(z)
> \Pop(z)\ = \Pop(z}\ = \PQQ(Z)\ = | -p0q(z)\.
Applying Rouche's theorem (see [90]) it follows that the polynomial Pkp(z) and pkp(z) — Poq(z) — zp^(z) have the same number of roots in \z\ < 1, which means that p^(z) 6 S. Suppose now that p ( z ) e 5 and |po| < \Pk\- It follows that on l , \ p k p ( z ) \ > | — poq(z)\ and again by Rouche's theorem the polynomial Pkp(z) has n roots inside the unit disk: that is, p(z) E S. D The previous theorem permits us to define an algorithm to check recursively if a polynomial is a Schur polynomial. The algorithm is very easily implemented on a computer. The next theorem, which is similar to the previous one, gives the possibility of finding the number of roots inside the unit disk. Consider the polynomial of degree k — 1 fc-i
Tp(z) =
pkq(z) =
The polynomial Tp(z] is called the Schur transform of p ( z } . The transformation can be iterated by defining Tsp = T(Ts~lp), for s — 2, 3, . . . , k. Let 7., = Theorem B.I. 2 Let 7S ^ 0, s = l.2....,k. and let si,S2...-.sm be an increasing sequence of indices for which 7>s < 0. Then the number of roots inside the unit disk is given by h(p] — '^Jjl-i(-^~l(k + 1 — Sj). Proof. See Henrici [90]. D Analogous results can be given for Von-Neumann's polynomials. Let A/" be such a set. Theorem B.I. 3 A polynomial p(z] is a Von- Neumann polynomial iff either (!) bo | < bfc| and (2)
Copyright © 2002 Marcel Dekker, Inc.
B.I. THE SCHUR CRITERIA
or (!') p(l\z) = Q and
(2') j/(z) G S. Proof.
See [127].
Copyright © 2002 Marcel Dekker, Inc.
D
261
Appendix C
The Chebyshev Polynomials C.I
Definitions
The solutions of the second-order linear difference equation yn+2 - 2zyn+i + yn = 0
(C.I)
where z G C, corresponding to the initial conditions
yo = 1,
3/1 = z,,
(C.2)
y_! = 0 ,
yo = 1,
(C.3)
and are polynomials as functions of z and are called Chebyshev polynomials of the first and second kind, respectively. They are denoted by Tn(z) and Un(z}.
We list the first five of them below: To(z) Ti(z) T2(z) T3(z) T4(z}
= l U-i(z) = 0 =z U0(z) = l = 2z2-l Ui(z} = 1z = 4z3 - 3z U2(z) =4z2 -I 2 4 = 8z -8z + l U3(z) = Sz3 - 42.
Since the Casorati determinant
Tl(z]
Uo(z)
is equal to 1, the general solution of (C.I) can be written as yn(z) - ciTn(z) + c2Un-i(z).
263 Copyright © 2002 Marcel Dekker, Inc.
(C.4)
264
APPENDIX C. THE CHEBYSHEV
POLYNOMIALS
Let w\ and w^ be the roots of the characteristic polynomial associated with (C.I), that is, w2 -2zw + l = Q. (C.5) It is easy to express Tn(z) and Un(z] in terms of w™ and w^. In fact, considering that w-2 = w^ , one obtains Tn(z)
=
^coshnlog^ + _ 1)1/2
-l)),
(C.6)
2
(C.7) ^
^
For z 6 [— 1, 1], by setting z = cos 9, it follows that w\ — eld and ^nl 2 ) — cosh n log el = coshnz'^ = cosnO. sin(n + 1)0
£w) = —S1I10 —^— , which arc the classical Chebyshev polynomials.
C.2
Properties of Tn(z) and Un(z)
One can easily prove that the roots of Tn(z) and Un+i(z) are 2fc = cos?^ti-,
n
2
fc = 0,l ..... n - 1 ,
(C.8)
and /T7T"
2fc = cos — , fc = l , 2 ..... n - 1 , (C.9) n respectively. Other properties of T n (z) and Un(z] are the following, which can be easil verified.
(1) w?=Tn(z) + (T*(z) (2) T n (2) = T-n(z), (symmetry) (3) Tjn(z] = Tj(Tn(z)},
(semigroup property)
(4) C/ 7 - n _ 1 ( z ) = £7 J -_ 1 (r n (^)), (5) U-n = -Un-2(z), (6) Tn-!(z) - ^r n (z) - (1 - z 2 )I/ n _i(e), (7)
Un^(z)-zUn(z}^-Tn+l(z},
Copyright © 2002 Marcel Dekker, Inc.
C.2.
PROPERTIES OF TN(Z) AND UN(Z)
(8) Un+j(z) + Un-j(z) =
265
2Tj(z)Un(z).
From the last property one can derive many others. For example, (9) Un+j-!(z) + Un-j-^z) = 2TJ(z)Un-l(z), (10) Un+j(z) + Un-j-2(z)
(11) 2Tn(z)Un(z)
= 2TJ + l(z)Un-l(z),
= U2n(z] + I ,
(12) 2Tn+l(z)Un-i(z)
= U2n(z)-l,
(13) 2Tn(z) = Un(z)-Un-2(z). Among the properties of Chebyshev polynomials the following has a fundamental importance in approximation theory. (14) Let Pn be the set of all n degree polynomials having the leading coefficient 1. Then for any p(z) € Pn, max \2l~nTn(z)\ < max
\p(z}\.
The proof of this property is not difficult and can be found in several books on numerical analysis. (15) Tn(z) and U n ( z ) , as functions of z, satisfy the differential equations 7^(z) = nC/ n _i(2:),
(C.10)
U'n(z) = -(z2 - l ) ~ l [ z U n ( z ) -(n (1 - z2)T^(z) - zT'n(z] + n2Tn(z) = 0,
(C.ll)
(1 - z }U'^(z) - 3zU^(z] + n(n + 2}Un(z) = 0.
(C.12)
2
The first two are consequences of the definitions (C.6) and (C.7), and the other can be derived from them. Finally, Tn(z) and Un(z) satisfy the orthogonal properties: (
}
(17)
2 [l Tn(z)Tm(z)
a2
I 26nm for Snm for
- [l (I - Z2)l/2Un(z)Um(z)dz 7T J-l
= 6nm,
(is) A; ^ 0, /c / m, fc = 0. k = n,
where Zj = cos ^ . j — 1 , . . . , n — 1 .
Copyright © 2002 Marcel Dekker, Inc.
n = 0, n>0,
Appendix D
Solutions to the Problems D.I
Chapter 1
1.1 From (1.4) 3/5 = E4yi = Y^j-o (j)^yi and from the scheme A° A1 A 2 A 3 A 4 2/i -2 2/2 -2 0 2/3 0 2 2 1 / 4 4 4 2 0 0 we get 2/5 = 10.
1.3 The first result of problem 1.2 can also be written
(
pia/2
_ f,-ia/2\
^
n
c
2
— 2zsin-e^ ox+6+a/2) /
2
By taking the real and imaginary parts one gets the result. 1.6 Using the Stirling transform one has j3 = j^+Sj^-t-j^ and A"1^3 = l(2) .^^ -,'(4) „ . . . . £1 , ,-(3) , ^ + 2 """J ^ "44 from which one obtains:
j=o
1.4 Let 7/05 2/1, • • • De a sequence. We have
Let ^ = (*+*). Consider then that (see 1.20) A n y 0 = ( TO ^ n ) and
267
Copyright © 2002 Marcel Dekker, Inc.
268
APPENDIX D. SOLUTIONS TO THE PROBLEMS
1.7 From the result of problem 1.3 we have A-I
sin(ax + 6) 2 sin a/2
Setting a = q. b = —q/2 and x = n, we have: A -1 s'm(qn — q/2) A cos an = ; 2 sm o/2 and
cos, =
=
1.10 From x("+*-«) = XW(X - n )(^-") and ;r( x ) - r(rr + 1), (x -
1.13 Using the Stirling transformation, p(x) can be written in the form P(X) — ICi^o a i^^^- Applying A , A 2 . . . . . A f c and putting x — 0 one gets au?, gets — ., 1.14 One has: q-n
/
\
q—n-l
= E
n
1
~ ~ ^—> /
1
. o // ,w,
—1\
J. \
o— n-1
q—n—l
-,
~
V----L
v—>
— —
,
. .. A I/ /U7
— 1
-*-
~
-.
1
n- I
By setting j — I = / in the second term, it is easily seen that the two sums cancel.
Copyright © 2002 Marcel Dekker, Inc.
269
D.I. CHAPTER 1 1.15 One has:
5(g,/,n)
—n
=
q-n-l
+ E
' -1
1.26 Letting yn — zn l the equation becomes zn+\ = zn + 1. 1.27 Letting zn = ^4 1//2 cothy n one has yn+\ — 2yn. 1.28 (B) From \ogyn+i = loga+p\ogyn, by posing zn = \ogyn one obtains — pzn + log a, and then
1.29 By substituting yn = ^f^ the equation becomes zn+\ = which one obtains yn = ^[1 — (1 — 2yo)2"']-
1.30 1.32 It is not restrictive to consider the case i > j. One has
ltf =
3
J
S 'r
s/ \ s
j
= E
5=0
Copyright © 2002 Marcel Dekker, Inc.
/.A /A
I — S I \S
-P + P \ I , I — S
I
\S
, from
270
APPENDIX D. SOLUTIONS TO THE PROBLEMS Then take n = i 4- j. p = m = j.
1.35 From Ps = P(I + K}\ we get R(K) = e~sK(I + K)s. 1.37 From the result of problem 1.36 one has /
un <6exp
\ / , n _i n _i n=i Hl/2M V --i— +6 V exp ,W2M V -
It is known that ££=1 k~l/2 < 2nl/2 ~ L un
Tnen one has
< $ ( exp(/i 1 / 2 M(2n 1 / 2 - 1)) + ]T exp(/i 1 / 2 M(2(n - s - I) 1 / 2 - 1)) V s =0
1.38 Let Vn = c + ~E^n+1ksVs. One has yn < Vn and Vn = Vn+l + kn+iVn+i, then is Vn = flUn+il 1 + fcs)^ < ^ ex P(EUn+i fcs) and for ^ -^ oo, yn < KO 1.39 For a < 1 one obtains y 2 +1 < y2 + bn(yn + y n +i) and them yn+i < Un + bn, form which yn < yo + H?=o ^'- ^or a > -^ one ^as Vn+i ~ a?/n ^ M2/n + yn+i) and then 1/2
^7
n
,
yn+i - a ' yn < bn^-^ [/z -•- < 6 n ,
a yn + yn+i
form which one obtains
D.2
Chapter 2
2.11 The roots of the characteristic polynomial are z\ — a + b\/~d and 2-2 = zf 1 . Then j/n = a.z™ + (3z^n. xn — 7zf + <5zf n . In order to have integer solutions it will be a = j3, 7 — —5. The initial condition leads to a = 1/2 and 5= I/ (26). 2.13 By the Cayley theorem we have p(A) — 0 where p(z) is the characteristic polynomial of A. It follows then that, for all n.
Copyright © 2002 Marcel Dekker, Inc.
D.2. CHAPTER 2
271
Consequently, each entry of the matrix a^v satisfyies
i=0
2.15 Let Cj+i = Si+i -5i+i,rj = ai+i -a^+i,^ = l C ( s ; + flj+i). One has = €i 4- ri — <5f, from which n-l
From this, one deduces that one must keep Y^$i\ as small as possible, and this can be achieved adding first the smaller a* in absolute value. If \a,i\ < a, then \Si\ < ia. Finally one has N-l
kwl < kN + 10~*a Y^ i < kn + lO^a showing that the error grows like TV2. 2.20 From (2.70) we have oo
-r^oo
J2)
n+2
where ^Q(2) = q\(2) — 0 and q^ — l/n. The roots of the denominator are 1 and 2. In the unit circle it can be written 1
2
2
2
3
~2
x-l
°°
2+
with 7i bounded and limn=o Y^o 1i — °° (why?)- Then one has
i=0 2) where Cj = En=2 9n7*-nThen yn = = cn_2 = = Ej=o j+i7n-j-2 <
2.21 Taking / = 0, the error equation becomes p(E)en = /?(!), whose solution is the sum of the general solution of the homogeneous equation and a particular solution. The general solution is given by (2.43), from which it follows that p(z) must be a Von-Neumann polynomial.
Copyright © 2002 Marcel Dekker, Inc.
272
APPENDIX D. SOLUTIONS TO THE PROBLEMS
2.22 Taking / = 0 as in the previous problem and e$ = e\ — . . . = ek-\ = 0, we have (see 2.60) en = p(I)/p(I), and this cannot be zero for n oo(nh < T) unless p ( l ) = 0. Similarly for / = I : p(E)en = 2.24 Use Theorems B.I.I and B.I. 2 of Appendix B. For case (a) one finds D = -i,i.
D.3
Chapter 3
3.3 The roots of the characteristic polynomial are z\ = a + b\/[d) and 22 = z if 1 . Then xn — az™ + (3z^n. yn = 72™ + 5z^n . In order to have integer solutions it will be a = j3, 7 = —5. The initial conditions lead t o a = 1/2. 6= 3.9 Exchanging with care the sum in the expression X^=o Pi(n} Z^j=n0 -^(n+ k — i , j ) g j , one has Y^j=n9i ^i,=QPiH(n + k~~i,j}+gn — gn. (Remember that some values of H are zero.) 3.13 \zs\ ^ 1.
3.14 It depends on the roots of 22 + p\z + p2. It will be H(n + l)+piH(n)+p2H(n-l) H(I)+PlH(0)+p2H(-I}
= 0 = 1.
forn ^ 0
According to the values of the roots, several cases are possible. For example: (a) |2! < 1 , N > 1 ~ \ (zi+Pi (b) \zi\ < I,\z2\ < l,H(n) = (21 - z2)-l(z? - z%) for n > 0 and H(ri) = 0 for n < 0. O. -L <3
XJ (V 77-
J7 IJ —
• Tr sin
1 :' 1/rj / • n •^' t - -— ^-—'7=U
• Ir. sin
^/'J -JJ •
3.16 Let y^ the nih element of the ith column of the inverse matrix C^1. By multiplying CAT and its inverse one shows that the columns of C^ are solutions of the boundary value problems:
= 5\
Copyright © 2002 Marcel Dekker, Inc.
D.3.
CHAPTERS
273
for i = 1, 2, . . . , N. Let z\ and z% be the roots of 7 4- az + fiz2 = 0. The solutions are: for
+
n
z _ 2 where #(n) = 2 z _ z 1 . One sees that for \z\zi\ < 1 and j ^ j > 1, oo. the elements remain bounded for N
3.17 Suppose that A(n) is a companion matrix that is of the form (3.20). AT(n) is not of the same form, and then the components of xn given by (3.46) are not successive values of the solution of a scalar equation. Now consider the matrix \
-Pk(n) 0
V(n} =
V
0
One easily verifies that x^ =
+ 1), where
and tn is the last component of xn. From this new vector one has Xn+iV(n + \)A(n)V-l(n) = xZ+1tl(n). The matrix fi(n) = K(n + V~ 1 (n) is given by 1 0 0 0
0 \
1
Pkn
+1)0
oy
which is the companion form for the adjoint equation defined in Section 2.2. 3.18 In order to have periodic solutions it must be CN = 7, where 7 is the identity matrix. It follows that the off-diagonal entries of CN must vanish. This is impossible for \z\ > 1, since Chebishev polynomials never vanish for such values of \z\ (see Appendix C). For z < 1, UN-I(Z) vanishes for zk = cos(^), k = 1, 2 , . . . , fc — 1. Moreover, we - There will be periodic have Uiv(zk) = ( — l)k and ~UN-2(zk] — (~ solutions in almost every point. 3.19 Since the fundamental matrix is a power of the matrix C, imposing that Cn+J = CnCi , the result is obtained by equating the entries of the both sides matrices.
Copyright © 2002 Marcel Dekker, Inc.
274
APPENDIX D. SOLUTIONS TO THE PROBLEMS
3.20 Let / and r be the two quantities. Divide the decagon in 10 isosceles triangles having all the the vertices at the center. Obviously the vertex angle of each triangle is equal to = Tr/5, while the other angles are both equal to 20. One has then r
. . = 2cos(d>) = p.
sn sin
3.27 From the definition, one has TV
i=0 and
N
x ]T L,$(n, i, j + l)T(j + 1. m) = I + AG(j, j). z=0
3.28 One has N
N
i=0
N i=0 N s=0 TV
TV
3.29
5= 0
n/v-1
^ G(n, s)bs] + 6n = ^4yn + bn. s=0
Copyright © 2002 Marcel Dekker, Inc.
DA.
CHAPTER 4
275
For what concerns the boundary conditions one has: TV
N nN-l
AT
bs = w.
3=0
D.4
Chapter 4
4.3 The system is symmetric with respect to the origin: gi(—x,—y] = —gi(x,y) for i — 1,2. This allows us to consider only the upper half plane. According to the signs of g\ and #2, let us consider the following sets: A = {{x,y)\y>2x} 9\(x,y] > 0 , 0 2 ( z , 2 / ) > 0 B - { ( x , 7 / ) | 7 / < 2 x a n d x 2 ( 7 / - x ) + 2 / 5 > 0} g i ( x , y ] >0,g2(x,y) < 0 C = {(x,y)\x2(y-x)+y5 < 0} 5i(^,y) < Q,9i(x,y} < 0 It is clear that for ( x k , y k ) ^ ^4,Axfc > 0, Ay/j > 0, that is both the sequences Xk,yk are not decreasing. If they remain in A, they will y never cross the line y = 2x and the ratio 9yin2\('xiy \) has to be greater than 2 for all k. This means that x^/ _~^ 5 > 2, which is impossible because the y in the denominator has degree larger than the y in the numerator. The sequences must cross the line y = 1x and enter in B. In a similar way it can be shown that they cannot remain in B. They will enter in C, where both Ao;/- and Ay/t are negative and the sequences are decreasing. Now if y^ > 0 it follows that . -
2/fc +
_ yk(r2k + r6k+y2k- 2xkyk) -
_
and similarly for x^. This shows that the sequences must remain in C and must converge to a point where both Ax^ and Ayk are zero, which is the origin.
Copyright © 2002 Marcel Dekker, Inc.
276
APPENDIX D. SOLUTIONS TO THE PROBLEMS Starting from (—5, 6) for small positive 8, it is easy to check that in the following iterations the points (xfc,yfc) have increasing distance from the origin until they reach the regions B or C, showing that the origin is unstable.
4.5 The solution is ((n no o)\
log n
( ° + 2) 0
^ ' ^ = Io^T^ n
The series X^n0 \y( -> ^o, 2/o)| does not converge, showing that the origin is not l\— stable. 4.9 In this case D is the set of all positive numbers. If yo G D, then y$ G D for all n. Consider V(y) = y/(l + y 0 and AT/
AV
The set E1 is (0, 1} and W(x) —> 0 for x —> oo. Then, according to the theorem, yn is either unbounded or tends to E. In fact the solution ( — 2)" is yn = T/Q , and it tends to zero if yo < 1 and n even and it is unbounded for n odd. If yo — I then yn~\ for all n. 4.11 Let y = 3 — 4z. The equation becomes 2 n+ i = 4z n (l — z n ), which coincides with the logistic equation considered in Chapter 8. 4.12 The eigenvalues of A are AI = | with multiplicity s — 1 and A2 = (1 — -s)/2. They can be obtained easily by considering that A is a circulant matrix and the eigenvalues are the values assumed by the polynomial p(z] — —\(z 4- z2 -f . . . + zs~l) on the si/l roots of the unity. There is global asymptotic stability for s — 2, only stability for s = 3 and instability for s > 3. 4.13 In order to have Vn+\ > 0, it must be Vn - uj(Vn) > 0. If u and v are such that u > n. u — u(u) > 0, i'~ w(v) > 0, and u — v > o;(u)— u;( / y) one has u — uj(u) > v — a;(v). The solution satisfies un = UQ — X^?=o a; ( w j)Since uj is increasing and it must remain positive, it follows that Uj —* 0. 4.14 Let x £ Q(yo), and let there exists a sequence n,; —> oo such that y(nl.n0.yo) -> .r. But y(n i + i,n 0 ,yo) = /(y(^?., ^o, 2/o)) and lim
n n ,
=
;r
^
showing that /(x) G ^(T/O) and that fi(yo) is invariant. Now let y^ be a sequence in Q(yo) converging to y. We shall prove that y G Q (yo). For each index /c, there is a sequence mf —» oo such that y(rn^n().yo} —> y fc . Suppose for simplicity that dist(^, y('m^, HQ. J/Q))
Copyright © 2002 Marcel Dekker, Inc.
D.5. CHAPTER 5
277
< A;"1 and raf > k for i > k. Consider the sequence m^ = ra£. Then dist(y,y(m fc ,no,yo)) < dist(y,yfc) + fc"1 which implies dist(y,y(mjb,no,yo)) -* 0 and then y G
D.5
Chapter 5
5.2 By multiplying equation (5.3) by pn-i we obtain (xpn,pn-l)
= —• &n
Change n in n — 1 in the same equation and multiply by pn(x). We have Q-n-l
5.3 Define Dh — diag(/ig , hi , . . . , /i^_j). The vector Dh-1p(x] is now normalized. The new matrix is then T = 5.6 From (<jj + r^) 2 < 1, we have 0 < (ai — TJ) < 1 — 4<7;T^ 5.8 Divide the two equation (5.29) and (5.30) and obtain
D.6
Chapter 6
6.2 From the mean value theorem one has F(x) - F(y) - F'(x}(y - x) = f\F'(x + s(y - x)) - F'(x)}ds(y - x); Jo \\F(x) - F(y) - F'(x}(y - x}\\ < 7 /' 8ds\\y - x\\2. Jo
6.4 Let QQ = !/?7- One has \\xn+i - xn\\ < and k+m— 1 \\Xk+m-Xk\\
Copyright © 2002 Marcel Dekker, Inc.
<
^ j=k
APPENDIX D. SOLUTIONS TO THE PROBLEMS
278
\xk -xk-i\
TL. i k-\\
x
_9
—2
6.5 Consider the equation un — ^~in un-\- One has -%£- — ^ * , that is ^- is constant and must maintain its initial value u\ft\. Then un — Ai n (wi/£i). It follows then that Ax n < un. Moreover "A^" is decreasing. That is, for m > n, " A ^ m '' < '' A ^ n '' and n - xn"II — < Z-/1' ^ Hz
3=0 p-1
Atn+j rllAXnl
from which it follows that t* — t
lim | x n + p - x
tn
Being ||Azn|| < un one has
6.6 In this case the comparison equation is AU Un =
Copyright © 2002 Marcel Dekker, Inc.
D.7. CHAPTER 1
279
. vy-i whose solution is un = At n (j 1 J . A s before one has II** - *»ll < (** - *n)
6.7 Let yn — I — zn. The equation becomes yn+i — \ (yn + the result of problem 1.27.
1 2
*° j . Apply
6.8 From the given solution one has ZQ — 1 — (1 — 2zo)1/'2 coth k from which k = logfl" 1 / 2 and then zn Z
l — -
1
1+
* 22"nI 1 —/jOn B
~
-,
1
2 2;..W Oj ZZ
from which
6.12 The equation (6.48) is the homogeneous equation related to (6.47) when one takes _ 2/n-i x — n
yn
Imposing that (6.50) satisfies the nonhomogeneous equation one gets (6.49). In fact one has yn+i — 25yn + xnyn + zn = gn from which _
D.7
z
Un+l
n ~ 9n _
Chapter 7
7.1 Consider the term £j=d An~*~lWj and the decomposition (A. 13). It follows that A = Zn + Y^k=2(^kZki + Zk2] = S+Si where d is the number of distinct eigenvalues of A. Using the properties of the component matrices Z^j it is seen that 5J = 5 for all j and limj--^ 5^ = 0. Moreover A? - # 4- S{. The sum E"=o An-i~lWj becomes
^E^ + E^r'-x-3=0
}=0
The quantity SWj is called the essential local error. If the errors are such that SWj = 0, one can proceed as usual. 7.5 Applying the theorem B.I. 3 one has p^(z) = —4Req and p'(z) — 2(z — q). It follows that p(z) € N iff Reg = 0 and |
Copyright © 2002 Marcel Dekker, Inc.
APPENDIX D. SOLUTIONS TO THE PROBLEMS
280
7.8 Rewrite the equation (7.14) such that the linear autonomous matrix A is the companion matrix of &(z), obtaining En+i = (A + Bn}En + Wn, where Bn — $kbT with b is now the vector with components. - cn+j}
, _ 3
2 - h(3kcn+k
Find a bound for bj independent on n and consider that by hypothesis \\A\\ < 1. 7.9 Consider the nonhomogeneous equation C^xN — \XN = 6^N\ where \\XN\\ = 1 for all N and ||<5^|| = e 2 . Component-wise one considers the equation fix^-\ + (a — ^)%n + jx^+l = 6^ whose solutions are chosen such that the initial condition (a — X)x^ +7^^ = 0 is satisfied. In the hypothesis made, the solution will diverge and (7.60) will not be satisfied. 7.12 The matrix AN is the matrix representing the centered second-order discretization, that is
o \
/ -2 1 1 - 2 0
0 0
1
V 0
0
Ax AM
-2 ) NxN
I '
which is symmetric. The spectrum of (Ax2)"1!)2^ is S — {—2 -f 2cos 0,0 < 9 < TT}. The spectrum of D^N is then 51//2 which is an interval of the imaginary axis. The midpoint rule can be used because its region of absolute stability lies on the imaginary axis and it can be chosen appropriately in order that the region contain
D.8
Chapter 8
8.2 From 2.70 one has X ( z ) =
M
— > . ,d.; t—ir — \ '•
The problem reduces to the
study of the roots of the denominator. They are the reciprocal of the characteristic roots and they are outside the circle 5(0, Xl ). The series X(z] = ~Y^=QX\(n}zn is then convergent in this circle. To see if z\ = A^ is less or equal to 1, one considers that 1 — Y^i—i aiz% is monotone decreasing on the real line from 1 to — oo and the z\ will be
Copyright © 2002 Marcel Dekker, Inc.
D.9. CHAPTER 9
281
less or greater than 1 according to the sign of 1 — ]T en. If this quantity is positive, then z\ > 1, otherwise z\ < 1. In the first case one has 1
V x (n} = n=0
^ *
and XI(H) —> 0. In the second case xi(n) —> oo. If 1 — ]T0i = 0, then by Theorem 2.6.2 the solution XI(H) is unbounded. 8.6 By definition f [ x j ( x ) ] = / ( x ) - / ( 3 : ) . If /( 2 )(x) = x then the fraction assumes the value — 1, and vice versa. 8.8 The model is Pi = PoPo (n + l)Pn+i - (Pn + n}Pn + pn-lPn-l = 0
NPN-pN-iPN-i=0 To obtain the solution, observe that
That is nPn — /9 n _iP n _i = K, where K is a constant. From the first equation one sees that K = 0. One obtains
n
and then from
D.9
= Pi — ^
one
obtains
Chapter 9
9.1 Let x* be the common limit. One has
i xn+i - x* = xn+i - (xnyn)2 = (x% - yn and i
Xn
i
/ 2 X * = Xn2 \Xn
i
\ yn2 )i
From which it follows that xn+\ — x* — x~l(xn — x*) 2 . 9.3 Just take yn = A/xn.
Copyright © 2002 Marcel Dekker, Inc.
282
APPENDIX D. SOLUTIONS TO THE PROBLEMS
9.4 One easily derives that
yn+i - yn
f(yn]
and such a ratio has to be equal to —1 since both points are on the line x + y — a = 0. 9.6 Let yo be prime, the yo and yo + lare relatively prime. So is yo(yo + 1)4-1 and so on. Since yo — 2 and yo + 1 — 3 are prime, then Corollary 9.4.1 applies and the same will be true for the successive values.
Copyright © 2002 Marcel Dekker, Inc.
Bibliography [1] C. A. ARNEODO, P. FERRERO AND TRESSER, Sharkovskii order for appearance of superstable cycles: An elementary proof, Comm. Pure and Appl. Math., XXXVII (1984), pp. 13-17. [2] E. A. WEISSBERGER, Technique of Organic Chemistry, Distillation, vol. Vol. IV, Interscience, New York, 1951. [3] O. ABERTH, Iteration methods for finding all zeros of a polynomial simultaneously, Math. Comp., 27 (1973), pp. 339-344. [4] L. ACETO AND D. TRIGIANTE, The matrices of Pascal and other greats, Amer. Math. Monthly, 108 (2001), pp. 232-245. [5] R. AGARWAL AND J. POPENDA, On periodic solutions of first order linear difference equations, Math. Comput. Modeling, 22 (1995), pp. 11-19. [6] R. P. AGARWAL, On multipoint boundary value problems for discrete equations, Jour, of Math. Analysis and Appl., 96 (1983), pp. 520-534. [7]
, Initial value methods for discrete boundary value problems, Jour. of Math. Analysis and Appl., 100 (1984), pp. 513-529.
[8]
, Difference equations and inequalities: theory, methods, and applications., Marcel Dekker, New York, 2 ed., 2000.
[9] R. P. AGARWAL AND E. THANDAPANI, On some new discrete inequalities, Appl. Math, and Computation, 7 (1980), pp. 205,224. [10] ——, Some inequalities of Gronwall type, Arialele Stjietifice Univ. lasi., XXVII (1981), pp. 139-144. [11] C. AHLBRANDT AND W. T. PATULA, Recessive solutions of block tridiagonal nonhomogeneous systems, J. of Diff. Eq., 1 (1995), pp. 1— 98. [12] C. AHLBRANDT AND A. C. PETERSON, Discrete Hamiltonian Systems, Kluwer. Boston, 1996. 283
Copyright © 2002 Marcel Dekker, Inc.
284
BIBLIOGRAPHY
[13] P. AMODIO, Optimized cyclic reduction for the solution of tridiagonal systems on parallel computers, Comp. Math. Appl., 26 (1993), pp. 4553. [14] P. AMODIO AND F. MAZZIA, Backward error analysis of cyclic reduction for the solution of tridiagonal systems, Math. Comput., 62 (1994), pp. 601-617. [15] F. ASHBY, T. MANTUEFFEL. AND P. SAYLOR, A taxonomy for conjugate gradient methods, SIAM J. Numer. Anal., 27 (1990), pp. 1452-
1568. [16] F. V. ATKINSON, Discrete and continuous boundary value problems, Academic Press, 1964. [17] N. S. BAKHVALOV, Numerical Methods, MIR, Moscow, 1977. [18] B. BARNA, Uber die diverenzpunbe des Newtonches verfahrens zur bestinemmung von wurzelu algebraischen, Publications Mathematical Debrecen. (1956), pp. 384-397. [19] S. BARNETT AND D. D. SILIJAK, Routh algorithm, a centennial survey, SIAM Review, 19 (1977), pp. 472-489. [20] A. BERMAND AND R. PLEMMONS, Nonnegative matrices in the Mathematical Sciences, Academic Press, New York, 1979. [21] L. BRAND, Differential York. 1966.
and Difference
Equations, John Wiley, New
[22] L. BRUGNANO, Numerical implementation of a new algorithm for polynomial with multiple roots, J. of Diff. Eq. and Appl., 1 (1995), pp. 187207. [23] L. BRUGNANO AND D. TRIGIANTE, Tndiagonal matrices: Invertibility and conditioning, Lin. Alg. Appl.. 166 (1992), pp. 131-150. [24]
, Toeplitz Matrices and Difference Equations in Numerical Analysis. In Proc. of First Int. Conf. on Difference Eq. Tampa, Gordon and Breach. 1994. pp. 79-94.
[25]
, Polynomial roots: The ultimate answer?, Lin. Alg. Appl., 225 (1995), pp. 207-219.
[26] L. BRUGNANO AND D. TRIGIANTE, Solving Differential Problems by Multistep Initial and Boundary Value Methods, vol. 6 of Stability and Control: Theory Methods and Applications. Gordon and Breach, Amsterdam. 1998.
Copyright © 2002 Marcel Dekker, Inc.
BIBLIOGRAPHY
285
[27] K. BuRRAGE AND J. BUTCHER, Nonlinear stability for a general class of differential equation methods, BIT, 20 (1980), pp. 185-203. [28] K. BURRAGE AND J. C. BUTCHER, Stability criteria a implicit Runge-Kutta methods, SIAM J. Numer. Anal., 15 (1979), pp. 46-57. [29] J. BUTCHER, A stability property for implicit Runge-Kutta methods, BIT, 15 (1975), pp. 358-361. [30] J. BUTCHER, The Numerical Analysis of Ordinary Differential tions, J. Wiley, Chichester, 1987.
Equa-
[31] B. BuZBEE, G. GOLUB, AND C. W. NiELSON, On direct methods for solving Poisson's equations, SIAM J. Numer. Anal., 7 (1970), pp. 627656. [32] B. CARLSON, Algorithms involving arithmetic and geometric means, Amer. Math. Monthly, (1971), pp. 496-505. [33] J. R. CASH, Stable Recursions, Academic Press, London, 1979. [34]
, A note on the solution of linear recurrence relations, Num. Math., 34 (1980), pp. 371-386.
[35] F. CASORATI, // calcolo delle differenze finite accresciuto di nuovi teoremi..., Mem. Acad. Lincei Ser. Ill, 5 (1880), pp. 195-208. [36]
-, II calcolo delle differenze finite accresciuto di nuovi teoremi..., Annali di Matematica, 10 (1889), pp. 10-45.
[37] F. CHAITIN-CHATELIN. V. TOUMAZOU, AND E. TRAVIESAS, Accuracy assessment for eigencomputations: Variety of backward errors and pseudospectra, Linear Algebra Appl., 309 (2000), pp. 73—83. [38] S. S. CHENG AND L. Y. HSIEH, Inverse of matrices arising from difference equations, Utilitas Mathematica. 38 (1990), pp. 65-77. [39] T. CHIHARA, An Introduction to Orthogonal Polynomials, Gordon and Breach, London, 1978. [40] C. W. CLENSHAW, A note on the summation of Chebyshev series, MTAC, 9 (1955), pp. 118-120. [41] L. COLLATZ, Functional Analysis and Numerical Mathematics, Academic Press, 1966. [42] K. COOKE AND L. LADEIRA, Applying Carvalho's method to find periodic solutions of difference equations, J. of Diff. Eq. and Appl., 2 (1996). pp. 105-115.
Copyright © 2002 Marcel Dekker, Inc.
286 [43] C. CORDUNEANU, Principles of Differential Chelsea, New York, 1977.
BIBLIOGRAPHY and Integral Equations,
[44] C. CORDUNEANU, Almost periodic discrete processes, Libertas Math, 2 (1982), pp. 159-169. [45] G. DAHLQUIST, A special stability problem for linear multistep methods, BIT, 3 (1963), pp. 27-43. [46] —— , Error analysis for a class a methods for stiff nonlinear initial value problems. Num. Anal., Dundee, Springer Lect. Notes in Math., 506 (1975). pp. 60-74. [47] - . On stability and error analysis for stiff nonlinear problems, Part 1. Report Trita-NA-7508, 1975. [48] - , G- stability is equivalent to A- stability, BIT, 18 (1978). pp. 384401. [49] - , On the local and lobal errors of one-leg methods, Report, TRITANA-8110, 1981. [50] - , Some comments on stability and error analysis far stiff nonlinear differential Systems, preprint, NAD A Stockholm, 1983. [51] G. DAHLQUIST AND A. BJORK, Numerical Methods. Prentice-Hall, 1974. [52] G. DAHLQUIST. L. W.. AND O. NEVANLINNA, Stability of two step methods for variable integration steps, SIAM J. Numer. Anal., 20 (1983), pp. 1071-1085. [53] F. DANNAN, S. ELAYDI. AND P. Liu, Periodic solutions of equations, J. of Diff. Eq. and Appl, 6 (2000), pp. 203-232.
difference
[54] P. DAVIS AND P. RABINOWITZ, Method of Numerical Integration., Computer Science and Applied Mathematics. Academic Press, New York. 1984. [55] P. V. DER CRUYSSEN, A reformulation of Olver's algorithm for the numerical solution of second order difference equations. Num. Math., 32 (1979), pp. 159-166. [56] P. DIAMOND, Finite stability domains for difference Austral. Soc.. 22A (1976). pp. 177-181.
equations. Jour.
[57] - , Discrete Liapunov function with 5~v > 0, Jour. Austral. Soc., 20B (1978), pp. 280-284.
Copyright © 2002 Marcel Dekker, Inc.
BIBLIOGRAPHY
287
[58] R. D. DRIVER, Note on a paper of Halanay on stability of finite difference equations. Arch. Rat. Mech., 18 (1965), pp. 241-243. [59] E. DURAND, Solution Numerique des Equations Algebriques, vol. I, Masson, Paris, 1968. [60] L. EHRLICH, A modified Newton method for polynomial, Comm. ACM, 10 (1967), pp. 107-109. [61] S. ELAYDI, Asymptotics for linear difference equations, J. of Difference Equations, 5 (1999), pp. 563-588. [62]
, An Introduction to Difference Equations, 2ed., Springer-Verlag, New York, 1999.
[63] H. ELMAN, A stability analysis of incomplete LU factorization, Math. Comp., 47 (1986), pp. 191-217. [64] G. FARIN, Curve and Surfaces for Aided Geometric Design, Academic Press, Boston, 1990. [65] T. FORT, Finite Differences and Diffrence Equations in the Real Domain, Oxford Univ. Press, Oxford, 1948. [66] V. L. G. S. LADDE AND A. S. VATS ALA, Monotone Iterative Techniques for Nonlinear Differential Equations, Pitman Publishers Co., 1985. [67] G. GANDOLFO, Mathematical Methods and Models in Economics Dynamics, North-Holland. Amsterdam. 1971. [68] L. GATTESCHI, New results on some two dimentional iterative algorithms, Acad. Naz. dei Lincei, 147 (1998), pp. 137-159. [69] W. GAUTSCHI, Computational aspects of three terms recurrence relations, SIAM Rev., 9 (1967), pp. 24-82. [70] W. GAUTSCHI, Numerical Analysis, Birkhauscr, Basel, 1997. [71] G. W. GEAR, Numerical Initial Value Problems in Ordinary Differential Equations, Prentice-Hall, Englewood Cliffs, 1971. [72] G. W. GEAR AND K. W. Tu, The effect of variable mesh size on the stability of multistep methods, SIAM J. Numer. Anal., 1 (1974), pp. 1025-1043. [73] G. GEIST, Reduction of a general matrix to tridiagonal form, SIAM J. Matrix Anal. Appl., 12 (1991), pp. 362-373.
Copyright © 2002 Marcel Dekker, Inc.
288
BIBLIOGRAPHY
[74] A. P. GELFOND, Calcul des Difference
Finies, Dunod, Paris, 1963.
[75]
Hindusten Publishing Corp.,
, Calculus of finite Differences, Delhi, 1971.
[76] S. K. GODUNOV AND V. S. RYABENKI, Theory of Difference North-Holland, 1964.
Schemes,
[77] S. GOLDBERG, Introduction to Difference Equations, John Wiley, New York, 1958. [78] G. GOLUB AND C. VAN LOAN, Matrix Iterative Analysis, John Hopkins University Press, Baltimore, 1983. [79] S. P. GORDON. Stability and summability of solutions of equations, Math. Syst. Theory, 5 (1971), pp. 56-75.
difference
[80] T. K. S. H. S. MICKLEY AND C. E. REED, Applied Mathcrnaitcs in Chemical Engineering, McGraw-Hill, N. Y., 1967. [81] W. HAHN, Stability of Motion, Springer, Berlin, 1967. [82] F. A. HAIGHT, Mathematidal Theories of Traffic Press, New York, 1963.
Flow, Academic
[83] E. HAIRER. S. NORSETT, AND G. WANNER, Solving Ordinary Differential Equations, vol. I, Springer-Verlag, Berlin, 1993. [84] A. H ALAN AY, Quelque questions de la, thorie de la stabilit pour les systmes aux differences finies, Arch. Rat. Mech., 12 (1963), pp. 150154. [85]
, Solution periodiques et presque-periodiques des systems d'equationes aux difference finies, Arch. Rat. Mech., 12 (1963), pp. 134-149.
[86] A. HALANAY AND D. WEXLER, Teoria Calitative a Sisternlor cu Impulsun, Bucharest, 1968. [87] P. HARTMAN AND A. WINTNER, On the spectre of Toeplitz matrices, Am. J. of Math., 72 (1950), pp. 359-366. [88] P. HENRICI. Discrete Variable Methods for Ordinary Equations, John Wiley, New York, 1962.
Differential
[89]
, Error Propagation for Difference York, 1963.
[90]
, Applied and Computational Complex Analysis, vol. Vol. 1, John Wilcv. New York, 1974.
Copyright © 2002 Marcel Dekker, Inc.
Methods, John Wiley, New
BIBLIOGRAPHY
289
[91] M. HESTENES AND E. STIEFEL, Methods of conjugate gradients for solving linear systems, J. of Ris. of the Nat. Bur. of Standards, 49 (1952), pp. 409-436. [92] F. B. HILDEBRAND, Methods of Applied Mathematics, Prentice-Hall, 1952. [93] F. C. HOPPENSTEADT, Mathematical Methods a Population Biology, Courant Inst. of Math. Science, 1976. [94] F. C. HOPPENSTEADT AND J. M. HYMAN, Periodic solutions of a logistic difference equation, SIAM JAM, 32 (1977), pp. 73-81. [95] J. HURT, Some stability theorems for ordinary difference SIAM J. Numer. Anal., 4 (1967), pp. 582-596.
equations,
[96] D. JAGERMANN, Difference Equations with Applications to Queues, vol. 233 of Pure and Applied Mathematics, Marcel Dekker, New York, 2000. [97] C. JORDAN, Calculus of Finite Difference,
Chelsea, New York, 1950.
[98] R. KANNAN AND M. B. RAY, Monotone iterative methods for nonlinear equations involving a non-invertible linear part, Num. Math., 45 (1984), pp. 219-225. [99] W. G. KELLEY AND A. C. PETERSON, Difference Equations, an Introduction with Applications, Academic Press, San Diego, 2001. [100] I. KERNER, Ein gesamtschrittverfahren zur berechnung der nullstellen von polynomen, Numer. Math., 8 (1966), pp. 290-294. [101] M. KHAVANIN AND V. LAKSHMIKANTHAM, The method of mixed monotony and first order differential systems, Nonlinear Analysis, 10 (1986), pp. 873-877. [102] V. Kocic AND G. LADAS, Global behavior of nonlinear difference equations of higher order with applications, Mathematics and its Applications, Kluwer Academic Publishers, Dordrecht, 1993. [103] W. KRATZ, Banded matrices and difference Appl. To appear.
equations, Linear Algebra
[104] V. LAKSHMIKANTHAM AND S. LEELA, Differential and Integral Inequalities, vol. I, II, Academic Press, New York, 1969. [105] V. LAKSHMIKANTHAM AND D. TRIGIANTE, Theory of difference equations, Numerical Methods and Applications, vol. 181 of Mathematics in Science and Engeneering, Academic Press, New York, 1988.
Copyright © 2002 Marcel Dekker, Inc.
290
BIBLIOGRAPHY
[106] V. LAKSHMIKANTHAM AND A. S. VATSALA, Method of mixed monotony for non linear equations with a singular linear part, Appl. Math, and Computations, 23 (1987), pp. 235-241. [107] J. A. LAMBERT, Computational Methods in Ordinary Equations. John Wiley, 1973.
Differential
[108] J. P. LASALLE, The stability of dynamical systems, Regional Conference Series in Applied Mathematics, SIAM, 1979. [109] G. D. LENA AND D. TRIGIANTE, On the stability and convergence of lines method. Rend, di Mat., 3 (1982), pp. 113-126. [110]
, Stability and spectral properties of incomplete Japan J. Appl. Math., 7 (1990), pp. 145-153.
factorization,
[Ill] J. W. LEWIS, Inversion of tridiagonal matrices, Num. Math., 38 (1982), pp. 333-345. [112] T. Y. Li AND J. YORKE, Period three implies chaos, Amer. Math. Monthly, 82 (1975), pp. 985-992. [113] E. N. LORENZ, The problem of deducing the climate from, the governing equations, TELLUS, 16 (1964), pp. 1-11. [114] D. G. LuENBERGER, Introduction to Dynamic Systems, John Wiley, New York. 1979. [115] Y. L. LUKE, The Special Functions and Their vol. Vol. 1. Academic Press, 1969.
Approximations,
[116] A. MATE AND P. NEVAI, Sublinear perturbations of the differential equation y' = 0 and the analogous difference equation, J. Diff. Eq., 53 (1984), p. 234 257. [117]
, A generalization of Poincare theorem for recurrence equation, J. Approx. Theory, 63 (1990), pp. 92-97.
[118] R. MATTHEIJ AND M. SMOOKE, Estimates of the inverses of tridiagonal matrices arising in boundary value problems, Lin. Alg. Appl., 73 (1986), pp. 33-57. [119] R. M. MATTHEIJ, Accurate estimates for the fundamental solutions of discrete boundary value problems, J. Math. Anal, and Appl., 101 (1984). pp. 444-464. [120]
, Stability of block LU-decompositions of the 'matrices arising from BVP, SIAM J. Alg. Dis. Math., 5 (1984), pp. 314-331.
Copyright © 2002 Marcel Dekker, Inc.
BIBLIOGRAPHY
291
[121] R. M. MAY, Biological populations with nonoverlapping generations. Stable points, stable cycles and chaos, Science, 186 (1974). [122]
, Simple mathematical models with very complicated dynamics, Nature, 261 (1976), pp. 459-467.
[123] F. MAZZIA AND D. TRIGIANTE, Numerical methods for second order singular perturbation problems, Comp. Math. Appl., 23 (1992), pp. 8189. [124] G. J. MEIL, Majorizing sequences and error bounds for iterative methods, Math, of Computations, 34 (1960), pp. 185-202. [125]
, An updated version of the Kantorovich theorem for Newton's method, Computing, 27 (1981), pp. 237-244.
[126] W. MELVIN, Stability properties of functional differential J. Math. Anal. Appl., 48 (1974), pp. 749-763.
equations,
[127] J. J. H. MILLER, On the location of zeros of certain classes of polynomial with applications to numerical analisis, J. Inst. Math. Appl., 8 (1971), pp. 397-406. [128] K. S. MILLER, An Introduction to the Calculus of Finite Differences and Difference Equations, Hold and Company, New York, 1960. [129]
, Linear Difference
Equations, Benjamin, New York, 1968.
[130] L. M. MILNE-THOMSON, The Calculus a Finite Differences, McMillan and Co., London, 1933. [131] O. NEVANLINNA AND W. LINIGER, Contratctive methods for stiff differential equations, BIT, 19 (1979), pp. 53—72. [132] O. NEVANLINNA AND F. ODEH, Multiplier techniques for linear multistep methods, Num. Funct. Anal, and Optimiz., 3 (4) (1981), pp. 377423. [133] R. O' SHEA. The extention of Zubov method to sampled data control systems described by difference equations, IEEE, Trans Auto. Conf., 9 (1964), pp. 62-69. [134] F. ODEH AND W. LINIGER, Non linear fixed h stability of linear multistep formulas, J. Math. Anal. Appl., 61 (1977), pp. 691-712. [135] O. OLIVEIRA FILHO AND L. CARVALHO, On Periodic solutions of x(t) = ax(t — 1) + bx(t — 2), In Proc. of First Int. Conf. on Difference Eq. To,mpa, Gordon and Breach, 1994, pp. 79-94.
Copyright © 2002 Marcel Dekker, Inc.
292
BIBLIOGRAPHY
[136] F. W. OLVER, Numerical solutions of second order linear difference equations, Jour, of Research, NBS, 71B (1967), pp. 111-129. [137] J. M. ORTEGA, The Newton-Kantorovich theorem,, Amer. Math. Monthly, 75 (1968), pp. 658-660. [138]
, Stability of difference equations and convergence of iterative processes, SIAM J. Numer. Anal, 10 (1973), pp. 268-282.
[139] J. M. ORTEGA AND W. C. RHEINBOLDT, Monotone iterations for nonlinear equations with application to Gauss-Seidel methods, SIAM J. Numer. Anal, 4 (1967), pp. 171-190. [140]
, Iterative Solution of Nonlinear Equations in Several Variables. Academic Press. New York, 1970.
[141] A. OSTROWSKI, Les points detraction el de repulsion pour ^iteration dans resace a n dimension, C. R. Acad. Sciences. Paris, 244 (1957), pp. 288-289. [142]
. Solution of Equations and Systems of Equations, Academic Press, New York. 1960.
[143] B. G. PACHPATTE, Finite difference inequalities and an extension a Lyapunov method, Michigan Math. J., 18 (1971). pp. 385-391. [144]
, On some discrete inequalities of Bellman-Bihari type, Indian J. Pure and Applied Math., 6 (1975), pp. 1479-1487.
[145]
, On some fundamental inequalities and its applications in the theory of difference equations, Ganita, 27 (1976), pp. 1-11.
[146] P. PANG AND R. AGARWAL, On the periodicity of difference equations of a general type, J. of Diff. Eq. and AppL 2 (1996), pp. 271-287. [147] B. PARLETT, Reduction to tridiagonal form and minimal realizaiions, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 567-593. [148] L. PASQUINI AND D. TRIGIANTE, A globally convergent method for simultaneously finding polynomial roots, Math, of Computation, 44 (1985), pp. 135-150. [149] W. PATULA, Growth, oscillations and comparison theorems for second order difference equations, SIAM J. Math. An., 10 (1979), pp. 12721279. [150] G. PIAZZA AND D. TRIGIANTE, Propagazione degli errori nella integrazione numerica di equazioni differenziali ordinane, vol. 120, Pubbl. IAC III, Roma, 1977.
Copyright © 2002 Marcel Dekker, Inc.
BIBLIOGRAPHY
293
[151] J. H. POLLARD, Mathematical Methods for the Growth of Human Populations, Cambridge University Press. 1973. [152] J. POPENDA, Finite difference pp. 79-87. [153]
inequalities, Fasciculi Math., 13 (1981),
, On the boundness of the solutions of difference ciculi Math., 14 (1985), pp. 101-108.
equations, Fas-
[154] F. A. POTRA, Sharp error bounds for a class of Newton-like methods, Libertas Matematica, 5 (1985). pp. 71-84. [155] R. B. POTTS, Nonlinear difference 6 (1982), pp. 659-665.
equations, Nonlinear Anal. TMA,
[156] S. REDDY AND L. TREFETHEN, Stability of the method of lines, Nurrier. Math., 62 (1992), pp. 235-267. [157] J. RiORDAN, Combinatorial Identities, John Wiley, New York, 1968. [158] T. L. SAATY, Elements of Queuing Theory, with Applications, Dover, New York, 1961. [159] A. SAMARSKI AND E. NILOAIEV, Methodes de Resolution des Equation des Mailles, MIR, Moscow, 1981. [160] C. W. SCHELIN, Counting zeros of real polynomials within the unit disk, SIAM J. Numer. Anal., 5 (1983), pp. 1023-1031. [161] I. J. SCHOENBERG, Monosplines and Quadrature Formulae, In T.N.E. Grcville, Theory and Applications of Spines Functions, Academic Press, New York, 1969. [162]
, Mathematical Time Exposures, Math. Ass. of America, 1982.
[163] A. N. SHARKOVSKII, Coexistence of cycles of continuous map of line into itself, Ukrainian Math. J., 16 (1964), pp. 61-71. [164] S. SMALE, The fundamental theorem of algebra and complexity theory, Bull. Amer. Math. Soc., 4 (1981), pp. 1-36. [165] P. STEPHAN, A theorem of Sharkovskii on the existence of periodic orbits of continuous endomorphisms of the real line, Comm. Math. Phys., 54 (1977), pp. 237-248. [166] G. W. STEWART. Afternotes goes to graduate school, SIAM, Philadelphia, 1998. [167] P. D. STRAFFIN, Periodic points of continuous functions, Math. Mag., 51 (1978), pp. 99-105.
Copyright © 2002 Marcel Dekker, Inc.
294
BIBLIOGRAPHY
[168] S. SUGIYAMA, Difference inequalities and their applications to stability problems, Lectures Notes in Math., Springer, 243 (1971), pp. 1-15. [169] Y. M. SVIREZHEV AND D. O. LOGOFET, Stability of Biological Community, MIR, Moscow, 1983. [170] J. SYLVESTER, On certain inequalities relating to prime numbers. Nature, XXXVIII (1888), pp. 259-262. [171] O. TOEPLITZ, Zur theorie der quadratischen und bilinearen formen von unendlichvielen veranderlichen. i. te.il: Theorie der l-formen., Mathematische Annalen, 70 (1911), pp. 351-376. [172] W. TRENCH, Asimptotic behavior of solutions of Poincare recurrence systems, Comp. Math. Appl, 28 (1994), pp. 317-314. [173] F. TRICOMI, Sugh algoritmi iterativi dell'analisi numerica, Accad. Naz. dei Lincei, (1975), pp. 105-117. [174] D. TRIGIANTE. On a system, of difference equations arising in the cyclic reduction., J. of Diff. Eq. and Appl., 3 (1998), pp. 369-384. [175] D. TRIGIANTE AND S. SIVASUNDARAM, A new algorithm for unstable three term recurrence relations, Appl. Math, and Comp, 22 (1987), pp. 277-289. [176] M. URABE, Nonlinear Autonomous Oscillations. Academic Press, New York, 1976. [177] R. A. USMANI, Applied Linear Algebra, Marcel Dekker, New York, 1988. [178]
, On the explicit inverse and conditioning of a tridiagonal matrix, Int. J. Comp. Math., 4 (1992), pp. 201-213.
[179] G. WARMER AND H. REITBERGER, On the perturbation formulas of Grobner and Alekseev. Bui. lust. Pol. lasi, XIX (1973), pp. 15-25. [180] J. WlMP, Computation with Recurrence Relations, Pitman, 1984. [181] M. YAMAGUTI AND H. MATANO, Euler's finite difference chaos, Proc. Japan Acad, 55A (1979), pp. 78-80.
scheme and
[182] M. YAMAGUTI AND S. USHIKI, Discretization and chaos, C.R. Acad. Sc. Paris, 290 (1980), pp. 637-640. [183]
, Chaos in numerical analysis of ordinary differential Physica, 3D (1981), pp. 618-626.
Copyright © 2002 Marcel Dekker, Inc.
equations.
BIBLIOGRAPHY
295
[184] T. YAMAMOTO, Error bound for Newton's iterated, derived, from the Kantorovich theorem, Num. Math., 48 (1986), pp. 91-98. [185] R. V. M. ZAHAR, Mathematical analysis of Miller's algorithm, Num. Math., 27 (1977), pp. 427-447.
Copyright © 2002 Marcel Dekker, Inc.
