Spectral Computations for Bounded Operators

Spectral Computations for Bounded Operators

APPLIED MATHEMATICS Editor: R.J. Knops This series presents texts and monographs at graduate and research levels covering a wide variety of topics of current research interest in modern and traditional applied mathematics, numerical analysis, and computation. 1 Introduction to the Thermodynamics of Solids J.L. Ericksen (1991) 2 Order Stars A. Iserles and S.P. Nørsett (1991) 3 Material Inhomogeneities in Elasticity G. Maugin (1993) 4 Bivectors and Waves in Mechanics and Optics Ph. Boulanger and M. Hayes (1993) 5 Mathematical Modelling of Inelastic Deformation J.F. Besseling and E van der Geissen (1993) 6 Vortex Structures in a Stratified Fluid: Order from Chaos Sergey I. Voropayev and Yakov D. Afanasyev (1994) 7 Numerical Hamiltonian Problems J.M. Sanz-Serna and M.P. Calvo (1994) 8 Variational Theories for Liquid Crystals E.G. Virga (1994) 9 Asymptotic Treatment of Differential Equations A. Georgescu (1995) 10 Plasma Physics Theory A. Sitenko and V. Malnev (1995) 11 Wavelets and Multiscale Signal Processing A. Cohen and R.D. Ryan (1995) 12 Numerical Solution of Convection-Diffusion Problems K.W. Morton (1996) 13 Weak and Measure-valued Solutions to Evolutionary PDEs J. Málek, J. Necas, M. Rokyta and M. Ruzicka (1996) 14 Nonlinear Ill-Posed Problems A.N. Tikhonov, A.S. Leonov and A.G. Yagola (1998) 15 Mathematical Models in Boundary Layer Theory O.A. Oleinik and V.M. Samokhin (1999) 16 Robust Computational Techniques for Boundary Layers P.A. Farrell, A.F. Hegarty, J.J.H. Miller, E. O’Riordan and G. I. Shishkin (2000) 17 Continuous Stochastic Calculus with Applications to Finance M. Meyer (2001) 18 Spectral Computations for Bounded Operators Mario Ahues, Alain Largillier and Balmohan V. Limaye (2001) (Full details concerning this series, and more information on titles in preparation are available from the publisher.)

Spectral Computations for Bounded Operators

MARIO AHUES ALAIN LARGILLIER BALMOHAN V. LIMAYE

CHAPMAN & HALL/CRC Boca Raton London New York Washington, D.C.

disclaimer Page 1 Tuesday, January 23, 2001 1:21 PM

Library of Congress Cataloging-in-Publication Data Ahués, Mario. Spectral computations for bounded operators / Mario Ahues, Alain Largillier, Balmohan V. Limaye. p. cm.— (Applied mathematics ; 18) Includes bibliographical references and index. ISBN 1-58488-196-8 (alk. paper) 1. Spectral theory (Mathematics) 2. Operator theory. I. Largillier, Alain. II. Limaye, Balmohan Vishnu. III. Title. IV. Series. QA320 .A37 2001 515′.7222—dc21

00-069348

This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval system, without prior permission in writing from the publisher. The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained in writing from CRC Press LLC for such copying. Direct all inquiries to CRC Press LLC, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation, without intent to infringe.

Visit the CRC Press Web site at www.crcpress.com © 2001 by Chapman & Hall/CRC No claim to original U.S. Government works International Standard Book Number 1-58488-196-8 Library of Congress Card Number 00-069348 Printed in the United States of America 1 2 3 4 5 6 7 8 9 0 Printed on acid-free paper

Contents

Preface Notation 1 Spectral Decomposition 1.1 1.2 1.3 1.4

General Notions . . . . . . Decompositions . . . . . . . Spectral Sets of Finite Type Adjoint and Product Spaces 1.4.1 Adjoint Space . . . . 1.4.2 Product Space . . . 1.5 Exercises . . . . . . . . . .

2 Spectral Approximation 2.1 2.2 2.3 2.4 2.5

Convergence of Operators Property U . . . . . . . . Property L . . . . . . . . Error Estimates . . . . . . Exercises . . . . . . . . .

3 Improvement of Accuracy

. . . . .

ix xiii 1

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. 69 . 76 . 81 . 90 . 106

3.1 Iterative Re nement . . . . . . . . . . . . . . . . . . . . 3.1.1 General Remarks . . . . . . . . . . . . . . . . . . 3.1.2 Re nement Schemes for a Simple Eigenvalue . . 3.1.3 Re nement Schemes for a Cluster of Eigenvalues 3.2 Acceleration . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Motivation . . . . . . . . . . . . . . . . . . . . . 3.2.2 Higher Order Spectral Approximation . . . . . .

1 21 33 41 41 48 64

69

113

. 114 . 114 . 119 . 134 . 148 . 148 . 156 v

vi 3.2.3 Simple Eigenvalue and Cluster of Eigenvalues . . . 162 3.2.4 Dependence on the Order of the Spectral Analysis 176 3.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

4 Finite Rank Approximations

4.1 Approximations Based on Projections . . . . . . . . . . 4.1.1 Truncation of a Schauder Expansion . . . . . . . 4.1.2 Interpolatory Projections . . . . . . . . . . . . . 4.1.3 Orthogonal Projections on Subspaces of Piecewise Constant Functions . . . . . . . . . . . 4.1.4 Finite Element Approximation . . . . . . . . . . 4.2 Approximations of Integral Operators . . . . . . . . . . 4.2.1 Degenerate Kernel Approximation . . . . . . . . 4.2.2 Approximations Based on Numerical Integration 4.2.3 Weakly Singular Integral Operators . . . . . . . 4.3 A Posteriori Error Estimates . . . . . . . . . . . . . . . 4.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . .

5 Matrix Formulations

5.1 Finite Rank Operators . . . . . . . . . . . 5.1.1 Singularity Subtraction . . . . . . 5.1.2 Uniformly Well-Conditioned Bases 5.2 Iterative Re nement . . . . . . . . . . . . 5.3 Acceleration . . . . . . . . . . . . . . . . . 5.4 Numerical Examples . . . . . . . . . . . . 5.5 Exercises . . . . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. 195 . 197 . 199 . 200 . 203 . 222 . 230 . 240

247

. . . . . . .

. . . . . . .

. 248 . 257 . 264 . 275 . 282 . 300 . 309

6.1 QR factorization . . . . . . . . . . . . . . . . . . . . . 6.1.1 Householder symmetries . . . . . . . . . . . . . 6.1.2 Hessenberg Matrices . . . . . . . . . . . . . . . 6.2 Convergence of a Sequence of Subspaces . . . . . . . . 6.2.1 Basic De nitions . . . . . . . . . . . . . . . . . 6.2.2 Krylov Sequences . . . . . . . . . . . . . . . . . 6.3 QR Methods and Inverse Iteration . . . . . . . . . . . 6.3.1 The Francis-Kublanovskaya QR Method . . . . 6.3.2 Simultaneous Inverse Iteration Method . . . . . 6.4 Error Analysis . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Condition Numbers or Forward Error Analysis 6.4.2 Stability or Backward Error Analysis . . . . . .

. . . . . . . . . . . .

. 316 . 319 . 324 . 327 . 327 . 331 . 336 . 336 . 342 . 344 . 344 . 347

6 Matrix Computations

. . . . . . .

185

. 185 . 187 . 190

315

vii 6.4.3 Relative Error in Spectral Computations and Stopping Criteria for the Basic QR Method . . . . 362 6.4.4 Relative Error in Solving a Sylvester Equation . . 369

References Index

373 379

Preface

This work addresses eigenvalue problems for operators on in nite dimensional spaces. Exact eigenvalues, eigenvectors, and generalized eigenvectors of operators with in nite dimensional ranges can rarely be found. It is thus imperative to approximate such operators by nite rank operators and solve the original eigenvalue problem approximately. In doing so, not just the eigenvalues but the spectral values of an in nite dimensional operator need to be considered. Starting with a suciently general theoretical framework, such as the one provided by a Banach space or by a Hilbert space, concrete approximation methods are given. They lead to nite dimensional problems which can be implemented on a computer. The rst chapter contains the classical spectral theory in the setting of a Banach space with an emphasis on spectral sets of nite type, that is, sets for which spectral projections are of nite rank. The product space structure is used to develop the notions of a Gram matrix and of a block reduced resolvent operator. The Jordan Canonical Form of a matrix is avoided for computational reasons. Also, the use of trans nite induction, in the form of the Hahn-Banach Theorem, is minimized. The second chapter shows how the approximation of a bounded operator T by a sequence (Tn ) of bounded operators is useful to approximate the spectral values of T by the spectral values of Tn . The classical results concerning norm convergence and collectively compact convergence are uni ed with the help of a new type of convergence. It is general enough to encompass a wide variety of approximation methods and, at the same time, simple enough to be veri ed in practice. Under this  -convergence, properties similar to the upper semicontinuity and the lower semicontinuity of the spectrum are shown to be available. This chapter contains ix

x error estimates for clusters of approximate eigenvalues and for approximate bases for the associated spectral subspaces. The third chapter outlines two ways of improving the accuracy of spectral approximations: iterative re nement and acceleration. Both yield sharper error estimates for the approximations. The special case of a simple eigenvalue is considered rst, before a full treatment is given for the general case of a cluster of a nite number of multiple eigenvalues. The fourth chapter gives a number of concrete methods for constructing nite rank approximations of an operator. These satisfy the conditions required for spectral approximation and also for its iterative re nement and acceleration. Methods based on projections are presented rst. They include the discretizations named after Galerkin and Sloan as well as the Finite Element Method. Special methods for integral operators involving degenerate kernels and numerical integration are considered next. They include the discretizations named after Nystrom and Fredholm. A singularity subtraction technique for weakly singular integral operators is also presented. A posteriori error estimates are discussed along the lines of a result of Brakhage. The fth chapter deals with the canonical matrix formulation of the eigenvalue problem for a nite rank operator and brings the reader to calculations that can be performed on a computer. Matrix formulations of the iterative re nement technique and of the acceleration procedure are also given. Special features of this chapter include discussions of uniformly well-conditioned bases of approximate spectral subspaces, the matrix formulation of the singularity subtraction technique, and the acceleration procedure. The last section of this chapter contains numerical examples involving several in nite dimensional model problems. The theory developed in the preceding chapters is illustrated by considering speci c nite rank approximations, then formulating and solving matrix eigenvalue problems and also employing iterative re nement as well as acceleration. For this purpose, it is necessary to use intensive large-scale computation either on a personal computer or on a mainframe computer. The numerical experiments in this section make use of MATLAB. The sixth chapter deals with eigenvalue problems for matrices. Instead of referring the reader to `black boxes' like the standard routines given in numerical libraries such as LAPACK, an in-depth analysis of the QR Methods is presented. Also, the forward error analysis (involving condition numbers) as well as the backward error analysis (related to stability considerations) are discussed in detail. A stopping criterion for

xi the QR Method which bounds the total relative error is proposed. Error analysis of the solution of a Sylvester equation concludes this chapter. Starting with operators from in nite dimensional spaces to in nite dimensional spaces in the rst three chapters, nite dimensional situations are progressively introduced. The fourth chapter deals with operators having in nite dimensional domains but nite dimensional ranges, while the last two chapters essentially deal with nite dimensional operators. De nitions and major results are illustrated by elementary examples. A large number of exercises are given at the end of each chapter except for the sixth. Hints are provided whenever appropriate. A list of symbols used in the book follows this preface. The rst part contains symbols that are used freely but not de ned in the text. The second part contains symbols that are introduced in the text. They are listed in the order of their appearance. Some of these symbols (like hx ; f i) are explained in the list itself, while only the names of the others (like x ; f ) are given in the list ; the reader may consult the Index at the end of the book to nd their meanings. The Index gives only the number of the page on which a term appears for the rst time. We remark that matrices are denoted by letters in the sanserif font throughout the book. Acknowledgements are due to the Service des Relations Internationales of the Universite Jean Monnet de Saint-E tienne, the Curriculum Development Programme of the Indian Institute of Technology Bombay, the Research in Pairs Programme of the Mathematisches Forschungsinstitut Oberwolfach, and the Association E quipe d'Analyse Numerique de Saint-E tienne. We thank Ra kul Alam, N. Gnaneshwar, Nick Higham, Rekha Kulkarni, Francoise Tisseur and Olivier Titaud for reading parts of the manuscript and making useful suggestions, C.L. Anthony for word processing parts of the manuscript, and Nirmala Limaye for support and encouragement. M. Ahues, A. Largillier, B.V. Limaye

Notation

Cj

the eld of complex numbers, the real part of z 2 Cj , the imaginary part of z 2 Cj , z the conjugate of z 2 Cj , i the square root of ;1 with imaginary part equal to 1, IR the eld of real numbers, [a; b] the set of t 2 IR such that a  t  b, ]a; b[ the set of t 2 IR such that a < t < b, ZZ the set of all integers, [r] the integral part of r 2 IR, [ m; n ] the set of all k 2 ZZ such that m  k  n, i;j the Kronecker symbol, equals 1 if i = j and equals 0 if i 6= j , Cj nm the space of all complex matrices with n rows and m columns, also called the space of nm complex matrices, u = [u(1); : : : ; u(n)] an element of Cj 1n , [u(1); : : : ; u(n)]> an element of Cj n1 , [ai;j ] a matrix having ai;j in the ith row and the j th column, A(i; j ) the entry in the ith row and the j th column of a matrix A, A(
; j ) the j th column of a matrix A, A(i;
) the ith row of a matrix A, A;1 the inverse of a nonsingular matrix A, I the identity matrix, In the identity matrix of order n, O the zero matrix in Cj nm ,

<(z ) =(z )

xiii

xiv tr(A) the trace of a square matrix A, det(A) the determinant of a square matrix A, diag [1 ; : : : ; n ] the nn diagonal matrix having j as the j th diagonal entry, dist(E; F ) the distance between subsets E and F in a metric space, dist(x; E ) dist(fxg; E ), spanfx1 ; : : : ; xn g the subspace spanned by elements x1 ; : : : ; xn , dim X the dimension of a linear space X , C 0 (E ) the space of all complex-valued bounded continuous functions de ned on a metric space E with the norm kxk1 := supfjx(t)j : t 2 E g; C p (I ) the space of all complex-valued functions de ned on an interval I whose rst p derivatives are continuous on I , C 1 (I ) the space of all complex-valued functions de ned on an interval I which are in nitely dierentiable on I , x0 the rst derivative of a dierentiable function x, (spn ) a sequence whose nth term is sn , ` the space of all complex sequences x := (x(1); x(2); : : :) which satisfy 1 X 1=p kxkp := jx(j )jp < 1 , where 1  p < 1, `

2

(ZZ )

j =1

the space of all complex sequences x := (: : : ; x(;2); x(;1); x(0); x(1); x(2); : : :) 1  X 1=2 which satisfy kxk := jx(j )j2 < 1, 2

ek Lp (I ) L1 (I ) I

j =;1

the kth standard vector in ` p or in ` p (ZZ ): ek (j ) = 0 for j 6= k and ek (k) = 1, the space of all equivalence classes of Lebesgue measurable functions x onan interval I which satisfy Z 1=p kxkp := jx(t)jp dt < 1 , where 1  p < 1, I the space of all equivalence classes of essentially bounded Lebesgue measurable functions on an interval I with the norm kxk1 := inf f > 0 : jxj  on I except for a subset of Lebesgue measure zerog, the identity operator,

xv

O

the zero operator, the null space (or the kernel) of a linear operator T , the range (or the image space) of a linear operator T , rank(T ) the rank of a linear operator T , that is, dim R(T ), TjY the restriction of an operator T from X to X to a subspace Y of X , TjY;Z the restriction of an operator T from X to X to a subspace Y of X such that R(T ) is contained in a subspace Z of X:

N (T ) R(T )

Notation introduced in the text (in order of appearance): hx ; yi y x;  A the conjugate-transpose of A 2 Cj mn ; j a Banach space over C; the set of all bounded linear operators on X; kT k the subordinated operator norm of T 2 BL(X ); re(T ) the resolvent set of T 2 BL(X ); R(T; z ) the resolvent operator of T 2 BL(X ) at z 2 re(T ); sp(T ) the spectrum of T 2 BL(X ); (T ) the spectral radius of T 2 BL(X ); int(;) the interior of a Jordan curve ;; ext(;) the exterior of a Jordan curve ;; X = Y  Z the decomposition of X into its closed subspaces Y and Z; int(C) the interior of a Cauchy contour C; ext(C) the exterior of a Cauchy contour C;  a spectral set; C (T; ) the set of all Cauchy contours which separate  from sp(T ) n ; P (T; ) the spectral projection associated with T 2 BL(X ) and a spectral set  for T; M (T; ) the spectral subspace associated with T 2 BL(X ) and a spectral set  for T; S (T; ) the reduced resolvent operator of T 2 BL(X ) at an isolated point  of sp(T ), X the adjoint space of X , hx ; f i f (x), where x 2 X and f 2 X , BL(X; Y ) the set of all bounded linear maps from a Banach space X to a Banach space Y ,

X

BL(X )

xvi

K E? X x kxk

the adjoint operator of K 2 BL(X; Y ); the annihilator of E  X; X 1m = f[x1 ; : : : ; xm ] : xj 2 X; 1  j  mg; an element of X ;

kxk k x k1

kxj k2 ; j =1 max kx k; j =1;:::;m j

m P

1

F

T xZ x; f 1 2

F

i( f )( x ) R ( T ; Z)

S' Z' x T T T T

1=2

the natural extension of iT 2 BL(X ) to X ; m P zi;1 xi ; : : : ; zi;n xi ; where Z := [zi;j ] 2 Cj mn ; i=1 j =1 the Gram matrix associated with x 2 X 1m and f 2 (X  )1n , the Gram product of x 2 X 1m with f 2 (X )1n ; the 1-norm of Z 2 Cj mn ; the 2-norm of Z 2 Cj mn ; the 1-norm of Z 2 Cj mn ; the F-norm or the Frobenius norm of Z 2 Cj mn ; m P fj (xj ), where x := [x1 ; : : : ; xm ] 2 X 1m j =1 and f := [f1 ; : : : ; fm ] 2 (X  )1m ; the block resolvent operator of T 2 BL( X ) at Z 2 Cj mm ; the block reduced resolvent operator associated with T 2 BL(X ), a spectral set  of nite type for T and an ordered basis ' for M (T; ); x T ' ; ' , where ' is an ordered basis for M (T; ) and ' is the adjoint basis for M (T ; ); the pointwise convergence of Tn to T; the norm convergence of Tn to T; the collectively compact convergence of Tn to T; the  -convergence of Tn to T , the length of a Cauchy contour C; minfjz j : z 2 Cg, where C is a Cauchy contour, an eigenvalue of Tn ; an eigenvector of Tn, hP m

kZk kZk kZk1 kZk

p Tn ! n Tn ! Tn cc!  Tn ! `(C) (C) n 'n

kxj k;

j=1 m P

xvii

nk 'nk Xq; X T q; T ( )

( )

[ ]

[ ]

Tnq ; Tn [ ]

nq 'n '[nq] [nq] 'n [ ]

X

q;

[ ]

x T Tn TnP TnS TnG TnD

X

kth eigenvalue iterate; kth eigenvector iterate; X q1 = f[x1 ; : : : ; xq ]> : xi 2 X; i = 1; : : : qg; the acceleratexd version of T 2 BL(X ) de ned on X q ; the accelerated version of Tn 2 BL(X ) de ned on X q ; an eigenvalue of Tn ; an eigenvector of Tn ; the rst of the q components of ' n ; a spectral set of nite type for Tn ; an ordered basis for M (Tn ; nq ); X qm = f[x1 ; : : : ; xm ] : xj 2 X q ; 1  j  mg = f[ x 1 ; : : : ; x q ]> : x i 2 X 1m ; 1  i  qg; an element of X ; the accelerated version of T 2 BL( X ) de ned on X ; also, the natural extension of T 2 BL(X ) to X ; the accelerated version of Tn 2 BL( X ) de ned on X ; also, the natural extension of Tn 2 BL(X ) to X ; a projection approximation of T 2 BL(X ); a Sloan approximation of T 2 BL(X ); a Galerkin approximation of T 2 BL(X ); [ ]

[ ]

[ ]

[ ]

a degenerate kernel approximation of an integral operator T; TnN a Nystrom approximation of an integral operator T; TnF a Fredholm approximation of an integral operator T; !(x;
) the modulus of continuity of x 2 C 0 ([a; b]); TnK a Kantorowich-Krylov approximation of an integral operator T with a weakly singular kernel; 2 (A); cond(A) condition number of A 2 Cj nn ; c (f (x)) computed value of f (x),

c

oating point map in chopping arithmetic;

r

oating point map in rounding arithmetic; uc unit roundo in chopping arithmetic; ur unit roundo in rounding arithmetic; ku

u (k) 1 ; ku , where k 2 [0; 1=u[:

Chapter 1

Spectral Decomposition

This chapter gives a treatment of the spectral theory for bounded operators, the main result being the Spectral Decomposition Theorem. Spectral sets of nite type are discussed in detail. While the development of the adjoint space is along the classical lines, the product space structure is used to introduce the Gram product of a nite set of elements in the space with a nite set of elements in the adjoint space. The product space structure also allows an integral representation of the block reduced resolvent.

1.1 General Notions We are interested in nding invariant subspaces of a linear operator: Given a linear space X over the complex eld Cj and a linear operator T : X ! X , we want to nd subspaces M of X such that T (M )  M . The operator T can be a matrix transformation, a linear integral operator, or a linear dierential operator. Let us begin by looking for one-dimensional invariant subspaces. If M is spanned by a nonzero vector x, that is, M = spanfxg, then T (M )  M if and only if there exists a scalar  2 Cj such that Tx = x. The complex number  is called an eigenvalue of T . Let I denote the identity operator on X . Then  2 Cj is an eigenvalue of T if and only if T ; I is not injective. If y is another nonzero vector spanning M , then y = cx for some c 2 Cj . Hence Ty = T (cx) = cTx = c(x) = (cx) = y. Thus we see that  depends on T and M but not on the particular choice of a vector spanning M . Any nonzero vector ' 2 X satisfying T' = ' 1

2

1. SPECTRAL DECOMPOSITION

is called an eigenvector of T corresponding to, or associated with, the eigenvalue . If E () := N (T ; I ), then T (E ())  E (). Moreover, the restriction of T to E (), namely, TjE();E() : E () ! E (), is a multiple of the identity operator by the scalar , that is, for all x 2 E (), TjE();E() x = x. The dimension of E () is called the geometric multiplicity of the eigenvalue , and E () is called the eigenspace of T corresponding to . We remark that the invariance of a one-dimensional subspace M under T can be written in the form of an equation, namely, T' = ', where spanf'g = M . Let us try to do the same thing in the case of an m dimensional invariant subspace M . If ' := ['1 ; : : : ; 'm ] forms an ordered basis for M , then the invariance of M under T is equivalent to the existence of m2 complex numbers i;j , i; j = 1; : : : ; m, such that for each j = 1; : : : ; m,

T'j = 1;j '1 +


+ m;j 'm : These m equations can be written brie y if we agree to extend the notation of matrix multiplication as follows: For x1 ; : : : ; xm 2 X , let

T [x1 ; : : : ; xm ] := [Tx1; : : : ; Txm]; and

2

3

1;1


1;m 6 2;1


2;m 7 6 7

[x1 ; : : : ; xm ] 64 .. .

.. .

.. .

m;1


m;m

7 5

:=

m hX i=1

i;1 xi ; : : : ;

m X i=1

i

i;m xi :

Then the m equations de ning the invariance of M under T can be written as T ' = ' ; m  m where  := [i;j ] 2 Cj . The particular case  := [] 2 Cj 11 gives the eigenequation T' = ', traditionally written as T' = '.

Remark 1.1 The characteristic polynomial of a matrix: Let X be a nite dimensional linear space, n := dim X , B be an ordered basis for X and let A be a linear operator on X represented in the basis B by an nn matrix A. Each 2 X is represented by a column matrix whose entries are the coordinates of with respect to the ordered basis B. This column matrix will be called the coordinate matrix of with

1.1. GENERAL NOTIONS

3

respect to B. If x is the coordinate matrix of an eigenvector ' of A corresponding to an eigenvalue  of A with respect to B, then A' = ' is equivalent to Ax = x. Since ' is nonzero, x is nonzero. Hence det(A ;  I) = 0, where I denotes the nn identity matrix. Conversely, if det(A ;  I) = 0, then A ;  I is singular and there exists a nonzero column x such that Ax = x. We conclude that  2 Cj is an eigenvalue of A if and only if  satis es the characteristic equation det(A ;  I) = 0, that is,  is a root of the characteristic polynomial of A: p(z ) := det(A ; z I): Thus every linear operator A on Cj n has at least one eigenvalue, and it has at most n distinct eigenvalues. Since for each nonsingular matrix V 2 Cj nn we have det(V;1 AV ;  I) = det(V;1 (A ;  I)V) = (det V);1 det(A ;  I) det V = det(A ;  I); we may choose any matrix representation of A to compute the eigenvalues of A. We agree to say that the roots of the characteristic polynomial of A are the eigenvalues of A. Also, a nonzero vector x such that Ax = x is called an eigenvector of A and the space fx 2 Cj n1 : Ax = xg is the eigenspace of A corresponding to its eigenvalue . We conclude that similar matrices have the same eigenvalues. It also follows that the eigenvalues of an upper triangular matrix are its diagonal entries and that the same holds for a lower triangular matrix. Note that if B := V;1 AV and x is an eigenvector of A corresponding to an eigenvalue  of A, then V;1 x is an eigenvector of B corresponding to the eigenvalue  of B. As remarked above, no computation is needed to obtain the eigenvalues of an upper triangular matrix. The following theorem shows that any linear map from a nite dimensional linear space into itself may be represented by an upper triangular matrix with respect to a suitable basis. A Hermitian positive de nite sesquilinear form h
;
i on a linear space X is called an inner product on X . We say that elements x, y in X are orthogonal if hx ; yi = 0. A subset E of X is said to be orthonormal if any two distinct elements of E are orthogonal, and kxk := hx ; xi1=2 equals 1 for every x 2 E .

4

1. SPECTRAL DECOMPOSITION

Theorem 1.2 Schur's Theorem: Let X be an n dimensional complex linear space and h
;
i be an inner product on X . If A : X ! X is a linear map, then there exists an orthonormal basis for X with respect to which A is represented by an upper triangular matrix.

Proof

The proof is by induction on the dimension of X . If dim X = 1, then any linear map A : X ! X has a 11 (upper triangular) matrix representation with respect to any basis of X . Assume that the result is true for all (n ; 1)-dimensional spaces. Let dim X = n and A : X ! X be a linear map. Let  be any of the eigenvalues of A and u1 a corresponding eigenvector such that hu1 ; u1i = 1. Choose n ; 1 elements vj , j = 2; : : : ; n, in X such that [u1 ; v2 ; : : : ; vn ] forms an ordered basis for X satisfying hu1 ; vj i = 0 for j = 2; : : : ; n. Since A is linear, there exist n linear functionals cj : X ! Cj , j = 1; : : : ; n, such that

Ax = c1 (x) u1 + c2 (x) v2 +


+ cn (x) vn for x 2 X: Let Y := spanfv2 ; : : : ; vn g. For y 2 Y de ne

By := c2 (y) v2 +


+ cn (y) vn : Then B : Y ! Y is a linear map to which the induction hypothesis applies. Hence there exists an orthonormal basis [u2; : : : ; un ] for Y with respect to which B is represented by an upper triangular matrix R. It follows that [u1 ; u2 ; : : : ; un ] forms an orthonormal basis for X with respect to which A is represented by an upper triangular matrix. The rst diagonal entry of this matrix is  since Au1 = u1 , and its other diagonal entries are the diagonal entries of R.

Remark 1.3 Schur form, square root and polar factorization of a matrix:

It should be noted that the preceding theorem has only a theoretical interest, since the proof of the existence of an orthonormal basis giving an upper triangular matrix representation is not constructive. In fact, Chapter 6 will be devoted to the numerical approximation of such an upper triangular representation. Nevertheless, the proof of this theorem points out an important fact, namely, the eigenvalues of a linear map A can be made to appear in a prescribed order along the diagonal of its upper triangular matrix representation.

1.1. GENERAL NOTIONS

5

Let n and m be positive integers such that m  n. If A 2 Cj nm , then A 2 Cj mn , where A (i; j ) := A(j; i). The matrix A is called the adjoint matrix or the conjugate-transpose matrix of A. Let Im denote the mm identity matrix. We say that A is a unitary matrix if A A = Im , that is, the columns of A form an orthonormal set in Cj n1 with respect to the canonical inner product given by hx ; yi := y x for x; y 2 Cj n1 : Then Schur's Theorem can be stated as follows: Given any matrix A 2 Cj nn , there exists a unitary matrix Q 2 Cj nn such that Q AQ is an upper triangular matrix. The upper triangular matrix Q AQ is called a Schur form of A. Simple examples, like the following one, show that Schur form of a matrix is not uniquely determined: Let       1 1 A := ;11 13 ; Q := p 11 ;11 ; Qe := p 11 ;11 : 2 2 Then both Q and Qe are unitary matrices and both     Q AQ = 2 10 ;11 and Qe  AQe = 2 10 11 are Schur forms of A. Let B 2 Cj nn be a normal matrix, that is, B B = B B , and let T := QBQ be a Schur form of B, where Q is a unitary matrix. Then TT = TT and hence T is a diagonal matrix. If, in fact, B is a Hermitian matrix, that is, B = B, then T is also Hermitian, that is, T is a diagonal matrix with real entries. If, moreover, B is positive de nite, that is, all the eigenvalues of B are positive
numbers, then so is T . De ne ; T1=2 := diag T(1; 1)1=2 ; : : : ; T(n; n)1=2 and P := QT1=2 Q . Then P is a positive de nite Hermitian matrix, it commutes with B and P2 = B. We now show that such a matrix P is unique. Let Pe be another positive de nite Hermitian matrix such that Pe 2 = B. Let q be any column of Q and t pthe corresponding diagonal entry of T. Then Pe 2 q = tq, that p is, (Pe + t In )(Pe ; t In )qp= 0. Since the eigenvalues of Pe pare positive numbers, the matrix Pe + t In is nonsingular. Hence (Pe ; t In )q = 0. e = Pq for each column q of Q, so that PQ e = PQ, or P e = P. Thus Pq Thus if B is a positive de nite Hermitian matrix, then there is a unique positive de nite Hermitian matrix whose square is equal to B. It will be denoted by B1=2 and called the square root of B. We remark that B1=2 commutes with B.

6

1. SPECTRAL DECOMPOSITION

If A 2 Cj nn is a nonsingular matrix, then A A is Hermitian positive de nite, so that P := (A A)1=2 exists, is nonsingular, and Q := AP;1 is unitary. This gives A = QP, which is known as the Polar Factorization of A. This factorization can be viewed as the matrix version of the polar representation z = ei  r of z 2 Cj . Let us give some examples of eigenvalues of operators de ned on in nite dimensional spaces.

Example 1.4 In this example, every complex number is an eigenvalue: Let X := C 1 (IR), the linear space of complex-valued functions de ned on IR whose derivatives of all orders exist. We de ne T : X ! X by Tx := x0 for x 2 X . Then every complex number  is an eigenvalue of T , and if '(t) := exp(t) for t 2 IR, then ' is an eigenvector of T corresponding to the eigenvalue .

Example 1.5 In this example, no complex number is an eigenvalue: Let X be the linear space of all functions x 2 C 1 (IR) such that x(t) = 0 if jtj  1. Other than the null function, this linear space contains, for instance,  2 x(t) := exp [1=(t ; 1)] if jtj < 1; 0 otherwise: De ne T : X ! X by Tx := x0 for x 2 X . Were  2 Cj an eigenvalue of T and ' 2 X a corresponding eigenvector, then

'0 = ';

so that '(t) = c exp(t); t 2 IR;

for some complex constant c 6= 0. But the condition `'(t) = 0 if jtj  1' implies that c = 0. Hence T has no eigenvalues at all.

Example 1.6 In this example, no nonzero real number is an eigenvalue: Let X be the linear space of all bounded functions x 2 C 1 (IR) such that for each positive integer j , the j th derivative of x, denoted by x(j); is also bounded. Let T be de ned by Tx := x0 for x 2 X . Since '(t) := exp(t) de nes a bounded function if and only if <() = 0, we conclude that the set of eigenvalues of T is f 2 Cj : <() = 0g. It is evident that if M1 and M2 are invariant subspaces of T , then so is M1 + M2 := fx1 + x2 : x1 2 M1 ; x2 2 M2 g. In particular, if 1

7

1.1. GENERAL NOTIONS

and 2 are distinct eigenvalues of T , then M := E (1 ) + E (2 ) is an invariant subspace of T . However, the restriction TjM;M : M ! M is not a scalar multiple of the identity operator on M . In fact, TjM;M will have 1 and 2 as eigenvalues (and only those). Given an eigenvalue  of T , it is clear that E () is the largest subspace M of X such that TjM;M is  times the identity operator on M . Let us ask the following question: Is E () the largest subspace M of X such that M is invariant under T and TjM;M has  as the only eigenvalue? The answer is in the negative, as the following example shows. 



Example 1.7 In this example, N (T ; I )2 is an invariant subspace which is larger than the eigenspace N (T ; I ): Let X be a linear space of dimension 5. Let B := ['1 ; '2 ; 1 ; 2 ;  ] form an ordered basis for X , and let T : X ! X be the linear operator de ned by the following equations in which ,  and  are three dierent complex numbers:

T'1 = '1 ; T'2 = '1 + '2 ; T

1

=  1; T

2

=

2

and T = :

The eigenvalues of T are ,  and  . In fact, '1 is an eigenvector of T corresponding to the eigenvalue , '2 is not an eigenvector of T , 1 and 2 are two linearly independent eigenvectors of T corresponding to the eigenvalue , and  is an eigenvector of T corresponding to the eigenvalue  . The eigenspaces of T are

E () = spanf'1 g; E () = spanf 1; 2 g and E ( ) = spanf g: The subspace M () := spanf'1 ; '2 g is invariant under T . The restriction TjM ();M () is represented in the ordered basis ['1 ; '2 ] of M () by the matrix   A1 := 0 1 ; which has  as the only eigenvalue but A1 6=  I2 , where I2 denotes the 22 identity matrix. A change of basis will not alter this situation, that is to say, for any nonsingular 2  2 matrix V, V;1 A1 V is dierent from  I2 . We remark that '1 = (T ; I )'2 and since '1 is an eigenvector of T associated with , (T ; I )2 '2 = 0. Hence 



E () = N (T ; I ) ;= N (T ; I )2 = M ():

8

1. SPECTRAL DECOMPOSITION

On the other hand,





E () = N (T ; I ) = N (T ; I )2 : We remark that









T ['1 ; '2 ] = ['1 ; '2 ] 0 1 and T [ 1 ; 2 ] = [ 1 ; 2 ] 0 0 : We observe that  is such that the restriction of T to E () can be represented by  I2 , which  is not the  case for . On the other hand, E ( ) = N (T ; I ) = N (T ; I )2 is one-dimensional. Our goal is to compute several eigenvalues of T . Also, for a given eigenvalue  of T , we want to nd the largest invariant subspace M such that the restriction TjM;M has  as the only eigenvalue. More generally, we are interested in computing a set  of eigenvalues of T as well as the largest invariant subspace M such that the restriction TjM;M has  as the set of its eigenvalues. The reason for considering this more general situation is that, in most of the applications, the eigenvalues and the corresponding maximal invariant subspaces of the operator T are not computable in exact arithmetic. Hence techniques of Numerical Analysis are needed to approximate the desired eigenvalues and the corresponding maximal invariant subspaces. In this context, the operator T will be approximated by an operator Te, which is often of nite rank. The eigenvalues of Te may be computed by reducing the eigenproblem for Te to a matrix eigenvalue problem and then by using standard routines. Then a single multiple eigenvalue of T is approximated by a cluster of eigenvalues of Te, and the corresponding maximal invariant subspace of T is approximated by the direct sum of all the maximal invariant subspaces of Te corresponding to the individual eigenvalues in that cluster. The word `cluster' is used here in a very primitive sense. It only signi es a subset of the eigenvalues of Te which may be identi ed by the user by noting that its elements are suciently close to one another and the subset itself is suciently far away from the rest of the eigenvalues of Te. If we were interested only in the nite dimensional case, so that T admits a matrix representation, then a somewhat brief theory of spectral analysis would be sucient. Since our goal includes the approximation of eigenvalues and of maximal invariant subspaces of integral and differential operators whose domains are in nite dimensional linear spaces, we need more sophisticated notions.

9

1.1. GENERAL NOTIONS

Our abstract setting is a complete normed space X over the complex eld Cj , that is, a complex Banach space. It is true that some of the results that we shall state and/or prove do not need the completeness of the norm on X . However, all our applications will take place in complete normed spaces. A linear map from a nite dimensional linear space to a linear space of the same dimension is injective if and only if it is surjective. This equivalence does not hold in in nite dimensional linear spaces. This means that to consider the bijectivity of T ; I ,  2 Cj , the notion of an eigenvalue is not enough.

Example 1.8 An injective or a surjective operator may not be1bijective: P

Let X := ` 2 and (ek ) be the standard basis for X . For x := x(k)ek , k=1 de ne

T1 x :=

1 X k=1

x(k)ek+1 and T2 x :=

1 X

k=1

x(k + 1)ek :

It is clear that T1 is injective but not surjective, while T2 is surjective but not injective. New de nitions are thus needed, and things become more interesting. In the rest of the book, X will denote a complex Banach space and BL(X ) the Banach algebra of all bounded linear operators from X into itself. The operator that sends each x 2 X to 0 will be denoted by O and, as we have mentioned before, the identity operator will be denoted by I . Let k
k denote the norm on X . The so-called subordinated operator norm is de ned as follows:

kT k := supfkTxk : x 2 X; kxk  1 g for T 2 BL(X ): The following de nitions concern an operator T 2 BL(X ): The resolvent set of T is de ned by re(T ) := fz 2 Cj : T ; zI is bijectiveg: For z 2 re(T ),

R(T; z ) := (T ; zI );1 is called the resolvent operator of T at z . We shall write R(z ) for R(T; z ) when there is no ambiguity.

10

1. SPECTRAL DECOMPOSITION

The spectrum of T is the set sp(T ) := Cj n re(T ): An element of sp(T ) will be called a spectral value of T . If X is nite dimensional, then sp(T ) consists of the eigenvalues of T . The spectral radius of T is de ned by

(T ) := supfjj :  2 sp(T )g: We shall prove in Proposition 1.15(a) that if X 6= f0g, then sp(T ) 6= ; and (T ) is nite.

Remark 1.9 Let T 2 BL(X ). If z 2 Cj is given, then for every y 2 X , a solution x 2 X of the linear equation (T ; zI )x = y will be uniquely determined by y if and only if z 2 re(T ) and in that case, x is given by

x = R(T; z )y:

This explains the name given to the operator R(T; z ). If  2 sp(T ) and y 2 X , then the existence and the uniqueness of solutions of the equation (T ; I )x = y can be considered with the help of the `reduced resolvent operator' of T associated with . See Remark 1.25.

Proposition 1.10 Properties of the resolvent operator: Let T 2 BL(X ). (a) If z 2 re(T ), then R(z) 2 BL(X ). (b) For z 2 re(T ) and z0 2 Cj , z0 2 re(R(z )) if and only if z0 = 0 or z + z1 2 re(T ): 0

11

1.1. GENERAL NOTIONS

(c) For z 2 re(T ),

(R(z )) = dist(z;1sp(T )) :

(d) First Resolvent Identity: For z1; z2 2 re(T ), R(z1 ) ; R(z2 ) = (z1 ; z2 )R(z1 )R(z2 ): (e) For z 2 re(T ),

TR(z ) = I + zR(z ):

(f) For z1; z2 2 re(T ), the operators T , R(z1) and R(z2) commute with each other.

Proof (a) This is a consequence of the Open Mapping Theorem. (See Theorem 11.1 of [55] or Theorem 4.12-2 of [48].) (b) Let z 2 re(T ). Since R(z) is invertible, z0 = 0 belongs to re(R(z)). If z0 6= 0, then 

 



R(z ) ; z0 I = R(z ) [I ; z0 (T ; zI )] = ;z0R(z ) T ; z + z1 I ; 0

so that

z0 2 re(R(z )) if and only if z + z1 2 re(T ): 0

(c) By (b) above, for z 2 re(T ),

 2 sp(T ) if and only if  6= z and  ;1 z 2 sp(R(z )): Hence





(R(z )) = sup j ;1 z j :  2 sp(T ) = dist(z;1sp(T )) : (d) If z1 and z2 belong to re(T ), then R(z1) ; R(z2 ) = R(z1 )[I ; (T ; z1 I )R(z2 )] = R(z1 )[T ; z2I ; (T ; z1 I )]R(z2 ) = (z1 ; z2)R(z1 )R(z2 ):

12

1. SPECTRAL DECOMPOSITION

(e) For z 2 re(T ),

I = (T ; zI )R(z ) = TR(z ) ; zR(z ): (f) Let z1 and z2 belong to re(T ). Since T (T ; z1I ) = (T ; z1I )T , multiplication on the left and on the right by R(z1) gives R(z1 )T = TR(z1). The commutativity of R(z1) and R(z2 ) follows from (d). Functions of a complex variable with values in a complex Banach space will play an important role in what follows. Let Y be a complex Banach space and an open subset of Cj . We say that F : ! Y is analytic in if for every z0 2 , the limit F 0 (z ) := lim F (z ) ; F (z0 ) 0

z ! z0

z ; z0

exists. In that case, the function F 0 is called the rst derivative of F . It can be shown that derivatives of higher order, de ned recursively, also exist. The theory of contour integration can be developed in this more general context with only minor changes in the case of complex-valued functions of a complex variable. A curve ; in Cj is de ned to be a continuous function from a compact interval [a; b] in IR to Cj . It is said to be closed if ;(b) = ;(a). A closed curve ; is called a simple curve if for all t; et 2 [a; b], t < et and ;(t) = ;(et) imply that t = a and et = b. In this case we shall identify ; with its image ;([a; b]). A curve ; is said to be recti able if the supremum of the set n nX i=1

j;(tn;i ) ; ;(tn;i;1 )j : a = tn;0 <


< tn;n = b; n = 1; 2; : : :

o

is nite. For example, a piecewise continuously dierentiable curve is recti able. By a Jordan curve we mean a simple closed recti able curve. The Jordan Curve Theorem states that if ; is a simple closed curve, then Cj n ;([a; b]) has two connected components, one bounded and the other unbounded. The interior of a Jordan curve ; is the bounded component of Cj n ;([a; b]) and it is denoted by int(;). The exterior of a Jordan curve ; is the unbounded component of Cj n ;([a; b]) and it is denoted by ext(;). A Jordan curve ; is said to be positively oriented if int(;) lies to the left as the curve is traced out.

13

1.1. GENERAL NOTIONS

A domain in Cj is an open connected subset of Cj . A domain is said to be simply connected if the complement of in Cj [f1g is a connected set. Let ; : [a; b] ! Cj be a recti able curve in Cj . If F : ;([a; b]) ! Y is continuous, then we de ne Z

;

F (z ) dz :=

b

Z

a

F (;(t)) d ;(t);

where the integral on the right is the limit of Riemann-Stieltjes sums n P of the form F (;(sn;i )) [;(tn:i ) ; ;(tn;i;1 )], where a = tn;0 <


< i=1 tn;n = b and sn;i 2 [tn;i;1 ; tn;i ], as the mesh of the subdivision of [a; b] tends to zero.

Remark 1.11

The most fundamental result about analytic functions is as follows. Cauchy's Theorem: Let be a domain in Cj . If is simply connected and F : ! Y is analytic in , then Z

;

F (z ) dz = 0

for every Jordan curve ; in . It can be proved that for any positively oriented Jordan curve ; in

and every z0 2 int(;), Z F (k) (z0 ) = 2k!i (z ;F z(z))k+1 dz for each nonnegative integer k: 0 ; These are known as Cauchy's Integral Formul. Let ;, ;1 ; : : : ; ;r be positively oriented Jordan curves in such that ;1 ; : : : ; ;r lie in int(;) and for i 6= j , each ;i lies in ext(;j ). If

e := int(;) \ ext(;1 ) \


\ ext(;r ) 

e then and F : ! Y is continuous in and analytic in , Z

;

F (z ) dz =

r Z X

j =1 ;j

F (z ) dz:

This result is known as Cauchy's Theorem for Multiply Connected Domains.

14

1. SPECTRAL DECOMPOSITION

We shall now consider power series with coecients in a complex Banach space Y . As in the special case Y := Cj , the classical results about the convergence of power series remain valid in general. Consider z0 2 Cj and y0 ; y1 : : : in Y . Let 

r := lim sup kyk k! 1

k1=k

;1

:

If z 2 Cj satis es jz ; z0j < r, then there exists y(z ) 2 Y such that n

X

lim n! 1

k=0

yk (z ; z0 )k ; y(z )

= 0: 1 P

Also, if z 2 Cj satis es jz ; z0 j > r, then the series yk (z ; z0 )k is k=0 not convergent. Hence r is called the radius of convergence of the power 1 P series yk (z ; z0 )k . k=0 Similarly, let t := lim sup kyk k1=k . If z 2 Cj satis es jz j > t, then there k! 1 exists y(z ) 2 Y such that n

X

lim n!1

k=0

and if jz j < t, then the series

yk z ;k;1 ; y(z )

= 0; 1 P

k=0

yk z ;k;1 is not convergent.

Remark 1.12

Let be a domain in Cj and F : ! Y be analytic. Taylor's Theorem: Let z0 2 and r > 0 be such that the open ball with center z0 and radius r is contained in . Then

F (z ) = where

1 X

k=0

ak (z ; z0 )k when jz ; z0 j < r;

(k) ak := F k (!z0 ) 2 Y for k = 0; 1; : : : Laurent's Theorem: If z0 2 , 0 < r1 < r2 are such that fz 2 Cj : r1 < jz ; z0 j < r2 g  and ; is a positively oriented Jordan curve such that fz 2 Cj : jz ; z0 j = r1 g  int(;); fz 2 Cj : jz ; z0 j = r2 g  ext(;);

15

1.1. GENERAL NOTIONS

then

1 X

F (z ) = where

k=;1

ck (z ; z0 )k when r1 < jz ; z0 j < r2 ;

Z

F (ze) dze for k = 0; 1; 2; : : : k+1 ; (ze ; z0 ) Liouville's Theorem: If F : Cj ! Y is bounded and analytic, then F is ck := 21 i

a constant function.

The results stated in Remarks 1.11 and 1.12 can be proved in exactly the same manner as the corresponding results for the complex-valued functions of a complex variable are proved. Consider now the case Y := BL(X ). Let ; be a recti able curve in Cj and F : ;([a; b]) ! BL(X ) be continuous. Let x 2 X and Fx : ;([a; b]) ! X be de ned by

Fx (z ) := F (z )x; z 2 ;([a; b]): Then Fx is continuous, and by the de nition of the integral we obtain Z



;

Z

Z

;

F (z ) dz,

Z

F (z ) dz x = Fx (z ) dz = F (z )x dz: ;

;

In particular, if T : X ! X , then for every x 2 X , we have Z



;

Z

F (z ) dz Tx = F (z )Tx dz: ;

Further, if T 2 BL(X ), then by the continuity and the linearity of T , we have Z  Z T F (z ) dz = TF (z ) dz: ;

;

We now use the preceding concepts to study the resolvent set and the resolvent operator of T 2 BL(X ).

16

1. SPECTRAL DECOMPOSITION

Theorem 1.13 Let T 2 BL(X ). (a) The set re(T ) is open in Cj . (b) The function R(
) : re(T ) ! BL(X ) is analytic in re(T ). (c) First Neumann Expansion: For z0 2 re(T ) and z 2 Cj such that jz ; z0 j < dist(z0 ; sp(T )), we have z 2 re(T ) and R(z ) has the Taylor Expansion

R (z ) =

1 X k=0

R(z0 )k+1 (z ; z0)k :

(d) For z 2 Cj such that jzj > (T ), we have z 2 re(T ) and R(z) has the Laurent Expansion

R(z ) = ; If in fact jz j > kT k, then

1 X k=0

T k z ;k;1 :

kR(z )k  jz j ;1kT k :

Proof (a) Let z0 2 re(T ). The radius of convergence of a power series in BL(X ) with kth coecient R(z0 )k+1 is r(z0 ) := lim sup kR1(z )k+1 k1=k = lim sup kR1(z )k k1=k  kR(1z )k > 0: 0 0 0 k! 1

k! 1

1 P

Hence the series R(z0 )k+1 (z ; z0)k converges in BL(X ) for all z 2 Cj k=0 satisfying jz ; z0 j < r(z0 ). Setting

An (z ) :=

n X k=0

R(z0 )k+1 (z ; z0 )k ;

we have An (z )(T ; zI ) = (T ; zI )An (z ) = [(T ; z0 I ) ; (z ; z0 )I ]An (z ) = I ; R(z0 )n+1 (z ; z0 )n+1 ;

17

1.1. GENERAL NOTIONS

which tends to I as n tends to in nity, provided jz ;z0 j < r(z0 ), and then the sum of the series is the operator R(z ). Thus for each z0 2 re(T ), the set re(T ) contains the open ball with center z0 and radius r(z0 ). Hence re(T ) is an open set. (b) Let z0 2 re(T ). As in part (a) above, R(z ) = R(z0 ) + [R(z0)]2 (z ; z0 ) + [R(z0 )]3 (z ; z0 )2 + : : : if jz ; z0 j < r(z0 ). Hence if 0 < jz ; z0 j < 1=kR(z0)k, then



1

R(z ) ; R(z0 ) ; [R(z )]2

=

[R(z )]2 X k k [ R ( z 0 0 0 )] (z ; z0 ) z ; z0 k=1 3 k R ( z ) k  1 ; kR0(z )kjzj;z ;z0zj j : 0 0

Thus

R(z0 ) 2 lim R(zz) ; ; z0 = R(z0 ) : This shows that the function R(
) is analytic in re(T ) and its derivative is R(
)2 . (c) Let z0 2 re(T ). Since re(T ) is open, sp(T ) is closed and we have dist(z0 ; sp(T )) > 0. Now the open ball centered at z0 and with radius dist(z0 ; sp(T ))) is contained in re(T ) and the function R(
) is analytic on it. Hence by Taylor's Theorem, the expansion z ! z0

R(z ) =

1 X

k=0

R(z0 )k+1 (z ; z0 )k ;

obtained in (a) above is valid for all z such that jz ; z0 j < dist(z0 ; sp(T )). (d) Let t := lim sup kT kk1=k . Then t  kT k. For z 2 Cj such that k! 1

1

jz j > t, it follows that the power series P T k z ;k;1 converges in BL(X ). Setting

we have

k=0

Bn (z ) := ;

n X k=0

T k z ;k;1 ;

 n+1 ; Bn (z )(T ; zI ) = (T ; zI )Bn(z ) = I ; Tz

18

1. SPECTRAL DECOMPOSITION

which tends to I as n ! 1 provided jz j > t, and then the sum of the series is the operator R(z ). If jz j > (T ), then z 2= sp(T ). Thus fz 2 Cj : jz j > (T )g is contained in re(T ) and the function R(
) is analytic in this set. Hence the Laurent Expansion

R(z ) = ;

1 X k=0

T k z ;k;1

is valid for all z such that jz j > (T ). For jz j > kT k, we have

1  kT k k

X 1

k ; k ; 1 kR(z )k = ; T z  jz j  k=0 k=0 jz j



1 X

1

jz j ; kT k ;

as desired.

Remark 1.14

By Theorem 1.13(b), z 2 re(T ) 7! kR(z )k 2 IR is a continuous function which takes only positive values. Hence it attains a maximum value and a positive minimum value on each compact subset of Cj contained in re(T ). If X 6= f0g, then sp(T ) is nonempty and the spectral radius of T is related to the powers of T in a peculiar manner.

Proposition 1.15 Let X = 6 f0g and T 2 BL(X ). (a) Gelfand-Mazur Theorem: The subset sp(T ) of Cj is nonempty and compact. (b) Spectral radius formula:

(T ) = klim kT k k1=k = k=1inf;2;::: kT k k1=k : !1

Proof (a) By Theorem 1.13(d), sp(T ) is a closed subset of Cj and jj  kT k for all  2 sp(T ). Hence sp(T ) is compact. If it were empty, then R(
) would be analytic in the whole complex plane; and, by Theorem

19

1.1. GENERAL NOTIONS

1.13(d), it follows that kR(z )k ! 0 as jz j tends to in nity. Hence R(
) is a bounded entire function. By Liouville's theorem, it must be a constant function; and in fact this constant must be equal to 0, but this is impossible because X 6= f0g. (b) The proof of Theorem 1.13(d) shows that

(T )  lim sup kT k k1=k = t: k! 1

In fact, (T ) = t since the Laurent Expansion must converge for jz j > (T ) and it must diverge for jz j < t. The identity

T k ; z k I = (T ; zI )

kX ;1 j =0

T k;1;j z j ;

which is valid for each positive integer k, shows that z 2 re(T ) whenever z k belongs to re(T k ). Hence k 2 sp(T k ) whenever  2 sp(T ), so that jj  kT k k1=k . Thus

(T )  k=1inf;2;::: kT k k1=k  lim inf kT k k1=k  lim sup kT k k1=k = (T ); k! 1 k! 1

which completes the proof.

Example 1.16 In this example, some spectral values are not eigenvalues: 2 Let X := ` . Let (ek ) be the standard basis for X . Consider the left shift operator on X :

Tx :=

1 X k=1

x(k + 1)ek ; x :=

1 X k=1

x(k)ek 2 ` 2 :

Let us rst compute the eigenvalues of T . The equation Tx = x leads to x(k + 1) = x(k) for each positive integer k. Fixing x(1) := 1, we get

x=

1 X k=1

k;1 ek :

Now x 2 X if and only if jj < 1. Hence any complex number  such that jj < 1 is an eigenvalue of T . Since sp(T ) is closed, this implies that sp(T ) contains f 2 Cj : jj  1g. It can be easily seen that for

20

1. SPECTRAL DECOMPOSITION

any integer k  0, kT k k = 1, so that (T ) = klim kT k k1=k = 1 and !1 hence sp(T ) = f 2 Cj : jj  1g: However, no point  on the boundary of the spectrum is an eigenvalue, 1 P because then k;1 ek 2= ` 2 . k=1

Example 1.17 An example in which no spectral value is an eigenvalue: Any eigenvalue of T belongs to sp(T ), since T ; I is not one-to-one

when  is an eigenvalue of T . When X is nite dimensional, sp(T ) is the set of all eigenvalues of T . However, when X is in nite dimensional, sp(T ) may contain scalars which are not eigenvalues of T . Consider the space X := C 0 ([0; 1]). Let x0 2 X be given. We de ne an operator T on X by (Tx)(t) := x0 (t)x(t); t 2 [0; 1]: It is easily seen that kT k = kx0 k1 and hence T 2 BL(X ). First we prove that sp(T ) = x0 ([0; 1]). Let z be a complex number not in x0 ([0; 1]). Then for any y 2 X , the equation (T ; zI )x = y has a unique solution x 2 X de ned by x(t) = y(t)=(x0 (t) ; z ) for all t 2 [0; 1]. Hence z 2 re(T ). This shows that sp(T )  x0 ([0; 1]). Now let  2 x0 ([0; 1]). The operator T ; I is not surjective, because the function y 2 X de ned by y(t) := 1 for t 2 [0; 1] is not in R(T ; I ). In fact, if t0 2 [0; 1] is such that x0 (t0 ) = , and x were a function in X such that (T ; I )x = y, then 0 = (T ; I )x(t0 ) = 1. We now show that  2 x0 ([0; 1]) is an eigenvalue of T if and only if x0 takes the value  in some nonempty open interval. Let  2 sp(T ). Suppose that there exist a and b such that 0  a < b  1 and x0 (t) =  for all t 2 ]a; b[. Then any nonzero continuous function vanishing on [0; 1] n [a; b] is an eigenvector of T corresponding to ; for example, the function ' : [0; 1] ! Cj de ned by 

b]; '(t) := (0t ; a)(b ; t) ifif tt 22 [[0a;; 1] n [a; b]: On the other hand, let  2 sp(T ) be an eigenvalue of T and ' a corresponding eigenvector. Then ' 6= 0 but (x0 ; )' = 0. Let s 2 [0; 1] be such that '(s) 6= 0. Since ' is a continuous function, there exist a and b such that 0  a < b  1, s 2 [a; b] and '(t) 6= 0 for all t 2 [a; b]. This implies that x0 (t) =  for all t 2 ]a; b[.

21

1.2. DECOMPOSITIONS

In the particular case x0 (t) := t for t 2 [0; 1], we have sp(T ) = [0; 1] and no spectral value of T is an eigenvalue because T' = ' implies that '(t) = 0 for all t 2 [0; 1].

1.2 Decompositions

Let Y and Z be closed subspaces of X . We say that (Y; Z ) is a decomposition of X , or decomposes X if

X = Y  Z; that is, each x 2 X has a unique decomposition

x = xY + xZ with xY 2 Y and xZ 2 Z: The operator P : X ! X de ned by x 7! Px := xY is linear and satis es P 2 = P , that is, P is a projection. Since R(P ) = Y and N (P ) = Z , we say that P projects onto Y along Z . Because Y and Z are closed, the

Closed Graph Theorem (see Theorem 10.2 of [55] or Theorem 4.13-2 of [48]) implies that P is continuous. Thus P 2 BL(X ). Let (Y; Z ) decompose X . Consider T 2 BL(X ). We say that (Y; Z ) is a decomposition of T (or decomposes T ) if both Y and Z are invariant under T : T (Y )  Y and T (Z )  Z: If P projects onto Y along Z , then it is easy to see that (Y; Z ) decomposes T if and only if P commutes with T .

Proposition 1.18

If (Y; Z ) decomposes T , then

sp(T ) = sp(TjY;Y ) [ sp(TjZ;Z ):

Proof Let z 2 re(T ). First notice that if x 2 Y , then y := Tx ; zx 2 Y . Also, if y 2 Y , Tx ; zx = y for x 2 X and x = xY + xZ for xY 2 Y and xZ 2 Z , then (TxY ; zxY ) + (TxZ ; zxZ ) = y;

22

1. SPECTRAL DECOMPOSITION

so that TxZ ; zxZ = 0, that is, xZ = 0 and x = xY 2 Y . Thus T ; zI is a bijection from Y onto itself. The same argument applies for Z . This shows that re(T )  re(TjY;Y ) \ re(TjZ;Z ). Since Y and Z are invariant under R(T; z ), we have

R(T; z )jY;Y = R(TjY;Y ; z ) and R(T; z )jZ;Z = R(TjZ;Z ; z ): Conversely, let P be the projection onto Y along Z . For z 2 re(TjY;Y ) \ re(TjZ;Z ) and x 2 X , it can be seen that (T ; zI )[R(TjY;Y ; z )P + R(TjZ;Z ; z )(I ; P )]x = x and

[R(TjY;Y ; z )P + R(TjZ;Z ; z )(I ; P )](T ; zI )x = x; so that z 2 re(T ) and

R(T; z ) = R(TjY;Y ; z )P + R(TjZ;Z ; z )(I ; P ): Thus

re(T ) = re(TjY;Y ) \ re(TjZ;Z ):

Hence we obtain the desired result. In general, sp(TjY;Y ) and sp(TjZ;Z ) may not be disjoint as the following example shows.

Example 1.19 A simple example of decomposition:

Let X be a two-dimensional complex space and an operator T on X be represented by the matrix 

A := 0 0



with respect to an ordered basis [v1 ; v2 ]. Let Y := spanfv1 g and Z := spanfv2 g. The pair (Y; Z ) decomposes T . Further, TjY;Y and TjZ;Z have disjoint spectra if and only if  6= . Our aim now is to nd a decomposition (Y; Z ) of X such that the restrictions TjY;Y : Y ! Y and TjZ;Z : Z ! Z have disjoint spectra: sp(TjY;Y ) \ sp(TjZ;Z ) = ;:

23

1.2. DECOMPOSITIONS

Further, if   sp(T ) and sp(T ) n  are closed in Cj , then we would like to nd a decomposition (Y; Z ) of X such that sp(TjY;Y ) =  and sp(TjZ;Z ) = sp(T ) n . With this in mind, we introduce the following concept. A subset  of sp(T ) such that  as well as sp(T ) n  are closed in Cj is called a spectral set for T . Since sp(T ) is a closed set in Cj , it follows that a subset  of sp(T ) is a spectral set for T if and only if  is closed as well as open in sp(T ). Note that a singleton set fg is a spectral set for T if and only if  is an isolated point of sp(T ). If  is a spectral set for T , and ; is a Jordan curve in re(T ) satisfying sp(T ) \ int(;) = , then we say that the Jordan curve ; separates  from sp(T ) n . We remark that for some spectral sets  for T , there may be no Jordan curve which separates  from sp(T ) n .

Example 1.20 In this example, a single Jordan curve is not enough to separate a spectral set  for T from sp(T ) n : Let X := ` 2 and (ek ) be the standard basis for X . Let f1; 2 ; : : :g := [0; 1] \ Qj . We de ne T 2 BL(X ) by Tx := 2x(2)e2 +

1 X

k=1

e2i k x(k + 2)ek+2 for x :=

)

1 X k=1

6

1 0

1

2

-

x(k)ek 2 X:

24

1. SPECTRAL DECOMPOSITION

Clearly, 0, 2 and e2i j for each positive integer j are eigenvalues of T . In fact, they are all the eigenvalues of T . Since f1 ; 2 ; : : :g is dense in [0; 1], we obtain sp(T ) = f0; 2g[f 2 Cj : jj = 1g. If  := f 2 Cj : jj = 1g, then at least two disjoint Jordan curves are needed to separate  from sp(T ) n . The same holds if  := f0; 2g. The situation illustrated in Example 1.20 motivates the notion of a Cauchy contour. We rst introduce the notion of a Cauchy domain. An elementary Cauchy domain is a bounded open connected subset of Cj whose boundary is the union of a nite number of nonintersecting Jordan curves. A nite union of elementary Cauchy domains having disjoint closures is called a Cauchy domain.

Theorem 1.21

If E is a compact subset of Cj contained in an open subset , then there exists a Cauchy domain D such that E is a subset of D and the closure of D is a subset of .

Proof

Let F := Cj n and  := dist(E; F ). Then  > 0. For all integers i, j , let zi;j := (i +3i j )

and

p

S

Bi;j := fz 2 Cj : jz ; zi;j j <  6 3 g:

Then Cj  Bi;j . Since E is compact, it can be covered with a nite i;j number of such open discs having nonempty intersection with E , say n S E  D := Bik ;jk . Let z be in the closure of D. Find w 2 D such k=1 that jz ;wj < =3. Then there exists k 2 f1; : : : ;png such that w 2 Bik ;jk , and jz ; zik ;jk j  jz ; wj + jw ; zik ;jk j <  + 2 3=6 = . This implies that z 2= F and hence z 2 . The boundary of D is contained in the union of the boundaries of the discs Bik ;jk , k = 1; : : : ; n. Moreover, the part of the boundary of D inside each square of the grid de ned by the centers zik ;jk , k = 1; : : : ; n, is composed of an arc of a circle, or of two arcs of circles having a common endpoint, or of a disjoint pair of two such arcs. Hence the boundary of each connected component of D is a union of a nite number of Jordan curves.

1.2. DECOMPOSITIONS

25



 E is the shaded set,

is the entire open rectangle, D is the union of the open disks and E  D  closure of D  :

Thus each connected component of D is an elementary Cauchy domain. As a consequence, D is a Cauchy domain. Let D be a Cauchy domain. If each Jordan curve involved in the boundary of D is oriented in such a way that points in D lie to the left as the curve is traced out, then the oriented boundary C of D is called a Cauchy contour. The interior of a Cauchy contour C determined by a Cauchy domain D is de ned to be int(C) := D, and the exterior of a Cauchy contour C determined by a Cauchy domain D is de ned to be ext(C) := Cj n (D [ C). We remark that this notation is consistent with our earlier notation int(;) and ext(;) for a Jordan curve ;. Let C be a Cauchy contour. If E and Ee are subsets of Cj such that E  int(C) and Ee  ext(C), then we say that C separates E from Ee.

Corollary 1.22 Let E be a compact subset of Cj and Ee be a closed subset of Cj . If E \ Ee = ;, then there is a Cauchy contour C separating E from Ee. Further, there is a Cauchy contour separating E [ C from Ee, and there is a Cauchy contour separating E from Ee [ C. Proof

Letting := Cj n Ee in Theorem 1.21, we obtain a Cauchy domain D

26

1. SPECTRAL DECOMPOSITION

such that E is a subset of D and the closure of D does not intersect Ee. Now let C be the oriented boundary of D. Then E  int(C) and Ee  Cj n (D [ C) = ext(C), that is, the Cauchy contour C separates E from Ee . Next, since E [ C is compact and (E [ C) \ Ee = ;, we may replace E by E [ C to obtain a Cauchy contour separating E [ C from Ee. Similarly, since Ee [ C is closed and E \ (Ee [ C) = ;, we may replace Ee by Ee [ C to obtain a Cauchy contour separating E from Ee [ C. By Corollary 1.22, for every spectral set  for T , there is a Cauchy contour C that separates  from sp(T ) n . If  is a spectral set for T , the set of all Cauchy contours separating  from sp(T ) n  will be denoted by C (T; ). For a spectral set  for T and C 2 C (T; ), de ne Z 1 P (T; ) := ; 2i R(T; z ) dz; C which is the contour integral along C of the BL(X )-valued function z 7! R(T; z ) of the complex variable z . Since any Ce 2 C (T; ) can be continuously deformed in re(T ) to C, the bounded operator P (T; ) does not depend on the choice of C 2 C (T; ). This operator allows us to obtain a decomposition of T for which the restricted operators have disjoint spectra. For this purpose a number of properties of P (T; ) need to be proved rst.

Proposition 1.23

Let  be a spectral set for T and P := P (T; ). Then P is a bounded projection and it commutes with T . Moreover, if Pe 2 BL(X ) is any projection such that Pe commutes with T and R(Pe) = R(P ), then Pe = P .

Proof

Let C 2 C (T; ). By Corollary 1.22, there exists a Cauchy contour Ce that separates  [ C from sp(T ) n . Then Ce 2 C (T; ) and hence  2 Z hZ i P 2 = ; 21 i R(z )R(ze) dze dz: C e C By the First Resolvent Identity (Proposition 1.10(d)), we have

R(z ) ; R(ze) = (z ; ze)R(z )R(ze) for z 2 C; ze 2 Ce :

27

1.2. DECOMPOSITIONS

Hence

P 2 = ; 21 i 

2 Z h Z

C

2 Z h

e C

[R(z ) ; R(ze)] z d;zeze dz i

Z

= ; 21 i R(z ) z d;zeze dz e C CZ 2 Z h i  1 R(ze) z dz dze ; ; 2 i e C C ; ze 

i

e so that by interchanging the order of integrations. But C lies in int(C), e for each z 2 C and each ze 2 C, we have by Cauchy's theorem, Z

dze = ;2i and Z dz = 0: e z ; ze C z ; ze C This proves that P 2 = P . Also, P 2 BL(X ) because it is the contour integral of a BL(X )-valued function. The operators P and T commute because T commutes with R(z ) for each z on C. e PT e = T Pe and R(Pe ) = Let now Pe 2 BL(X ) be such that Pe2 = P; e (z ) = R(z )Pe for every z on C 2 C (T; ), so that R(P ). Then PR e e e 2 R(P ) and Px 2 R(Pe ), we have PP = P P . Let x 2 X . Since Px e = P Px e = PPx e Px = Px: Thus Pe = P . Let  be a spectral set for T and P := P (T; ). The fact that P and T commute implies that M := R(P ) and N := N (P ) = R(I ; P ) are invariant under T , and this pair of invariant subspaces decomposes T . Moreover, we shall show that they satisfy sp(TjM;M ) \ sp(TjN;N ) = ;.

For this purpose, we need an additional concept: Let C 2 C (T; ) and for 0 2 sp(T ) de ne Z S (T; ; 0 ) := ; 21 i R(T; z )  dz; z : 0 C Clearly, S (T; ; 0) 2 BL(X ) and it does not depend on C 2 C (T; ) in the same way as P (T; ).

Proposition 1.24 Let  be a spectral set for T , C 2 C (T; ), P := P (T; ), M := R(P ), N := N (P ), 0 2 sp(T ) and S := S (T; ; 0). Then

28

1. SPECTRAL DECOMPOSITION

(a) S commutes with T and with P . (b) If 0 2 , then S (T ; 0 I ) = I ; P and SP = O; so that 0 2 re(TjN;N ), ;




R TjN;N ; 0 = SjN;N and SjM;M = OjM;M :

(c) If 0 2 sp(T ) n , then S (T ; 0 I ) = ;P and SP = S; so that 0 2 re(TjM;M ), ;
R TjM;M ; 0 = ;SjM;M and SjN;N = OjN;N : Proof (a) Since T is continuous and commutes with R(z) for each z 2 C, T and S commute. Similarly, since P is continuous and commutes with R(z ) for z 2 C, P and S commute. (b) Let 0 2  and C 2 C (T; ). Then S (T ; 0 I ) = ; 21 i = ; 21 i = ; 21 i

Z

ZC

R(z )(T ; 0 I )  dz; z 0

R(z )(T ; zI + zI ; 0 I )  dz; z 0 ZC dz I + 1 Z R(z ) dz = I ; P; 2i C C 0 ; z Z since 0 2 int(C) and  dz; z = ;2i . C 0

Next, by Corollary 1.22, there exists a Cauchy contour Ce separating  [ C from sp(T ) n . Using C in the integral de ning S and Ce in the integral de ning P , and taking into account the First Resolvent Identity, we obtain Z 1 SP = ; 2i R(z )P  dz; z 0 ZC Z h i 1 1 = ; 2i R(z ) ; 2i R(ze) dze  dz; z C

e C

0

29

1.2. DECOMPOSITIONS 2 Z h Z i 1 [R(z ) ; R(ze)] ( ; zd)(ze z ; ze) dz = ; 2i 0 e C C  2 Z h R(z ) Z 1 d z e i = ; 2i dz e z ; ze CZ 0 ; z Z C  2 h i ; ; 21 i R(ze) ( ; zdz)(z ; ze) dze = O; e C C 0 

e so because C  int(C), for every ze 2 Ce and Z

dze = ;2i for every z 2 C, Z dz = 0 e C z ; ze C z ; ze

Z

Z Z dz 1 dz 1 =  ; ze z ; ze +  ; ze  dz; z = ; 2;i ze (  ; z )( z ; z e ) 0 0 0 C 0 C C 0

e for every ze 2 C. Now, S maps M into M , and it maps N into N , since S and P commute. Hence

SjM;M = OjM;M and SjN;N (T ; 0 I )jN;N = IjN;N : ;




In particular 0 2 re(TjN;N ) and R TjN;N ; 0 = SjN;N . (c) Let 0 2 sp(T ) n . As in part (b), Z Z 1 dz 1 S (T ; 0 I ) = ; 2i  ; z I + 2i R(z ) dz: C 0 C Z

But now 0 2 ext(C), so  dz; z = 0 and hence S (T ; 0 I ) = ;P . C 0 Next, by Corollary 1.22, there exists a Cauchy contour Ce separating  [ C from sp(T ) n . Using C in the integral de ning S and Ce in the integral de ning P , and taking into account the First Resolvent Identity, we obtain, as in part (b),

R(z ) Z dze i dz C 0 ; z Z e C z ; ze  2 Z h i 1 ; ; 2i R(ze) ( ; zdz)(z ; ze) dze:

SP = ; 21 i 

2 Z h

e C

Z

C

0

Z dz Since 0 2 sp(T ) n   ext(C), ( ; z )(z ; ze) = 0 and z d;zeze = e C C 0 ;2i . Hence SP = S .

30

1. SPECTRAL DECOMPOSITION

Again,

SjM;M (T ; 0 I )jM;M = ;IjM;M and SjN;N = OjN;N : ;
In particular 0 2 re(TjM;M ) and R TjM;M ; 0 = ;SjM;M .

Remark 1.25 Let T , , 0 , P and S be as in Proposition 1.24, and assume that 0 2 . Part (b) of Proposition 1.24 suggests a method of nding Sy for a given y 2 X . In fact, Sy = x if and only if x satis es the equations (T ; 0 I )x = y ; Py and Px = 0. This method will be useful later in computing successive iterates of certain re nement schemes.

Theorem 1.26 Spectral Decomposition Theorem: Let T 2 BL(X ),  be a spectral set for T , P := P (T; ), M := R(P ) and N := N (P ). Then T is decomposed by (M; N ), sp(TjM;M ) =  and sp(TjN;N ) = sp(T ) n : In particular, sp(TjM;M ) \ sp(TjN;N ) = ;. Proof

Since T and P commute, the pair (M; N ) decomposes T . By Proposition 1.18, we obtain sp(T ) = sp(TjM;M ) [ sp(TjN;N ): By Proposition 1.24, we have   re(TjN;N ) and sp(T ) n   re(TjM;M ): Considering the complements of the preceding sets in Cj and noting that sp(TjN;N ) as well as sp(TjM;M ) are subsets of sp(T ), we obtain sp(TjN;N )  sp(T ) n  and sp(TjM;M )  : Again considering the complements of the preceding sets in sp(T ), we see that   sp(T ) n sp(TjN;N ) and sp(T ) n   sp(T ) n sp(TjM;M ). Thus   sp(TjM;M ) and sp(T ) n   sp(TjN;N ).

Corollary 1.27 Let T 2 BL(X ),  be a spectral set for T and P := P (T; ). Then P = O if and only if  = ;; P = I if and only if  = sp(T ):

1.2. DECOMPOSITIONS

31

Proof

Let C 2 C (T; ). If  = ;, then the function z 7! R(z ) is analytic in int(C), so P = O by part (a) of Cauchy's Theorem. Conversely, if P = O, then M := R(P ) = f0g and so  = sp(TjM;M ) = ;. Next, if  = sp(T ) and we let N := N (P ), then sp(TjN;N ) = sp(T ) n  = ;, so N = f0g by the Gelfand-Mazur Theorem (Proposition 1.15(a)) and hence P = I . Conversely, if P = I , then N = f0g and sp(T ) n  = sp(TjN;N ) = ;, that is,  = sp(T ). The projection P := P (T; ) is called the spectral projection associated with T and , and the range of P is known as the spectral subspace associated with T and . This subspace will be denoted by M (T; ), or simply by M if no confusion is likely to arise. We now prove that it is the largest closed subspace of X which is invariant under T such that the spectrum of the restricted operator is contained in .

Proposition 1.28

Let  be a spectral set for T and P := P (T; ). If Y is a closed subspace of X such that T (Y )  Y and sp(TjY;Y )  , then Y  R(P ).

Proof

Let C 2 C (T; ). Since sp(TjY;Y )    int(C), we have C  re(TjY;Y ). As in the proof of Proposition 1.18, R(TjY;Y ; z ) = R(T; z )jY;Y for z 2 C. As a consequence, by Corollary 1.27,   Z Z PjY;Y = ; 21 i R(T; z ) dz = ; 21 i R(T; z )jY;Y dz Y;Y C C Z = ; 21 i R(TjY;Y ; z ) dz = IjY;Y : C Hence y = Py for all y 2 Y . This shows that Y  R(P ). Let  be a spectral set for T 2 BL(X ). The de nition of the spectral projection associated with T and , namely Z P (T; ) = ; 21 i R(T; z ) dz; C where C 2 C (T; ), is admittedly mysterious! The following result provides a motivation for this de nition. If a pair (Y; Z ) of closed subspaces

32

1. SPECTRAL DECOMPOSITION

of X decomposes T and sp(TjY;Y ) \ sp(TjZ;Z ) = ;, then the projection P onto Y along Z must equal P (T; ), where  := sp(TjY;Y ). (See Exercise 1.6.) Finally, we note that if  is an isolated point of sp(T ), then fg is a spectral set for T , and the operator Z S (T; ) := S (T; fg; ) = ; 21 i R(T; z )  dz ;z C does not depend on C 2 C (T; fg). S (T; ) is known as the reduced resolvent operator of T associated with . The reason for this name is that if N := N (P (T; fg)), then S (T; )jN;N = R(TjN;N ; ) = (TjN;N ; IjN;N );1 ; as can be seen by letting 0 =  in Proposition 1.24(b). We conclude this section with a result which will be of use on several occasions.

Proposition 1.29 Let X and Y be Banach spaces over Cj , T 2 BL(X ) and U 2 BL(Y ). Consider E  Cj such that sp(T ) n E = sp(U ) n E , and let   Cj satisfy  \ E = ;. Then  is a spectral set for T if and only if  is a spectral set for U . In this case, there is some C 2 C (T; ) such that E  ext(C), and every such C belongs to C (U; ) as well. Proof Let  be a spectral set for T . Then  is a closed subset of sp(T ), and sp(T ) n  is also a closed set. Since  \ E = ; and sp(T ) n E  sp(U ) n E , it follows that  is a closed subset of sp(U ). To show that sp(U ) n  is also a closed set, let (n ) be a sequence in sp(U ) n  such that n !  in Cj . As n 2 sp(U ) for each n, n !  and sp(U ) is a closed set, we see that  2 sp(U ). We prove that  2= . If  2 E , then clearly  2=  since  \ E = ;. Next, assume that  2= E . Since E is a closed set, there is some  > 0 such that fz 2 Cj : jz ; j < g\ E = ;. Also, as n ! , there is a positive integer n0 such that jn ; j <  and hence n 2= E for all n > n0 . Thus n 2 sp(U ) n E = sp(T ) n E and so n 2 sp(T ) n  for all n > n0 . Since sp(T ) n  is a closed set and n ! , it follows that  2 sp(T ) n . In particular,  2=  as desired. This shows that  is a spectral set for U . The converse follows by interchanging the roles of T and U .

1.3. SPECTRAL SETS OF FINITE TYPE

33

Let now  be a spectral set for T (and hence for U ). Letting E1 :=  and E2 := E [ (sp(T ) n ) in Corollary 1.22, we nd a Cauchy contour C which separates  from E [ (sp(T ) n ). As   int(C) and E [ (sp(T ) n )  ext(C), we see that C 2 C (T; ) and E  ext(C). Finally, since E2 = E [ (sp(U ) n ), we obtain C 2 C (U; ) as well.

1.3 Spectral Sets of Finite Type

Suppose that  is an isolated point of sp(T ). Clearly,  := fg is a spectral set for T . Let P := P (T; fg). Three questions arise naturally: 1. Must  be an eigenvalue of T ? 2. Can we relate M := R(P ) to the null space of T ; I or to the null space of some power of T ; I ? 3. Is the invariant subspace M nite dimensional? Matrix arguments allow us to give an armative answer to all of these questions if X is nite dimensional. In the general case, the answers are not always armative. Let T 2 BL(X ). If there exists a positive integer n such that T n = O, then T is called a nilpotent operator of degree n. If (T ) = 0, then T is called a quasinilpotent operator. By the Spectral Radius Formula (Proposition 1.15(b)), each nilpotent operator is quasinilpotent. The following example shows that the converse does not hold.

Example 1.30 In this example, sp(T ) = f0g but 0 is not an eigenvalue of T : Let X := C 0 ([0; 1]) and T : X ! X be de ned by (Tx)(s) :=

s

Z

0

x(t) dt; x 2 X; s 2 [0; 1]:

Clearly, T is a linear operator. Let x 2 X . Then j(Tx)(s)j  skxk1 for s 2 [0; 1]. Let us suppose that for a positive integer n, j(T n x)(s)j 

34

1. SPECTRAL DECOMPOSITION

kxk1 sn for s 2 [0; 1]. Then n!

j(T n+1 x)(s)j  kxnk!1

s

Z

0

tn dt = (nkx+k11)! sn+1 for s 2 [0; 1]:

This proves that for all positive integers n and all x 2 X , kT nxk1  kxk1 and hence kT nk  1 . So (T ) = lim kT nk1=n  lim p1 = n! 1 n! 1 n n! n! n! 0. This implies that sp(T ) = f0g. Although 0 is the only spectral value of T , it is not an eigenvalue since for x 2 X , (Tx)0 (s) = x(s) for s 2 [0; 1]. Thus Tx = 0 implies that x = 0. This shows that T is quasinilpotent but not nilpotent. By Corollary 1.27, P := P (T; f0g) = I and hence M := R(P ) = C 0 ([0; 1]). Note that M is in nite dimensional and N (T n ) = f0g for n = 1; 2; : : : Let  be an isolated point of sp(T ). The dimension of the spectral subspace M (T; fg) associated with T and  is called the algebraic multiplicity of , and we will denote it by m(T; ) or more simply by m when the context is clear. If m(T; ) is nite, then  is called a spectral value of nite type. We have taken this terminology from [38].

Proposition 1.31

Let  be an isolated point of sp(T ) and denote by P the corresponding spectral projection. Let M := R(P ) and N := N (P ). Then

(a)

1 S

j =1





N (T ; I )j  M and N 

1 T

j =1





R (T ; I )j ,

(b) if  is a spectral value of nite type with algebraic multiplicity m, then

M = N [(T ; I )m ] ; N = R [(T ; I )m ]

and

X = N [(T ; I )m ]  R [(T ; I )m ] : Further,  is an eigenvalue of T .

Proof (a) Let x 2 N (T ; I )j  for some j  1. Since T and P commute, we have (T ; I )j (x ; Px) = 0. Now (I ; P )x 2 N and (T ; I )jjN;N =

35

1.3. SPECTRAL SETS OF FINITE TYPE

(T ; I )jN;N j is invertible by Proposition 1.24(b). Hence (I ; P )x = 0, 1   S that is, x 2 M . Thus N (T ; I )j  M . j =1  Next, let x 2 N and j  1. Again by the invertibility of (T ;  I )jN;N j , there is some u 2 N such that 



  x = (T ; I )jN;N j u = (T ; I )j u: 



Hence x 2 R (T ; I )j . Thus N 

1 T j =1





N (T ; I )j .

(b) Let the algebraic multiplicity of  be m < 1. By Theorem 1.2, the operator TjM;M : M ! M can be represented with respect to an ordered basis for M by an upper triangular mm matrix whose  ; only eigenvalue
 is . Hence (T ; I )jM;M m = OjM;M . So M  N (T ; I )jM;M m  N [(T ; I )m ]. Next, let x 2 R [(T ; I )m ]. Then x = [(T ; I )]m u for some u 2 X . Since P commutes with T and Pu 2 M , we have Px = P (T ; I )m u = (T ; I )m Pu = 0; that is, x 2 N (P ) = N . Thus R [(T ; I )m ]  N . Now part (a) shows that

M = N [(T ; I )m ] and N = R [(T ; I )m ] : Hence X = M  N = N [(T ; I )m ]  R [(T ; I )m ]. Finally, since  2 sp(TjM;M ) and M is nite dimensional, we see that  is an eigenvalue of TjM;M and hence of T . Let  2 Cj and ' 2 X be such that (T ; I )k;1 ' 6= 0 and (T ; I )k ' = 0 for some positive integer k. Then := (T ; I )k;1 ' satis es

6= 0 and T =  ; so that  is an eigenvalue of T and a corresponding eigenvector. In this case, we say that ' is a generalized eigenvector of T corresponding to the eigenvalue . The integer k is called the grade of the generalized eigenvector '. Eigenvectors themselves are generalized eigenvectors of grade 1.

36

1. SPECTRAL DECOMPOSITION

The preceding proposition shows that if  is a spectral value of T of nite type, then  is an eigenvalue of T and the corresponding spectral subspace M (T; fg) consists of all generalized eigenvectors of T corresponding to  together with the null vector of X . Also, since N (T ; I )  M , the geometric multiplicity g of  is less than or equal to its algebraic multiplicity m. Further, we have   f0g ;= N (T ; I )  N (T ; I )2 









 N [(T ; I )m ] = N (T ; I )m+1 :

If we let ` be the smallest positive integer such that     N (T ; I )` = N (T ; I )`+1 ; then ` is called the ascent of the eigenvalue . Clearly, `  m. Note that the highest grade of a generalized eigenvector of T corresponding to  is `. If m = 1,  is called a simple eigenvalue. If ` = 1, then g = m and  is called a semisimple eigenvalue. If ` > 1 then g < m and  is called a defective eigenvalue. For relationships between the integers m, g and `, see Exercise 1.16. Since we shall be interested in clusters of spectral values of nite type, we introduce the following concept. If  is a nonempty spectral set for T and the corresponding spectral subspace is nite dimensional, then  is called a spectral set of nite type.

Theorem 1.32

Let  be a nonempty spectral set for T and denote by P the corresponding spectral projection. Then  is a spectral set of nite type for T if and only if  consists of a nite number of spectral values of T , each of which is of nite type. If  = f1 ; : : : ; r g and Pj denotes the spectral projection corresponding to T and j for j = 1; : : : ; r, then

P = P1 +


+ Pr ; Pi Pj = O if i 6= j and rank(P ) = m1 +


+ mr , where mj is the algebraic multiplicity of j , j = 1; : : : ; r.

Proof

Let M := R(P ). By the Spectral Decomposition Theorem, sp(TjM;M ) = :

1.3. SPECTRAL SETS OF FINITE TYPE

37

Assume rst that rank(P ) = dim M := m < 1. Now the spectrum of TjM;M consists of a nite number of eigenvalues. Thus  = f1 ; : : : ; r g for some positive integer r. Let C 2 C (T; ) and for j = 1; : : : ; r, let ;j denote a Jordan curve such that sp(T )\int(;j ) = fj g and int(;j )[;j  ext(;i ) for every i = 1; : : : ; r, i 6= j . Then by Cauchy's theorem, Z r Z r X X 1 1 P := ; 2i R(z ) dz = ; 2i R(z ) dz = Pj : C j =1 ;j j =1

Also, if i 6= j , then by the First Resolvent Identity, we have Z 1 Pi Pj = ; 2i R(z )Pj dz ; Z i Z h i 1 = ; 2i R(z ) ; 21 i R(w)dw dz ;i ;j  2 n Z h Z i o 1 = ; 2i [R(z ) ; R(w)] z dw ; w dz ;i ;j Z Z  2 n h dw idz = ; 21 i R (z ) ;i ;j z ; w

;

Z

;j

R(w)

hZ

dz idwo: ;i z ; w

But since ;i lies in ext(;j ) and ;j lies in ext(;i ), we have Z

dw = 0 for all z 2 ; and i z ;j ; w

Z

dz = 0 for all w 2 ; : j z ;i ; w

Thus Pi Pj = O if i 6= j , and

R(P ) = R(P1 ) 


 R(Pr ): r P

Hence rank(P ) = rank(Pj ). In particular, rank(Pj ) is nite for each j =1 j = 1; : : : ; r, showing that each j is a spectral value of T of nite type, and if mj := rank(Pj ) is its algebraic multiplicity, then rank(P ) = m1 +


+ mr . Conversely, if  = f1 ; : : : ; r g, where each j is a spectral value of T of nite type, then the argument given above shows that rank(P ) = r P rank(Pj ), which is nite since each rank(Pj ) is nite. j =1

38

1. SPECTRAL DECOMPOSITION

If X is nite dimensional and T 2 BL(X ), then every subset of sp(T ) is a spectral set of nite type for T . Let an nn matrix A represent T with respect to some basis B for X . Thus for z 2 re(T ), the matrix (A ; z I);1 represents R(T; z ) with respect to B. Also, if   sp(T ) and Z 1 C 2 C (T; ), then the matrix ; 2i (A ; z I);1 dz represents P (T; ) C with respect to B. We therefore, agree to say that the set of coordinate matrices of elements in P (T; ) with respect to B constitutes the spectral subspace associated with the matrix A and the spectral set . (Compare our comments about the eigenvalues, eigenvectors and eigenspaces of a matrix A in Remark 1.1.)

Remark 1.33 An upper triangular block diagonal form of a square matrix: Let X be an n dimensional linear space and T : X ! X be a linear operator. Let A 2 Cj nn be the matrix representing T with respect to an ordered basis B of X . We keep the notation of Theorem 1.32. Let  := sp(A). By Corollary 1.27, P (T; ) = I . For each eigenvalue j of A, let Pj := P (T; fj g). Then P1 +


+ Pr = I and R(P1 ) 


 R(Pr ) = X: Let Uj 2 Cj nmj be a matrix whose columns contain the coordinates relative to B of an ordered basis for R(Pj ), j = 1; : : : ; r. Since the spectral subspace R(Pj ) is invariant under T and sp(TjR(Pj );R(Pj ) ) = fj g, there exists Aj 2 Cj mj mj such that AUj = Uj Aj ; sp(Aj ) = fj g; j = 1; : : : ; r: In fact, Aj is the matrix which represents TjR(Pj );R(Pj ) : R(Pj ) ! R(Pj ) with respect to the ordered basis of R(Pj ), whose coordinates relative to B are given by the columns of Uj . By Schur's Theorem (1.2), there exists a unitary matrix Qj 2 Cj mj mj such that Tj := Qj Aj Qj is an upper triangular matrix. Then Tj = j Imj + Nj ; j = 1; : : : ; r;

where Nj 2 Cj mj mj is strictly upper triangular. In fact, Nj is a nilpotent matrix of degree `j , the ascent of the eigenvalue j of A. Let U := [U1 ; : : : ; Ur ] 2 Cj nn and Q := diag [Q1 ; : : : ; Qr ] 2 Cj nn . Then U is a nonsingular matrix, Q is a unitary matrix and T := (UQ);1 A (UQ) = diag [T1 ; : : : ; Tr ];

39

1.3. SPECTRAL SETS OF FINITE TYPE

since A(UQ) = [AU1 ; : : : ; AUr ] diag [Q1 ; : : : ; Qr ] = [U1 A1 ; : : : ; Ur Ar ] diag [Q1 ; : : : ; Qr ] = [U1 A1 Q1 ; : : : ; Ur Ar Qr ] = [U1 Q1 T1; : : : ; Ur Qr Tr ] = [U1 Q1 ; : : : ; Ur Qr ] diag [T1 ; : : : ; Tr ] = UQ diag [T1 ; : : : ; Tr ] = UQT:

Thus T is an upper triangular block diagonal matrix which is similar to A, and for j = 1; : : : ; n, all the diagonal entries of the j th mj mj diagonal block are equal to j . It follows that the characteristic polynomial of A is r Y p(z ) = (j ; z )mj : j =1 Hence the algebraic multiplicity mj of j as an eigenvalue of T is equal to the multiplicity of j as a root of the characteristic polynomial of A.

As a consequence, we deduce the Cayley-Hamilton Theorem:

p(A) := (1 In ; A)m1


(r In ; A)mr = UQ(1 In ; T)m1


(r In ; T)mr (UQ);1 = O: Let the characteristic polynomial of A be written as p(z ) = a0 +

n X i=1

ai z i :

If A is nonsingular, then 0 6= det A = a0 and hence h

; a1 a1 I + 

0

nX ;1

nX ;1

ai+1 Ai A = In = A ; a1 a1 I + ai+1 Ai : 0 i=1 i=1 i

This shows that A;1 = ;

h



;1  1 a I + nX i a A i+1 ; a 1 0

i=1

which is a polynomial in A of degree less than n.

i

40

1. SPECTRAL DECOMPOSITION

There is another upper triangular block diagonal form similar to A called the Jordan canonical form of A. This form will not be used in this book. The main reason for this is that, in general, the Jordan canonical form J(A) of a matrix A is not a continuous function of A. (See Section 3.1 in [43] and Exercise 1.17.)

Remark 1.34 Compact operators:

Important examples of spectral values of nite type are obtained by considering a class of operators which we now describe. Let T : X ! X be a linear operator. Then T is called a compact operator if the set E := fTx 2 X : x 2 X; kxk  1g is relatively compact, that is, its closure is a compact subset of X . Hence, if T is compact, then T 2 BL(X ). The set of all compact operators is a closed subset of BL(X ) and every bounded nite rank operator is compact. (See 17.2(c) and 17.1(c) of [55] or Theorems 8.1-5 and 8.1-5 of [48].) If T 2 BL(X ) is compact and A 2 BL(X ), then AT and TA are compact. Also, for each nonzero z 2 Cj , R(T ; zI ) is a closed subspace of X . A projection P 2 BL(X ) is compact if and only if rank(P ) is nite, since the restriction of P to R(P ) is the identity operator on R(P ) and the closed unit ball of a normed space is compact if and only if the normed space is nite dimensional. (See 5.5 of [55] or Lemma 8.1-2 of [48].) Let T 2 BL(X ) be a compact operator. Then sp(T ) is a countable (that is, a nite or a denumerable) set having 0 as the only possible limit point. In particular, each nonzero spectral value  of T is an isolated point of sp(T ). (See 18.5(a) of [55] or Section 8.3 of [48].) Let  be a spectral set for T not containing zero. Since f0g[(sp(T )n) and  are disjoint compact sets, there is, by Corollary 1.22, a Cauchy contour C separating  from fh0g [ sp(T ) n .i Clearly C 2 C (T; ) and 0 2 ext(C). Since R(T; z ) = 1z T R(T; z ) ; I for all nonzero z 2 re(T ), we have Z h i 1 T Z R(T; z ) dz; T R(T; z ) ; I dz = ; P (T; ) = ; 21 i z 2i C z C Z

because dz = 0. This proves that P (T; ) is a compact projection C z and hence its rank is nite. Thus every spectral set for T not containing 0 is of nite type. In particular, each nonzero spectral value of T is of nite type and, by Proposition 1.31, it is an eigenvalue of T .

1.4. ADJOINT AND PRODUCT SPACES

41

On the other hand, if X is in nite dimensional and  is a spectral set for T containing 0, then rank(P ) = 1. (See Exercise 1.15.)

1.4 Adjoint and Product Spaces 1.4.1 Adjoint Space

Let X be a complex Banach space. We say that f : X ! Cj is a conjugate-linear functional if for all x, y 2 X and c 2 Cj ,

f (cx + y) = cf (x) + f (y): The set of all conjugate-linear continuous functionals on X is called the adjoint space of X , and it will be denoted by X . It is easy to show that X  is a complex linear space. We shall use the notation

hx ; f i := f (x); x 2 X; f 2 X : Since the continuity of f 2 X  on X is equivalent to its continuity at 0, and since this is equivalent to the boundedness of f on the closed unit ball of X , we de ne the following norm on X : kf k := supfjhx ; f ij : kxk  1g: This de nition leads to the inequality

jhx ; f ij  kxk kf k; x 2 X; f 2 X : Thus the function h
;
i : X X  ! Cj is linear with respect to its rst argument, conjugate-linear with respect to its second argument, and continuous on X X . Since Cj is complete, X  is a complex Banach space. Let Y be a Banach space and K 2 BL(X; Y ), the space of all bounded linear maps from X to Y . We de ne the adjoint operator of K to be the operator K  : Y  ! X  given by

K f := fK for all f 2 Y  :

Then

hx ; K  f i = hKx ; f i for all x 2 X and f 2 Y :

42

1. SPECTRAL DECOMPOSITION

Given a nonempty subset E of X , we de ne its annihilator to be the following subset of X :

E ? := ff 2 X  : hx ; f i = 0 for all x 2 E g:

Remark 1.35

The following properties of the adjoint operator can be veri ed easily: (i) If K 2 BL(X; Y ), then K  2 BL(Y  ; X ). In fact, it can be easily seen that kK k  kK k. The inequality kK k  kK k can be proved by using the Hahn-Banach Extension Theorem. Hence kK k = kK k.

(ii) I  is the identity operator on X .

(iii) If K1 ; K2 2 BL(X; Y ), then for all c 2 Cj , (cK1 + K2) = cK1 + K2. (iv) If K 2 BL(X; Y ) and L 2 BL(Y; X ), then (KL) = LK  . (v) If E  X , then E ? is a closed subspace of X .

(vi) If K 2 BL(X; Y ), then N (K  ) = R(K )? and R(K  )  N (K )? .

(vii) K  2 BL(X ) is compact whenever K 2 BL(X ) is compact. The converse also holds. The proofs of the rst ve assertions can be found in Section 13 of [55] or Section 4.5 of [48], and of assertion (vii) in 17.3 of [55] or Theorem 8.2-5 of [48]. The following results concern spectral values of T and of T :

Proposition 1.36 Let T 2 BL(X ). (a) A complex number z belongs to re(T ) if and only if its conjugate z belongs to re(T ), and then R(T ; z) = R(T; z ). (b) A subset  of Cj is a spectral set for T if and only if  = f :  2 g is a spectral set for T , and then P (T ; ) = P (T; ).

43

1.4. ADJOINT AND PRODUCT SPACES

Proof (a) Let z 2 re(T ). Taking adjoints of both sides of (T ; zI )(T ; zI );1 = I = (T ; zI );1 (T ; zI ); we see that z 2 re(T ) and R(T ; z) = R(T; z ). Conversely, let z 2 re(T ). Since the operator T  ; zI  is injective, R(T ; zI )? = N (T  ; zI  ) = f0g; that is, R(T ; zI ) is dense in X . Consider now x 2 X . By the HahnBanach Extension Theorem, there is some f 2 X  such that hx ; f i = kxk and kf k = 1. Then kxk = hx ; f i = hx ; (T  ; zI  )(T  ; zI );1 f i = h(T ; zI )x ; (T  ; zI  );1 f i  k(T ; zI )xk k(T  ; zI  );1 k: This shows that the operator T ; zI is injective. Moreover, since X is complete, it can be seen that R(T ; zI ) is closed. As R(T ; zI ) is dense in X as well, the operator T ; zI is surjective. Thus z 2 re(T ). (b) We have sp(T ) = f :  2 sp(T )g by (a) above. It follows that a subset  of Cj is a spectral set for T if and only if  is a spectral set for T . For each Jordan curve ; : [a; b] ! Cj , de ne the conjugate Jordan curve  ; : [a; b] ! Cj by ; (t) := ;(a + b ; t); t 2 [a; b]: Then ; is a Jordan curve in re(T ). Note that if ; is positively oriented, then so is ;. Now if ; lies in re(T ), then ; lies in re(T ) and 1

2i

Z

;

R (T ; z )dz = 21 i 

b

R T ; ;(a + b ; t) d ;(a + b ; t)

Z

b

Z

= ; 21 i

a

a









R T ; ;(s) d ;(s)

Z b 1 = ; 2i R(T; ;(s))d ;(s) a

44

1. SPECTRAL DECOMPOSITION

1

Z

b



!

= 2i R(T; ;(s))d ;(s) a   Z = 21 i R(T; z )dz ; ; where we have used the change of variable s := a + b ; t, t 2 [a; b]. Let C 2 C (T; ) and denote by C the Cauchy contour comprising the conjugate curves of the curves involved in C. Then   int(C ) and sp(T ) n   ext(C ), so that C 2 C (T ; ). It follows that Z P (T ; ) = ; 21 i  R(T ; z )dz C   Z 1 = ; 2i R(T; z )dz C = P (T; ); as desired. Special properties concerning the adjoint space and the adjoint operator hold when the Banach space X has the following additional structure. A Banach space X is called a Hilbert space if the norm on X is induced by an inner product, that is, by a Hermitian positive de nite sesquilinear form h
;
i, as follows: kxk := hx ; xi1=2 for x 2 X . As we have mentioned earlier, elements x; y 2 X are said to be orthogonal if hx ; yi = 0. The subspace E ? := fx 2 X : hx ; yi = 0 for all y 2 E g is known as the orthogonal complement of the subset E  X .

Remark 1.37

The following results hold in a Hilbert space X : (i) Polarization Identity: For each linear operator A : X ! X and all x; y 2 X , 4hAx ; yi = hA(x + y) ; x + yi ; hA(x ; y) ; x ; yi +i hA(x + i y) ; x + i yi ; i hA(x ; i y) ; x ; i yi: (ii) Schwarz Inequality: For all x; y 2 X ,

jhx ; yij  kxk kyk:

1.4. ADJOINT AND PRODUCT SPACES

45

As a result, for each xed y 2 X , the conjugate-linear map f : X ! Cj given by f (x) := hy ; xi for x 2 X , is continuous and in fact kf k = kyk. (iii) Riesz Representation Theorem: For each f 2 X , there exists a unique y 2 X such that f (x) = hy ; xi for all x 2 X .

We see that the adjoint space X  is linearly isometric with X itself. We shall identify X  with X and write X  = X . Thus the notation h
;
i introduced for Banach spaces is consistent with the notation of an inner product on a Hilbert space. Also the annihilator is identi ed with the orthogonal complement. (iv) If Y is a closed subspace of X , then X = Y  Y ? and (Y ? )? = Y . (v) The following three conditions are equivalent to each other if  is a bounded projection de ned on X : (a)  = , (b) kk  1, (c) N () = R()? . If one of these conditions (and hence each of them) is satis ed, then  is called an orthogonal projection.

Proposition 1.38

Let X be a Hilbert space and T 2 BL(X ). Then kT k = supfjhTx ; yij : x; y 2 X; kxk  1; kyk  1g = kT T k1=2:

Proof If kxk  1 and kyk  1, then clearly jhTx ; yij  kT k. Now, if T= 6 O, there exists x0 such that kx0 k = 1 and Tx0 6= 0. Hence for y0 := Tx0 =kTx0k, we have jhTx0 ; y0 ij = kTx0k. The rst equality is proved. The second is proved as follows: kT T k  kT k kT k = kT k2 = sup kTxk2 = sup hTx ; Txi = sup hT Tx ; xi  kT T k.

kxk1

kxk1

kxk1

Proposition 1.39 Gram-Schmidt Process: Let X be a Hilbert space. If fx1 ; : : : ; xn g is a linearly independent set, then the following procedure leads to an orthonormal basis for the subspace spanfx1 ; : : : ; xn g, that is, to an ordered basis [e1 ; : : : ; en ] of elements of X which have norm 1 and which are orthogonal to each other: Since x1 6= 0, de ne e1 := x1 =kx1k.

46

1. SPECTRAL DECOMPOSITION Suppose that for some integer k satisfying 2  k < n, e1 ; : : : ; ek;1 have been constructed. kP ;1 De ne eek := xk ; hxk ; ei iei . Then eek 6= 0. i=1 De ne ek := eek =keek k.

Proof

Since x1 6= 0, e1 := x1 =kx1 k 6= 0 and spanfe1g = spanfx1 g. Suppose that for some integer k satisfying 2  k < n, we have spanfe1; : : : ; ej g = kP ;1 spanfx1 ; : : : ; xj g for j = 1; : : : ; k ; 1. If eek := xk ; hxk ; ei iei , then i=1 eek 6= 0, since otherwise xk 2 spanfx1 ; : : : ; xk;1 g, which is impossible. Let ek := eek =keek k. Clearly, fe1; : : : ; ek g is an orthonormal set and spanfe1 ; : : : ; ek g = spanfx1 ; : : : ; xk g.

Remark 1.40 Fourier series in separable Hilbert spaces:

Suppose that the Hilbert space X is a separable space, that is, X has a countable dense subset. Then X is equal to the closure of the span of some countable linearly independent set ff1; f2 ; : : :g and the GramSchmidt Process yields a countable orthonormal basis for X . Then each x 2 X has a Fourier Expansion

x=

1 X j =1

hx ; ej i ej = nlim !1 n x;

where n : X ! X , de ned by n x :=

n P

hx ; ej iej for x 2 X , is an orthogonal bounded nite rank projection onto spanfe1 ; : : : ; en g. j =1

Let T 2 BL(X ). We say that T is a normal operator if T T = TT , that T is a selfadjoint operator if T  = T , and that T is a unitary operator if T T = I = TT .

Proposition 1.41

Let X be a Hilbert space and T 2 BL(X ) be normal. (a) For each positive integer k, we have

N (T k ) = N (T ) and kT 2k k = kT k2k :

47

1.4. ADJOINT AND PRODUCT SPACES Also, (T ) = kT k.

(b) If  is an isolated point of sp(T ), then  is an eigenvalue of T ; and the corresponding eigenspace coincides with the spectral subspace associated with T and .

(c) Krylov-Weinstein Inequality: Let z 2 Cj . Then there exists  2 sp(T ) such that for all nonzero x 2 X, jz ; j  kTxk;xkzxk : Proof (a) For y 2 X , we have kTyk2 = hTy ; Tyi = hT Ty ; yi = hTT y ; yi = hT y ; T yi = kT yk2: Let x 2 X . Considering y = Tx, we obtain kTxk2 = hT Tx ; xi  kT (Tx)k kxk = kT (Tx)k kxk = kT 2xk kxk: Hence N (T 2)  N (T ). Since N (T )  N (T 2 ) always, we have N (T 2 ) = N (T ). By induction, it follows that N (T k ) = N (T ) for k = 1; 2; : : : Also, the preceding inequality shows that kT k2  kT 2k. Since kT 2k  kT k2 always, we obtain kT 2k = kT k2. Noting that the operator T 2 is k 2 normal, it follows by induction that kT k = kT k2k for k = 1; 2; : : : By the Spectral Radius formula (Proposition 1.15(b)),

n 1=n (T ) = nlim kT 2k k1=2k = kT k: !1 kT k = klim !1

(b) Let  be an isolated point of sp(T ), P := P (T; fg) and x 2 R(P ). Consider 0 <  < dist(; sp(T ) n fg), and let ; denote the positively oriented circle fz 2 Cj : jz ; j = g. Then ; 2 C (T; fg), and P = ; 21 i

Z

R(z )dz; TP = ; 21 i ;

Z

;

TR(z )dz:

For z 2 re(T ), we have by Proposition 1.10(e),

TR(z ) = I + zR(z ) = I + (z ; )R(z ) + R(z );

48

1. SPECTRAL DECOMPOSITION

so that (T ; I )R(z ) = I + (z ; )R(z ). Hence Z TP ; P = ; 21 i (T ; I )R(z )dz Z; 1 = ; 2i [I + (z ; )R(z )]dz Z; 1 = ; 2i (z ; )R(z )dz: ; Since T is normal, R(z ) is normal for each z 2 re(T ). By (b) above and Proposition 1.10(c), we obtain kR(z )k = (R(z )) = dist(z;1sp(T )) ; z 2 re(T ): Now assume that   dist(; sp(T ) n fg)=2. Then dist(z; sp(T )) =  for all z on ;. As a result,  kTP ; P k  `2(;) fjz ; j kR(z )kg = 22  jzmax ;j=   = : Letting  ! 0, we obtain TP = P . It follows that R(P )  N (T ; I ). Also, N (T ; I )  R(P ) by Proposition 1.31. Thus R(P ) = N (T ; I ). Also, N (T ; I ) 6= f0g, since P 6= 0 by Corollary 1.27. Thus  is an eigenvalue of T . (c) If z 2 sp(T ), then let  := z. If z 2 re(T ) and x 2 X , we have

kTx ; zxk : kxk = kR(z )(Tx ; zx)k  dist( z; sp(T ))

Since sp(T ) is a compact set, there is some  2 sp(T ) such that dist(z; sp(T )) = jz ; j; so that for all nonzero x 2 X .

jz ; j  kTxkx;kzxk ;

1.4.2 Product Space

We consider now a nite Cartesian product of the Banach space X . Let m be a positive integer and X 1m := f[x1 ; : : : ; xm ] : xj 2 X for j = 1; : : : ; mg:

49

1.4. ADJOINT AND PRODUCT SPACES

For x := [x1 ; : : : ; xm ] 2 X 1m , we de ne

k x k1 :=

m X j =1

kxj k; k x kF :=

m X j =1

kxj k2

1=2

; k x k1 := j=1max kx k : ;:::;m j

Then k
k1 , k
kF and k
k1 are equivalent norms on X 1m , and X 1m is a Banach space.

Remark 1.42 n1

Let X := Cj with the 2-norm. We identify the product space X 1m n  m with Cj . For A := [a1 ; : : : ; am ] = [ai;j ] 2 Cj nm , we obtain

kAkF :=

v um uX t

j =1

v um n uX X 2=t

kaj k2

j =1 i=1

jai;j j2 ;

which is called the Frobenius norm or the F-norm of the matrix A. That is why we have chosen the notation k
kF rather than k
k2 . (See also Exercise 1.20.) Let n be a positive integer. In the Banach algebra BL(X 1m ; X 1n ) of bounded linear operators from X 1m into X 1n , we consider the corresponding subordinated norm denoted simply by k
k. Let T 2 BL(X ). Then T induces an operator T 2 BL(X 1m ), called the natural extension of T to X 1m , as follows:

T [x1 ; : : : ; xm ] := [Tx1 ; : : : ; Txm] for [x1 ; : : : ; xm ] 2 X 1m : Clearly, I is the identity operator on X 1m . Also, k T k = kT k. We recall that for x := [x1 ; : : : ; xm ] 2 X 1m and Z = [zi;j ] 2 Cj mn ,

x Z :=

m hX i=1

zi;1 xi ; : : : ;

m X i=1

i

zi;n xi :

It easy to see that

T ( x Z) = ( T x )Z for all x 2 X 1m and Z 2 Cj mm : For this reason, it is unambiguous to write T x Z.

50

1. SPECTRAL DECOMPOSITION

Let x := [x1 ; : : : ; xm ] 2 X 1m and f := [f1 ; : : : ; fn ] 2 (X )1n . Then the Gram matrix G associated with x and f , or the Gram product of x with f , is de ned by G(i; j ) := hxj ; fi i:

It will be denoted by x ; f .

Proposition 1.43 If x := [x1 ; : : : ; xm ] 2 X 1m and f := [f1; : : : ; fm ] 2 (X )1m are such that the mm matrix x ; f is nonsingular, then the set fx1 ; : : : ; xm g is linearly independent in X and the set ff1; : : : ; fm g is linearly independent in X . Proof If fi 2 spanffj : j = 1; : : : ; m; j 6= ig, then the ith row of x ; f is a linear combination of the others rows. If xj 2 spanfxi : i = 1; : : : ; m; i = 6 j g, then the j th column of x ; f is a linear combination

of the others columns. In any of these events, x ; f would be a singular matrix. Let x 2 X 1m and x 2 (X )1m . We say that x is adjoint to x if

x ; x = Im ;

the mm identity matrix.

Remark 1.44 Let k, `, m and n be positive integers. We observe that for all x 2 X 1m , f 2 (X )1k , A 2 Cj mn and B 2 Cj k` , we have x A; f B = B x ; f A: This shows that if x 2 X 1m and f 2 (X )1m are such that V := x ; f is a nonsingular matrix, then y := f is adjoint to y := x V;1 and x := f (V );1 is adjoint to x .

Example 1.45 The Gram-Schmidt process revisited: Let ff1 ; : : : ; fn g be a linearly independent set in a Hilbert space X , and

51

1.4. ADJOINT AND PRODUCT SPACES

let e := [e1 ; : : : ; en ] be the orthonormal set obtained by applying the Gram-Schmidt Process to the ordered set f := [f1 ; : : : ; fn ]. It follows that e ; e = In and that f = e R for some upper triangular matrix R with positive diagonal entries. Moreover, f ; e = e R; e = e ; e R = R; which gives R in terms of e and f . For Z = [zi;j ] 2 Cj mn , consider the following matrix norms: m X

kZk1 := j=1max jz j; which is called the 1-norm; ;:::;n i=1 i;j kZk2 := (Z Z)1=2 ; which is called the 2-norm; kZk1 := i=1max ;:::;m

n X j =1

jzi;j j; which is called the 1-norm:

These matrix norms are subordinated operator norms. (See Exercise 1.21.) We shall also consider the F-norm of a matrix, de ned in Remark 1.42, m X n X 1=2 kZkF := tr(Z Z)1=2 = jzi;j j2 : i=1 j =1

p

This norm is not a subordinated operator norm since kInkF = n 6= 1, unless n = 1.

Proposition 1.46 For all x 2 X 1m and all f 2 (X  )1n , k x ; f k1  k x k1 k f k1 ; k x ; f k1  k x k1 k f k1 ; k x ; f kF  k x kF k f kF : Proof

For x := [x1 ; : : : ; xm ] 2 X 1m and f = [f1 ; : : : ; fn ] 2 (X )1n ,

k x ; f k1 = j=1max ;:::;m

n X i=1

jhxj ; fi ij  j=1max ;:::;m

= k x k1 k f k1 ;

n X i=1

kxj k kfik

52

1. SPECTRAL DECOMPOSITION

k x ; f k1 = i=1max ;:::;n

m X j =1

jhxj ; fi ij  i=1max ;:::;n

= k x k1 k f k 1 ;

m X j =1

kxj k kfik

and

k x ; f k2F = 

n X m X i=1 j =1

m X j =1

jhxj ; fi ij2 

kxj k2

n  X i=1

n X m X i=1 j =1

kxj k2 kfi k2



kfi k2 = k x k2F k f k2F :

This completes the proof. We shall show that the adjoint space (X 1m ) of X 1m can be identi ed with the product space (X )1m by means of a linear isometry. For f := [f1; : : : ; fm ] 2 (X )1m , let i( f ) 2 (X 1m ) be de ned by

i( f )( x ) :=

m X j =1

fj (xj ) for all x := [x1 ; : : : ; xm ] 2 X 1m :

Proposition 1.47 The map i : (X )1m ! (X 1m ) is linear, bijective and h x ; i( f )i = tr x ; f Also,

Proof

for all x 2 X 1m ; f 2 (X  )1m :

8 <

k f k1 if X 1m is given the 1-norm; ki( f )k = : k f k1 if X 1m is given the 1-norm; k f kF if X 1m is given the F-norm:

It is clear that the map i is linear, bijective and that h x ; i( f )i = m P fj (xj ) = tr x ; f for all x 2 X 1m and f 2 (X )1m .

j =1

53

1.4. ADJOINT AND PRODUCT SPACES

Let X 1m be given the 1-norm. Then for f 2 (X )1m ,

ki( f )k := supfjh x ; i( f )ij : k x k1  1g = sup

m n X i=1

n



hxi ; fi i : m X

 sup k f k1

i=1

m X

i=1 m X

kxi k :

o

kxi k  1

i=1

o

kxi k  1 = k f k1 :

For f 2 (X )1m , let j be such that k f k1 = kfj k. For x 2 X satisfying kxk  1, we have jhx ; fj ij = ji( f )[0; : : : ; 0; x; 0; : : : ; 0]j, where x is the j th entry. Taking supremum over all x 2 X satisfying kxk  1, we obtain

k f k1 = kfj k  ki( f )k := supfjh x ; i( f )ij : k x k1  1g: Thus ki( f )k = k f k1 . Let X 1m be given the 1-norm. Then for f 2 (X )1m , ki( f )k := supfjh x ; i( f )ij : k x k1  1g = sup

m n X n

i=1

 sup k x k1 =

m X i=1



o

hxi ; fi i : maxfkxi k : i = 1; : : : ; mg  1 m X i=1

kfi k : maxfkxi k : i = 1; : : : ; mg  1

o

kfi k = k f k1 :

Now, given  > 0, let xi 2 X be such that kxi k  1, hxi ; fi i  0 and kfi k < hxi ; fi i +  for i = 1; : : : ; m. De ne x := [x1 ; : : : ; xm ]. Then

jh x ; i( f )ij =

m X

m X

i=1

i=1

hxi ; fi i >

kfi k ; m:

m P

Hence ki( f )k  kfi k = k f k1 . Thus ki( f )k = k f k1 . i=1 Let X 1m be given the F-norm. Then for f 2 (X )1m ,

ki( f )k2 := supfjh x ; i( f )ij2 : k x kF  1g

54

1. SPECTRAL DECOMPOSITION

 sup  sup

m n X i=1

m nX i=1

2

jhxi ; fi ij :

kxi k2

m X i=1

m X

kxi k2  1

i=1 m X

kfi k2 :

i=1

o o

kxi k2  1 = k f k2F :

Now, given  > 0, let xi 2 X be such that kxi k  1 and kfik2 < jhxi ; fi ij2 +  for i = 1; : : : ; m: Then m m X X k f k2F = kfi k2  jhxi ; fi ij2 + m: i=1

This shows that

k f k2F  sup

m nX i=1

i=1

o

jhxi ; fi ij2 : kxi k  1; i = 1; : : : ; m :

Consider xi 2 X such that kxi k  1 for i = 1; : : : ; m and m X

 :=

i=1

jhxi ; fi ij2

1=2

De ne yi := hxi ; fi i xi , i = 1; : : : ; m. Then m X

Also,

i=1

kyi k2 = 12

m X i=1

m X i=1

and so

m X i=1

Hence

k f k2F  sup 

jhxi ; fi ij2 kxi k2  12

hyi ; fi i = 1

m X i=1

2

i=1

m X i=1

jhxi ; fi ij2 = 1:

jhxi ; fi ij2 = 

hyi ; fi i = 2 =

m nX

> 0:

m X i=1

jhxi ; fi ij2 : o

jhxi ; fi ij2 : kxi k  1; i = 1; : : : ; m

m n X sup i=1

2

hyi ; fi i :

m X i=1

o

kyi k2  1 = ki( f )k2 :

55

1.4. ADJOINT AND PRODUCT SPACES

Thus ki( f )k = k f kF .

Proposition 1.48 Let T 2 BL(X ). The adjoint of the extension T and the extension of the adjoint T  are related by ( T ) i( f ) = i[ (T ) f ] for f 2 (X  )1m : Proof

Let x 2 X 1m and f 2 (X )1m . Then by Proposition 1.47,

h x ; ( T ) i( f )i = h T x ; i( f )i = tr T x ; f = tr x ; (T ) f = h x ; i[ (T ) f ]i;

as we wanted to prove.

Proposition 1.49 Let Z 2 Cj mm and Z : X 1m ! X 1m be de ned by Z x := x Z. Suppose X 1m is given the p-norm, p = 1; 1 or F. Denote by k
k the corresponding subordinated operator norm on BL(X 1m ), as usual. Then Z 2 BL(X 1m ) and 8 < kZk1 if X 1m is given the 1-norm; kZk  : kZk1 if X 1m is given the 1-norm; kZkF if X 1m is given the F-norm: Moreover, the adjoint Z  of Z satis es Z i( f ) = i( f Z) for f 2 (X )1m : Proof

Let zi;j denote the (i; j )th entry of Z and x := [x1 ; : : : ; xm ] 2 X 1m . Then

kZ x k1 =

m X m X

j =1 i=1

kZ x k1 = max

zi;j xi



m n X

i=1

m X i=1

kxi k

m X j =1

jzi;j j  kZk1 k x k1 ; o

zi;j xi

: j = 1; : : : ; m

56

1. SPECTRAL DECOMPOSITION

 max

m nX i=1

jzi;j j kxi k : j = 1; : : : ; m

 k x k1 max and

kZ x k2F =

m X m X

j =1 i=1

m nX i=1

2

zi;j xi



o o

jzi;j j : j = 1; : : : ; m = kZk1 k x k1 ;

m X m X j =1 i=1

jzi;j j2

m X j =1

kxi k2  kZk2F k x k2F :

Hence the bounds for kZk follow. Let x 2 X 1m and f 2 (X )1m . Then by Proposition 1.47,

hZ x ; i( f )i = tr Z x ; f = tr x Z; f

= tr x ; f Z = tr Z x ; f = tr x ; f Z = h x ; i( f Z)i;

as we wanted to prove. We shall identify (X 1m ) with (X )1m without writing explicitly the linear isometry i : (X )1m ! (X 1m ) and, as a consequence, the adjoint of the extension of T will be identi ed with the extension of the adjoint of T . It will be denoted by T  . Then for x 2 X 1m and f 2 (X )1m , we have

T x; f = x; T f :

The notions of the resolvent operator and the reduced resolvent operator can be extended to the product space in the following way. Instead of considering the equation Tx ; zx = y, where z 2 Cj and y 2 X , we consider the Sylvester equation

T x ; xZ = y where Z 2 Cj mm and y 2 X 1m .

Proposition 1.50 mm Let T 2 BL(X ), Z 2 Cj and S : X 1m ! X 1m be de ned by S ( x ) := T x ; x Z for x 2 X 1m :

1.4. ADJOINT AND PRODUCT SPACES

57

Then sp(S ) = f ;  :  2 sp(T );  2 sp(Z)g. As a consequence, for every y 2 X 1m the equation

T x ; xZ = y has a unique solution x 2 X 1m if and only if

sp(T ) \ sp(Z) = ;:

Proof

By Remark 1.3, there exists a unitary matrix Q 2 Cj mm such that R := QZQ is an upper triangular matrix. For x ; y 2 X 1m , let

u = [u1; : : : ; um] := x Q and v = [v1 ; : : : ; vm ] := y Q: Let  2 Cj . Then (S ;  I ) x = y if and only if ( T ;  I ) u ; u R = v . Also, ( T ;  I ) u ; u R = v if and only if the following m equations in X are satis ed: (E 1) (T ; ( + R(1; 1))I )u1 = v1 jP ;1 (Ej ) (T ; ( + R(j; j ))I )uj = vj + R(i; j )ui for j = 2; : : : ; m: i=1

Suppose that there exists  2 sp(Z) such that  :=  +  2 sp(T ). By Remark 1.3, Q can be so chosen that R(1; 1) = . Then equation (E 1) has either a nonzero solution in X when v1 = 0, or it has no solution in X for some v1 . Hence (S ;  I ) x = y has either a nonzero solution in X 1m when y = 0 , or it has no solution in X 1m for some y . Thus  ;  =  2 sp(S ). Conversely, suppose that  2 sp(S ). Then there exists an integer j such that 1  j  m and  + R(j; j ) 2 sp(T ) since, otherwise, equation (E 1) has a unique solution for each v1 2 X and equation (Ej ) has a unique solution for each vj 2 X for j = 2; : : : ; m, that is, (S ;  I ) x = y has a unique solution for each y 2 X 1m . Let  := R(j; j ) and  =  + . Then  =  ; , where  2 sp(Z) and  2 sp(T ). This proves that sp(S ) = f ;  :  2 sp(T );  2 sp(Z)g. As a consequence, 0 2 re(S ) if and only if sp(T ) \ sp(Z) = ;, proving the second assertion of the proposition. Let Z 2 Cj mm and Z be the operator de ned in Proposition 1.49: Z x := x Z for x 2 X 1m . Proposition 1.50 says that the operator T ; Z is bijective if and only if sp(Z) \ sp(T ) = ;. In that case, it has

58

1. SPECTRAL DECOMPOSITION

a bounded inverse in BL(X 1m ), which we denote by R( T ; Z) and call it the block resolvent operator of T at Z. Although general nite rank operators will be studied in Chapter 4, the following result concerning bounded nite rank projections is needed for representing the spectral projection associated with a spectral set of nite type.

Lemma 1.51 Let P 2 BL(X ) be a projection with nite rank equal to m and let 2 X 1m form an ordered basis for R(P ). Then there exists a unique ordered basis  2 (X )1m for R(P  ), which is adjoint to . Also, Px = x;  for all x 2 X; P x = x ;  for all x 2 X 1m ; P  f =  ; f  for all f 2 X ; P  f =  ; f  for all f 2 (X )1m : In particular, P  is a projection and rank(P ) = rank(P ). Proof

Let := [ 1 ; : : : ; m ]. Given x 2 X , there exist m scalars ci (x), i = 1; : : : ; m, uniquely de ned by x, such that

Px =

m X i=1

ci (x) i :

Since P is linear, given x; y 2 X and c 2 Cj , we have P (cx + y) = cPx + Py, and so m X i=1

[ci (cx + y) ; cci (x) ; ci (y)] i = 0:

As the set f 1 ; : : : ; m g is linearly independent, we conclude that the functionals ci , i = 1; : : : ; m, are linear. Given a1 1 +


+am m 2 R(P ), we de ne ka1 1 +


+ am m k1 := maxfjai j : i = 1; : : : ; mg: Then k
k1 is a norm on R(P ) and since all norms on R(P ) must be equivalent, k
k1 is equivalent to the norm induced on R(P ) by the

59

1.4. ADJOINT AND PRODUCT SPACES

norm of X . Hence there exists a constant  such that for all x 2 X ,

kPxk1 = maxfjci (x)j : i = 1; : : : ; mg  kPxk  kP k kxk: This proves that each linear functional ci is continuous. De ne i 2 X  by hx ; i i := ci (x) for x 2 X , i = 1; : : : ; m, and  := [ 1 ; : : : ; m ]. Let x 2 X . Then Px =

m X i=1

x;  :

hx ; i i i =

Thus the formul for Px and P x follow. Let f 2 X . Then for x 2 X , (P  f )x := f (Px) = f =

m X i=1 m P

m X i=1



c i (x ) i =

m X i=1

hx ; i if ( i )

h i ; f i i (x):

and hence P  f = h i ; f i i . i=1 Now the formul for P f and P  f follow. This also shows that R(P  )  spanf 1 ; : : : ; m g. Moreover, since P is a projection, P i = i for i = 1; : : : ; m. Thus ;  = Im , that is,  is adjoint to . By Proposition 1.43, this equality implies the linear independence of the set f 1 ; : : : ; m g in X . But since m X P  j = h i ; j i i = j ; i=1

2 R(P  ) for j = 1; : : : ; m. This proves that ordered basis for R(P  ). we have j

 forms an

To prove the uniqueness of  , let e form an ordered basis for R(P  ) which is adjoint to . Then there exists a matrix V 2 Cj mm such that e =  V and Im =

; e =

;  V = V

Hence V = Im , that is, e =  .

;  = V :

60

1. SPECTRAL DECOMPOSITION

We note that a much shorter proof of the preceding result can be given if we use the Hahn Banach Extension Theorem: Let f1 ; : : : ; fm in X  be such that h j ; fi i = i;j , i; j = 1; : : : ; m. Let us de ne j := P  fj , j = 1; : : : ; m. Then  := [ 1 ; : : : ; m ] forms an ordered basis for R(P  ), which is adjoint to .

Theorem 1.52 Let T 2 BL(X ),  be a spectral set of nite type for T , P := P (T; ), m := rank(P ) and ' be an ordered basis for M := R(P ). Then there is a unique ordered basis ' for R(P  ) which is adjoint to ' . Let  := T ' ; ' . Then T ' = ' , T  ' = '  and sp() = . Also,

P x = ' x ; '

and

P T x = '  x ; '

for x 2 X 1m :

Proof

The existence and uniqueness of ' follows from Lemma 1.51, as also the relation P x = ' x ; ' for all x 2 X 1m . Since the subspace M is invariant under T , there exists a matrix Z 2 Cj mm such that T ' = ' Z. As ' ; ' = Im , we obtain  = T ' ; ' = ' Z; ' = Im Z = Z: Thus T ' = ' . Now the matrix  represents the operator TjM;M with respect to the ordered basis ' for M . Hence sp() = sp(TjM;M ). But sp(TjM;M ) =  by the Spectral Decomposition Theorem. Thus sp() = . Again, since R(P  ) is invariant under T , there is a matrix W 2 Cj mm such that T  ' = ' W. But as ' ; ' = Im , W  = ' ; ' W = ' ; T  '  = T ' ; '  =  : Thus T  ' = ' . Finally, for all x 2 X 1m , we have P T x = ' T x ; ' = ' x ; T  ' = ' x ; '  = '  x ; ' ; as desired.

Remark 1.53 Dependence of  on ' :

The matrix  in Theorem 1.52 does not depend on ' 2 (X )1m but

61

1.4. ADJOINT AND PRODUCT SPACES

only on T and ' . Indeed, if f 2 (X )1m is adjoint to ' , then also T ' ; f = ' ; f = ' ; f  = Im  = : If forms another ordered basis for M , then there exists a nonsingular matrix V 2 Cj mm such that = ' V, that is, the columns of V contain the coordinates of the elements in relative to the ordered basis ' . Since the corresponding ordered basis  for R(P  ) adjoint to must satisfy  = ' B for some matrix B 2 Cj mm , we have

;  = ' V; ' B = B ' ; ' V = B V

Im =

and hence B = V;1  . Thus the matrix representing TjM;M with respect to the ordered basis is ;




;
T ;  = T ' V; ' V;1  = V;1 V;

as we learn in our rst course in linear algebra! Let T , , P and  be as in Theorem 1.52 As sp() =   sp(T ), we have re(T )  re(). Since T ' = ' , we observe that for all z 2 re(T ), R(T; z ) ' = ' R(; z ) and hence R(; z ) = R(T; z ) ' ; ' : We de ne the block reduced resolvent operator S' : X 1m ! X 1m associated to T ,  and ' as follows. Let C 2 C (T; ) and Z S' x := ; 21 i R(T; z ) x R(; z ) dz for x 2 X 1m : C Note that the operator S' does not depend on C 2 C (T; ).

Remark 1.54 The semisimple case: If  is a semisimple eigenvalue of T , then R(P ) = N (T ; I ), and any ordered basis ' of R(P ) contains only eigenvectors of T corresponding to . Hence  =  Im , R(; z ) =  ;1 z Im for all z = 6  and S' x := ; 21 i

Z

C

R(T; z ) x  dz ; z = S (T; ) x

for x 2 X 1m :

62

1. SPECTRAL DECOMPOSITION

Thus the operator S' is a generalization of the operator S (T; ) to the case of a cluster of possibly defective eigenvalues of T .

Remark 1.55 Dependence of S' on ' : By Remark 1.53, if we change the basis ' for M (T; ) to a basis := ' V, where V is a nonsingular matrix, then the matrix R(; z ) in the de nition of S' must be replaced by the matrix V;1 R(; z )V. Hence Z   1 S x = ; 2i R(T; z ) x V;1 R(; z )V dz = S' ( x V;1 ) V C

for all x 2 X 1m . Let ' form an ordered basis for R(P ), ' form the corresponding adjoint ordered basis for R(P  ), and let Z' : X 1m ! X 1m be de ned by Z' x := x T ' ; ' for x 2 X 1m :

Proposition 1.56

The following properties hold: (a) The operators T , P , S' and Z' commute. (b) ( T ; Z' )S' = I ; P .

(c) S' P = O . Proof

Let  := T ' ; ' . (a) Since ( T x ) = T ( x ) for x 2 X 1m , Z' and T commute. The same argument holds for P and Z' . Since T and P commute, so do T and P . Since T is continuous and commutes with R(T; z ) for z 2 C, T and S' commute. The same argument holds for P and S' . Since  and R(; z ) commute for z 2 C, so do Z' and S' . (b) Let C 2 C (T; ). Then Z 1 ( T ; Z' )S' x = ; 2i R(T; z ) ( T x ; x )R(; z ) dz C

63

1.4. ADJOINT AND PRODUCT SPACES

Z 1 = ; 2i R(T; z ) ( T x ; z x + z x ; x )R(; z ) dz C Z Z h i i h = x ; 21 i R(; z ) dz + 21 i R(T; z ) dz x C C = (I ; P )x;

since sp()  int(C) and hence

Z

C

R(; z ) dz = ;2i Im .

(c) Let C 2 C (T; ). By Corollary 1.22, there exists Ce separating  [ C from sp(T ) n . Using C in the integral de ning S' and Ce in the integral de ning P , and taking into account the First Resolvent Identity, we obtain for x 2 X 1m , Z S' P x = ; 21 i R(T; z ) ( P x )R(; z ) dz C

Z Z h i 1 1 = ; 2i R(T; z ) ; 2i R(T; ze) x dze R(; z ) dz e C Z Z C 2 h  ; z ) dzei dz 1 [ R(T; z ) ; R(T; ze) ] x Rz( = ; 2i ; ze e Z ZC h C   i 2 = ; 21 i R(T; z ) x R(; z ) z d;zeze dz e C C Z  2 Z h i 1 R ( ; ; 2 i R(T; ze) x z ;;zez ) dz dze: Z

e C

C

Z

dze = ;2i for every z 2 C and dz = 0 e so But C  int(C), e z ; ze C C z ; ze e Also, by the First Resolvent Identity, we have for ze 2 C, e for every ze 2 C. Z R(; z ) dz = R(; ze)Z dz + R(; ze)Z R(; z ) dz = ;2i R(; ze); C

z ; ze

C z ; ze

2 X 1m ,

C

since sp()  int(C). Thus for x Z Z S' P x = ; 21 i R(T; z ) x R(; z ) dz + 21 i R(T; ze) x R(; ze) dze= 0 e C C (Compare the proof of part (b) of Proposition 1.24.) The operator S' can be described with the help of the operator G' : ! X 1m de ned by G' := ( I ; P ) T ; Z' :

X 1m

64

1. SPECTRAL DECOMPOSITION

Proposition 1.57

Let  be a spectral set of nite type for T , P := ; P
(T; ) and N := N (P ). If ' forms an ordered basis for R(P ), then G' jN 1m ;N 1m is bijective and ;
;1 S' = G' jN 1m ;N 1m ( I ; P ):

Proof

By Proposition 1.56, S' x 2 N 1m for all x 2 N 1m and G' S' x = ( T ; Z' )S' x = ( I ; P ) x :

Let ' form a basis for R(P  ) which is adjoint to ' and consider  := T ' ; ' . Replacing X 1m by N 1m = R( I ; P ) = [R(I ; P )]1m in Proposition 1.50 and noting that sp() \ sp(TjN;N ) = sp(TjM;M ) \ sp(TjN;N ) = ;; ;
we see that the operator G' jN 1m ;N 1m is invertible in BL(N 1m ). Hence the desired result follows.

Remark 1.58

The preceding result suggests a method of nding S' y for a given y 2 X 1m . In fact, since S' y = S' ( y ; P y ) and y ; P y 2 N ( P ) = N (P )1m , we observe that S' y is the unique element x 2 X which satis es T x ; x T ' ; ' = y ; P y and P x = 0 . (Compare Remark 1.25.) Assume now that 0 2= . Since sp((I ; P )T ) = sp((I ; P )T (I ; P )) = sp(TjN;N ) [f0g = (sp(T ) n ) [f0g and sp() = , we see that sp((I ; P )T ) \ sp() = ;. Then, by Proposition 1.50, the operator G' is invertible in BL(X 1m ) and hence S' = G';1 ( I ; P ).

1.5 Exercises

Unless otherwise stated, X denotes a Banach space over Cj , X 6= f0g and T 2 BL(X ). Prove the following assertions.

65

1.5. EXERCISES

1.1 Let Y be a closed subspace of X which is invariant under T . Then (TjY;Y )  (T ). In general, sp(TjY;Y ) may not contain sp(T ), and sp(T ) may not contain sp(TjY;Y ). 1.2 Let K 2 BL(X; Y ) and L 2 BL(Y; X ). Then re(LK ) n f0g = re(KL) n f0g and for z in this set, R(KL; z ) = z1 [K R(LK; z )L ; IY ];

R(LK; z ) = z1 [L R(KL; z )K ; IX ]:

1.3 Let C be a Cauchy contour Z contained in re(T ). Then (a) If sp(T )  int(C), then R(z ) dz = ;2i I . C Z 1 (b) If sp(T )  ext(C), then R(z0 ) = 2i zR;(zz) dz for every z0 2 C

0

int(C). 1.4 Let  be an isolated point of sp(T ) and de ne M := R(P (T;fg)). Then M = fx 2 X : k(T ; I )n xk1=n ! 0 as n ! 1g. Hint: For x 2 X such that k(T ; I )n xk1=n ! 0, and z 2 re(T ), R(z )x = k  1 ; P ((Tz ;; I)k)+1x . k=0

1.5 Let  be a spectral set for T ,  2  and j be any positive integer. Then N [(T ; I )j ]  M (T; ). (Hint: Y := N [(T ; I )j ] is a closed subspace of X which is invariant under T and sp(TjY;Y ) = fg  . Use Proposition 1.28.) 1.6 Let (Y; Z ) decompose T 2 BL(X ), P denote the projection on Y along Z , and C be a Cauchy contour in re(T ) suchZthat sp(TjY;Y )  int(C) and sp(TjZ;Z )  ext(C). Then P = ; 21 i R(T; z ) dz, and C Z R(TjZ;Z ; z ) 1 that for every z0 2 int(C), R(TjZ;Z ; z0 ) = 2i dz . (These C z ; z0 results provide a motivation for the formul of the spectral projection and the reduced resolvent operator associated with T .) 1.7 Let U be in BL(X ). As for matrices, T is said to be similar to U , if there exists an invertible operator V 2 BL(X ) such that U = V ;1 TV . Let  be a spectral set for T and T be similar to U . Then  is a spectral set for U and rank P (U; ) = rank P (T; ).

66

1. SPECTRAL DECOMPOSITION

1.8 Let  be a spectral set for T , C 2 C (T; ), P := P (T; ) and k be a nonnegative integer. Then Z k Z R (z ) T 1 k k T P = ; 2i z R(z ) dz and P = ; 2i z k dz: C C k If T is a compact operator, then the spectral set  is of nite type. 1.9 Let T be compact, U 2 BL(X ),  2 sp(U ). Then  2 sp(T + U ), if  is not an eigenvalue of U . (Hint: If z 2 re(T + U ), then U ; zI = (T + U ; zI )[I ; (T + U ; zI );1 T ].) 1.10 Let  2 Cj . (a)  is an isolated point of sp(T ) if and only if there are closed subspaces Y and Z of X such that (Y; Z ) decomposes T , sp(TjY;Y ) = fg and  2= sp(TjZ;Z ). In this case, S (T; fg) = zlim R(T; z )(I ; P (T; fg)). ! (b)  is a spectral value of nite type for T if and only if there are closed subspaces Y and Z of X such that (Y; Z ) decomposes T , dim Y is nite, sp(TjY;Y ) = fg and  2= sp(TjZ;Z ). 1.11 Not every eigenvalue of a bounded operator is a spectral value of nite type. 1.12 Let  be a spectral set for T and e := sp(T ) n . Then P (T; )+ e = I and P (T; )P (T; ) e = O. P (T; ) 1.13 Let 1, 2 be spectral subsets for T . Then 1  2 if and only if M (T; 1 )  M (T; 2): (Hint: For the `only if' assertion, E := 1 n 2 is a spectral set for T , R(P (T; E ))  M (T; 1) and R(P (T; E )) \ M (T; 2) = f0g.) 1.14 Let  be an isolated point of sp(T ) and z 2 1Cj be such that P k 0 < jz ; j < dist(; sp(T ) n fg). Then R(z ) = S (z ; )k ; k=0

1 Dk;1 P ;P z ;  k=2 (z ; )k , where P := P (T; fg), S := S (T; fg) and D := (T ; I )P . Hence  is a pole of R(
) if and only if D is nilpotent. In that case, the order ` of the pole is the smallest positive integer such that D` = O. It is also the smallest positive integer ` such that N [(T ; I )` ] = N [(T ; I )`+1 ]. Further, R(P (T; fg)) = N [(T ; I )` ] and N (P (T; fg)) = R[(T ; I )` ] and  is, in fact, an eigenvalue of T with ascent `.

67

1.5. EXERCISES

1.15 Let T be a compact operator and  a spectral set for T containing 0. Then if dim X is in nite, rank P (T; ) is in nite. 1.16 Let g, ` and m be the geometric multiplicity, the ascent and the algebraic multiplicity of an isolated spectral value  of T . Then g + ` ; 1  gl  m. Conversely, if g, ` and m are positive integers such that g + ` ; 1  g`  m, then there is a matrix A and an eigenvalue  of A such that g, ` and m are its geometric multiplicity, ascent, and algebraic multiplicity, respectively. (See [59].)   1.17 The Jordan canonical form may be discontinuous: Let A := 10 11  

1 1 , n = 1; 2; : : : Let J and J , n = 1; 2; : : : be their and An := 1=n n 1 Jordan canonical forms, respectively. If k
k be any norm in Cj 22 , then 6 0. nlim !1 kAn ; Ak = 0 but nlim !1 kJn ; Jk =

1.18 Let  be a spectral set for T . Then S (T ; ; 0 ) = S (T; ; 0 ) for each  2 sp(T ):

1.19 Let X be Hilbert space and h
;
i its inner product. If T is selfadjoint, then sp(T ) is a subset of IR. 1.20 Let X be a Hilbert space and h
;
i its inner product. Consider the product space X 1m and de ne, for x 2 X 1m , k x k2 := ( x ; x )1=2 . Then k
k2 is a norm on X 1m , which is equivalent to the norm k
kp for p = 1; F; 1. (Hint: k x k2 = maxfk x uk : u 2 Cj m1 ; kuk2 = 1g.)

1.21 Let Z := [zi;j ] 2 Cj mn. Then (a) kZk1 = maxfkZxk1 : x 2 Cj n1 ; kxk1  1g. (b) kZk2 = maxfkZxk2 : x 2 Cj n1 ; kxk2  1g. (c) kZk1 = maxfkZxk1 : x 2 Cj n1 ; kxk1  1g. 1.22 Let Z := [zi;j ] 2 Cj mn and de ne the matrix jZj 2 Cj mmn by jZj := [ jzi;j j ]. Then k j
j kp , p = 1; 2; 1; F, de nes a norm on Cj n , and for all Z := [zi;j ] 2 Cj mn , we have (a) kZk1 = k jZj k1 = kZk1 . (b) kZk1 = k jZj k1 = kZk1 . (c) k jZj k2  k jZj kF = kZkF . p (d) p1 kZk  k jZj k  m kZk . m

2

2

2

68

1. SPECTRAL DECOMPOSITION

1.23 Let X := Cj n1 with a norm k
k such that formeach standard basis vector ej , kej k = 1, j = 1; : : : ; n. Let Z 2 Cj m and de ne Z : Cj nm ! Cj nm by Z x := x Z for x := [x1 ; : : : ; xm ] 2 Cj nm . Then m P (a) kZk = kZk1 if Cj nm is given the 1-norm, k x k1 := kxj k. j =1

(b) kZk = kZk1 if Cj nm is given the 1-norm, k x k1 := j=1max kx k. ;:::;m j

1.24 Let m, n and p be integers. (a) If A 2 Cj nm and B 2 Cj mp , then kABkF  kAkF kBk2 and kABkF  kAk2 kBkF g. (b) Let m and n be such that m  n. Let V 2 Cj nm and [e1 ; : : : ; em] be the standard basis for Cj m1 . Then V is a unitary matrix if and only if kVk2 = kVej k2 = 1 for all j = 1; : : : ; m. (c) A matrix A 2 Cj nn is normal if and only if there exists a nonsingular matrix V such that V;1 AV is a diagonal matrix and 2 (V) := kVk2 kV;1 k2 = 1. 1.25 Let  be a spectral set for T . Consider a positive integer p, x := [x1 ; : : : ; xp ] 2 X 1p and Z 2 Cj pp such that T x = x Z. Let Y := spanfx1 ; : : : ; xp g. Then (a) Y is; a closed invariant subspace for T .
(b) sp TjY;Y   if and ;only if
Y  M (T; ). (c) If dim Y = p, then sp TjY;Y = sp(Z).

Chapter 2 Spectral Approximation

Our main interest lies in determining clusters of spectral values of nite type and the associated spectral subspaces of a given bounded linear operator T . Since exact computations are almost always impossible, we attempt to obtain numerical approximations. Usually these approximations are the exact results of spectral computations on a bounded linear operator Te which is close to the given operator T . Often the operator Te is chosen to be a member of a sequence (Tn ) of bounded linear operators which converges to T pointwise, or with respect to the operator norm, or in a `collectively compact' manner. Other than these three modes of convergence, we study a new mode of convergence which is strong enough to yield the desired spectral results and general enough to be applicable in a number of situations. Under this mode of convergence, properties similar to the upper semicontinuity and the lower semicontinuity of the spectrum are proved. The chapter concludes with error bounds rst for the approximation of a simple eigenvalue and the corresponding eigenvector, and then for the approximation of a cluster of multiple eigenvalues and a basis for the associated spectral subspace.

2.1 Convergence of Operators

In this chapter, T and Tn denote bounded linear operators on a complex Banach space X , that is, T; Tn 2 BL(X ). Unless otherwise mentioned, the convergence is as n ! 1. If (Tn ) converges to T in some sense, the following questions arise naturally: 69

70

2. SPECTRAL APPROXIMATION

1. If n 2 sp(Tn ) and n ! , does  2 sp(T )? 2. If  2 sp(T ), does there exist n 2 sp(Tn ) for each large n such that n ! ? We shall say that under a given mode of convergence, denoted by !, Property U holds if, whenever Tn ! T , n belongs to sp(Tn ) and (n ) converges to , we have  2 sp(T ) ; Property L holds if, whenever Tn ! T and  2 sp(T ), there exists some n belonging to sp(Tn ) for each large enough n such that (n ) converges to .

Let us consider the following property: `Whenever Tn ! T , sup fdist(; sp(T )) :  2 sp(Tn )g ! 0.' It is known as the upper semicontinuity of the spectrum under the given mode of convergence, and Property U is a consequence of it. Property L is known as the lower semicontinuity of the spectrum under the given mode of convergence. It can also be stated as follows: `Whenever Tn ! T and  2 sp(T ), dist(; sp(Tn )) ! 0. The upper semicontinuity of the spectrum and the lower semicontinuity of the spectrum, taken together, give the continuity of the spectrum in the sense of Kuratowski. If Tn ! T ,  is a spectral value of T of nite type, n is a spectral value of Tn of nite type, and (n ) converges to  in Cj , we would like to know how the algebraic multiplicities of the n 's are related to the algebraic multiplicity of  and whether a basis for a spectral subspace of T corresponding to  can be approximated by bases for the corresponding spectral subspaces of Tn . Let us consider three well-known modes of convergence. p The pointwise convergence, denoted by Tn ! T:

kTnx ; Txk ! 0 for every x 2 X: n The norm convergence, denoted by Tn ! T:

kTn ; T k ! 0:

71

2.1. CONVERGENCE OF OPERATORS cc The collectively compact convergence, denoted by Tn ! T: p Tn ! T , and for some positive integer n0 ,

[ f(Tn ; T )x : x 2 X; kxk  1g

nn0

is a relatively compact subset of X . If T is compact, then the latter condition is equivalent to the condition that for some positive integer n0 , the set

[ fT x : nn0 n

x 2 X; kxk  1g

is a relatively compact subset of X . While pointwise convergence and norm convergence are classical concepts, the concept of a `collectively compact' convergence was developed by Atkinson and by Anselone. (See [21] and [17].) p n If Tn ! T or Tn cc! T , then clearly Tn ! T . But the converse is not true. n

cc

= I and Tn ! = I: Example 2.1 In this example, Tn !p I but Tn ! 1 P p Consider X := ` , 1  p < 1. For n = 1; 2; : : : and x := x(k)ek in k=1

X , let

Tn x :=

n X k=1

x(k)ek :

p Then each T is a bounded nite rank operator on X and T ! I . But n n n cc = = Tn ! I since kTn ; I k = 1 for each n, and Tn ! I , since given any positive integer n0 ,

ek 2 f(I ; Tn )x : x 2 X; kxk  1g for k = n0 + 1; n0 + 2; : : : but the sequence (ek ) has no convergent subsequence. If X is an in nite dimensional Banach space, then neither norm convergence nor collectively compact convergence is stronger than the other. n For example, if T := I and Tn := cn I , where cn ! 1 in Cj , then Tn ! T cc = T , unless cn := 1 for all large n. On the other hand, by a result but Tn ! of Josefson and Nissenzweig ([36], Chapter XII), there is a sequence (fn )

72

2. SPECTRAL APPROXIMATION

in X  such that kfnk = 1 for each n and hx ; fn i ! 0 for each x 2 X . For a xed nonzero x0 2 X , let Tn x := hx ; fn ix0 , x 2 X , and T := O. n = T. Then Tn cc ! T , but Tn ! Under additional hypotheses, norm convergence may imply collectively compact convergence, or collectively compact convergence may imply norm convergence. (See Exercise 2.1.) We shall see that under pointwise convergence neither Property U nor Property L holds. On the other hand, we shall prove that under norm convergence Property U holds, and Property L holds `at each isolated point of the spectrum'. However, there are operators of practical interest and there are natural approximations of such an operator T which are not norm convergent. An example of this kind is provided by the Nystrom approximation of a Fredholm integral operator on C 0 ([a; b]) with a continuous kernel. (cf. Proposition 4.6.) Many of these situations are covered by collectively compact convergence. However, compactness is an essential feature of this mode of convergence. This will not be the case of the new mode of convergence which we de ne as follows:  T: The  -convergence, denoted by Tn !

(kTnk) is bounded, k(Tn ; T )T k ! 0 and k(Tn ; T )Tnk ! 0: These conditions on the norms of the operators Tn , (Tn ; T )T and (Tn ; T )Tn have evolved from [1] and [61]. Simple examples show that none of these conditions is implied by  the  0 1 2  1 remaining two. For example, let X := Cj with any norm, A := 0 0   and An := 00 n0 for n = 1; 2; : : : Then k(An ;A)Ak = 0 = k(An ;A)An k, but (kAn k) is unbounded. For other examples, see Exercise 2.18. We shall show that under  -convergence, Property U holds (Corollary 2.7) and Property L holds `at each nonzero isolated point of the spectrum' (Corollary 2.13 and the remark thereafter). In Chapter 3 we shall give several practically useful examples of  -convergence.

Lemma 2.2 n  T . Conversely, if 0 2= sp(T ) and T !  (a) If Tn ! T , then Tn ! n T, n then Tn ! T .

2.1. CONVERGENCE OF OPERATORS

73

n  T + U if and only if (b) Let Tn ! Tn and Un ! U . Then Tn + Un !  n (Tn ; T )U ! O. In particular, (i) if Tn ! T and Un ! O, then   n n Tn + Un ! T , and (ii) if Tn ! O, Un ! U and Tn U ! O; then  U. Tn + Un ! cc  T. (c) If Tn ! T and T is a compact operator, then Tn ! cc n (d) Let Tn; Un 2 BL(X ), Tn ! T , T be a compact operator, Un ! O n  b b = T , then and Tn := Tn + Un . Then Tn ! T . In addition, if Tn ! n cc cc b b = T ; and if Un ! = O, then Tn ! = T. Tn !

Proof n (a) Let Tn ! T . Since kTnk  kTn ; T k + kT k; k(Tn ; T )T k  kTn ; T k kT k and k(Tn ; T )Tnk  kTn; T k kTnk, we see that Tn ! T . Conversely, let 0 2= sp(T ) and Tn ! T . Then T is invertible and n kTn ; T k = k(Tn ; T )TT ;1k  k(Tn ; T )T k kT ;1k, so that Tn ! T. (b) Since kTn + Unk  kTnk + kUnk, we see that the sequence (kTn + n Un k) is bounded. Assume that (Tn ; T )U ! O. As k(Tn + Un ; T ; U )(T + U )k  k(Tn ; T )T k +k(Tn ; T )U k + kUn ; U k kT + U k; k(Tn + Un ; T ; U )(Tn + Un)k  k(Tn ; T )Tnk +k(Tn ; T )Unk + kUn ; U k(kTnk + kUnk); where k(Tn ; T )Unk  kTn ; T k kUn ; U k + k(Tn ; T )U k, we see that  T + U. Tn + Un !  T + U . Since Conversely, assume that Tn + Un ! (Tn ; T )U = (Tn + Un ; T ; U )(T + U ) ;(Tn ; T )T ; (Un ; U )(T + U ); n we obtain (Tn ; T )U ! O.

The particular cases (i) and (ii) follow easily. cc (c) Let Tn ! T . By the Uniform Boundedness Principle (Theorem 9.1 of [55] or Theorem 4.7-3 of [48]), the sequence (kTnk) is bounded and the pointwise convergence of (Tn ) to T is uniform on the relatively compact sets fTx : x 2 X; kxk  1g and n[n fTnx : x 2 X; kxk  1g. 0  T. Hence k(Tn ; T )T k ! 0 and k(Tn ; T )Tnk ! 0. Thus Tn !

74

2. SPECTRAL APPROXIMATION

 T and hence by (b)(i) above, Tb !  (d) By (c) above, Tn ! n T . If n n = T , then Tbn ! = T since kTn ; T k = kTbn ; Un ; T k  kTbn ; T k + kUnk. Tn ! p Finally, we show that if Tbn cc ! T , then Un cc! O. Clearly, Un ! O. Also, for any positive integer n0 , the set F := n[n fUnx : x 2 X; kxk  1g 0 is a subset of Eb ; E , where Eb is the closure of the set n[n fTbnx : x 2 0 X; kxk  1g and E is the closure of the set n[n fTn x : x 2 X; kxk  1g. 0 cc Since Tn ! T and T is a compact operator, the set E is compact for some n0 . If Tbn cc ! T , the set Eb would also be compact for some n0 , implying the compactness of the set Eb ; E and, in turn, the relative

compactness of the set F for some n0 .

We make some remarks on the preceding lemma. Part (a) shows that norm convergence implies  -convergence. Part (c) shows that collectively compact convergence to a compact operator implies  -convergence. On the other hand, if we let X := ` 2 , T := O, 1 P Tn x := x(n + 1)en for n = 1; 2; : : : and x := x(k)ek 2 ` 2 , then p

Tn !

k=1 n cc  = T , Tn ! = T . In this example, each Tn is of T , Tn ! T , but Tn !

nite rank.  T is equivalent to T ! n Part (a) says that Tn ! n T if 0 2= sp(T ). However, we shall be often interested in situations where 0 2 sp(T ). For example, if X is in nite dimensional and T is compact, then 0 2 sp(T ). Part (b) implies that  -convergence is stable under norm perturbations. Part (d) shows how  -convergence can be available even in the absence of norm convergence and collectively compact convergence. For instance, let T be a Fredholm integral operator on C 0 ([a; b]) with a continuous kernel, Tn a Nystrom approximation of T and Un := cn I , where cn ! 0 in Cj . Then the conditions of part (d) are satis ed. In Subsection 4.2.3, we shall consider Fredholm integral operators with `weakly singular' kernels. A modi cation of the Nystrom approximation of such an n  T , but Tb ! operator will then yield a useful example in which Tbn ! n= T cc = T. and Tbn ! n We note that if Tn ! T , or Tn cc! T and each Tn is a compact operator,  T and then T is necessarily compact. On the other hand, when Tn ! each Tn is a compact operator (or even a bounded nite rank operator), the operator T need not be compact. We give a simple example to

2.1. CONVERGENCE OF OPERATORS

75

illustrate this feature.

Example 2.3 In this example, Tn ! T , rank(Tn) < 1, but T is not compact: 1 P Consider 1  p < 1. For x := x(k)ek 2 ` p , let k=1

Tx :=

1 X k=1

x(2k)e2k;1

and for each positive integer n,

Tnx :=

n X k=1

x(2k)e2k;1 :

Clearly, T; Tn 2 BL(` p ), kTnk = 1 and rank(Tn ) = n. Since for all p x2` , (Tn ; T )x = ;

1 X

k=n+1

x(2k)e2k;1 ;

 T . But T is we see that (Tn ; T )T = O = (Tn ; T )Tn. Hence Tn ! not a compact operator. This follows since for the bounded sequence (e2k ) in ` p , the sequence (Te2k ) = (e2k;1 ) does not have a convergent n cc = T and Tn ! = T. subsequence. As a result, we have Tn !

There are several other notions of convergence of a sequence of operators which yield spectral results: compact convergence and regular convergence ([19]), stable and strongly stable convergence ([25]), resolvent operator convergence ([54]), spectral convergence ([4]), convergence of (Tn ) to T in the sense that the spectral radius (Tn ; T ) of Tn ; T tends to zero and k(Tn ; T )Tnk tends to zero ([12]), asymptotically compact convergence ([20]) and also discrete versions of some of these ([68], [40], [69]), to name a few. Our reason for considering the three conditions which comprise  -convergence is twofold. On the one hand, these conditions are general enough to encompass a wide variety of approximation methods; and on the other hand, they are simple enough to be veri ed in practice, as we shall see in Chapter 3. We point out that the operations of addition and composition in BL(X ) are not compatible with  -convergence. (See Exercise 2.3.)

76

2. SPECTRAL APPROXIMATION

Also,  -convergence is a `pseudo-convergence' in the sense that it is  U , where U 6= T . (See Exercise possible to have Tn ! T and Tn !  T and T !  2.4.) However, if Tn ! n U , then sp(U ) = sp(T ) and Ux = Tx whenever x belongs to a spectral subspace of U corresponding to a spectral set not containing 0. (See Exercise 2.12.)

2.2 Property U

p If Tn ! T , then we may have n 2 sp(Tn ) with n ! , but  2= sp(T ).

Example 2.4 Property U does not hold under pointwise convergence: 1 P 2 Consider X := ` . For x := x(k)ek 2 X , let Tx := x(1)e1 and for k=1 each integer n  2, Tn x := x(1)e1 ; x(n)en : p Since kTnx ; Txk2 = jx(n)j ! 0 for every x 2 X , we see that Tn ! T. Now sp(T ) = f0; 1g and sp(Tn ) = f;1; 0; 1g: Since n := ;1 2 sp(Tn ) for each n; but ;1 2= sp(T ), we see that Property U does not hold. Also, if for x 2 X , we let Tn x := x(1)e1 + x(n)en ; p then Tn ! T , 1 is an eigenvalue of Tn of algebraic multiplicity 2, but 1 is an eigenvalue of T of algebraic multiplicity 1. On the other hand, if we let for x 2 X , 1 e := x(1)e1 + x(2)e2 Tx and Ten x := x(1)e1 + n ; n x(2)e2 ; e k = jx(2)j=n ! 0 for every x 2 X , so that then again kTenx ; Tx 2 p e e Tn ! T , 1 is an eigenvalue of each Ten of algebraic multiplicity 1, while 1 is eigenvalue of Te of algebraic multiplicity 2.

We conclude that the algebraic multiplicities are not preserved under pointwise convergence. We now wish to show that Property U holds under  -convergence. For this purpose, we prove some preliminary results.

77

2.2. PROPERTY U

e
) by R(
) For T and Te in BL(X ), we shall denote R(T;
) and R(T; e and R(
), respectively.

Proposition 2.5

Let T and Te be in BL(X ). (a) Second Resolvent Identity: Let z 2 re(T ) \ re(Te). Then

Re(z ) ; R(z ) = Re(z )(T ; Te)R(z ) = R(z )(T ; Te)Re(z ):

(b) Second Neumann Expansion: Let z 2 re(T ) be such that ((T ; Te)R(z )) < 1. Then z 2 re(Te) and Re(z ) = R(z )

1 X k=0

[(T ; Te)R(z )]k :

If in fact k(T ; Te)R(z )k < 1, then

kR(z )k ; 1 ; k(T ; Te)R(z )k e kRe(z ) ; R(z )k  kR(z )k k(T ;e T )R(z )k : 1 ; k(T ; T )R(z )k Also, if k[(T ; Te)R(z )]2 k < 1, then e kRe(z )k  kR(z )k (1 + k(Te ; T )R2(z )k) ; 1 ; k[(T ; T )R(z )] k e e kRe(z ) ; R(z )k  kR(z )k k(T ; T )R(z )k e(1 + k(T2 ; T )R(z )k) : 1 ; k[(T ; T )R(z )] k kRe(z )k 

Proof (a) For z 2 re(T ) \ re(Te), we have Re(z )(T ; Te)R(z ) = Re(z )[(T ; zI ) ; (Te ; zI )]R(z ) = Re(z ) ; R(z ): Interchanging T and Te, we obtain the other equality. (b) For z 2 re(T ), consider the identity

Te ; zI = T ; zI ; (T ; Te) = [I ; (T ; Te)R(z )](T ; zI ):

78

2. SPECTRAL APPROXIMATION

Since ((T ; Te)R(z )) < 1, the operator I ; (T ; Te)R(z ) is invertible. The identity stated above shows that z 2 re(Te) and by Theorem 1.13(d),

Re(z ) = (T ; zI );1 [I ; (T ; Te)R(z )];1 = R(z )

1 X k=0

[(T ; Te)R(z )]k :

Let k(T ; Te)R(z )k < 1. Then ((T ; Te)R(z ))  k(T ; Te)R(z )k < 1 and we have 1 X kR(z )k kRe(z )k  kR(z )k k(T ; Te)R(z )kk = : 1 ; k(T ; Te)R(z )k k=0 Let now k[(T ; Te)R(z )]2 k < 1. Then by Proposition 1.15(b),

((T ; Te)R(z ))  k[(T ; Te)R(z )]2 k1=2 < 1; so that z 2 re(Te) and we have 1 X

Re(z ) = R(z )

j =0

[(T ; Te)R(z )]2j +

= R(z )[I + (T ; Te)R(z )] Hence

1 X

1 h; X j =0

j =0

[(T ; Te)R(z )]2j+1
2 ij

(T ; Te)R(z )



:

kRe(z )k  kR(z )k (1 + k(Te ; T )R2(z )k) : 1 ; k[(T ; T )R(z )] k e

Also, by (a) above,

kRe(z ) ; R(z )k  kRe(z )k k(T ; Te)R(z )k: Thus the desired bounds for kRe(z ) ; R(z )k follow. For Tn in BL(X ), we denote R(Tn;
) by Rn (
).

Theorem 2.6 Let T 2 BL(X ) and E be a nonempty closed subset of re(T ). Then 1 (E ) := supfkR(z )k : z 2 E g < 1:

79

2.2. PROPERTY U

 T , then there is a positive integer n such that E  re(T ) If Tn ! 0 n for all n  n0 and 2 (E ) := sup fkRn (z )k : z 2 E; n  n0 g < 1:

Proof If jz j > kT k, then by Theorem 1.13(d),

kR(z )k  jz j ;1kT k :

Hence kR(z )k ! 0 as jz j ! 1 and there is some  > 0 such that kR(z )k  1 for all z 2 Cj with jz j > . Now E0 := fz 2 E : jz j  g is a compact subset of re(T ) and the function z 7;! kR(z )k is continuous on E0 by Remark 1.14. There is, therefore, some > 0 such that kR(z )k  for all z 2 E0 . Thus 1 (E ) := supfkR(z )k : z 2 E g  maxf1 ; g < 1:

 T. Let Tn ! Case (i): 0 2 E . Since E  re(T ), we see that 0 2= sp(T ). It follows from Lemma 2.2(a) n that Tn ! T . Find n0 such that kTn ; T k < 1=(21(E )) for all n  n0 . Then for z 2 E and n  n0 , k(Tn ; T )R(z )k  kTn ; T k1(E )  12 and by Proposition 2.5(b), z 2 re(Tn ) with kRn(z )k  1 ; k(TkR;(zT)k)R(z )k  21 (E ): n This shows that 2 (E ) := sup fkRn(z )k : z 2 E; n  n0 g  21 (E ) < 1: Case (ii): 0 2= E . As E is a closed subset of Cj , there is some  > 0 such that jz j   for all z 2 E . For z 2 E , we have by Proposition 1.10(e) [(T ; Tn )R(z )]2 = (T ; Tn) TR(zz) ; I (T ; Tn )R(z ) = 1 [(T ; T )TR(z )(T ; T ) ; (T ; T )T + (T ; T )T ] R(z ):

z

n

n

n

n n

80

2. SPECTRAL APPROXIMATION

Since the sequence (kTn k) is bounded, there is some t  0 such that kT ; Tnk  t for all n. As k(T ; Tn )T k ! 0 and k(T ; Tn)Tn k ! 0, nd n0 such that for n  n0 , 

(1 (E )t + 1)k(T ; Tn )T k + k(T ; Tn)Tn k 1 (E )  2 : 

Then for all z 2 E and n  n0 ,

k[(T ; Tn)R(z )]2 k  12 ; and by Proposition 2.5(b), z 2 re(Tn ) with

kRn (z )k  kR(1z;)kk(1((+T k;(TT ;)RT(nz)))R2(kz )k)  21 (E )(1 + 1 (E )t): n

The theorem is proved.

Corollary 2.7  T;  2 sp(T ) Property U holds under  -convergence, that is, if Tn ! n n and n ! , then  2 sp(T ). Proof

Suppose for a moment that  2 re(T ). Since the set re(T ) is open in Cj by Theorem 1.13(a), there is some r > 0 such that E := fz 2 Cj : jz ; j  rg  re(T ). By Theorem 2.6, E  re(Tn ) for all large n. Since n ! , we see that n 2 E  re(Tn ) for all large n, which is contradictory to the hypothesis that n 2 sp(Tn ) for each n. Hence  must belong to sp(T ). Under  -convergence one can in fact prove the upper semicontinuity of each spectral set for T . (See Exercise 2.6.) We note that Corollary 2.7 does not hold if, from the de nition of the  -convergence, one of the two conditions k(Tn ; T )T k ! 0, k(Tn ; T )Tnk ! 0 is omitted or if these two conditions are replaced by the condition k(Tn ; T )2k ! 0. (See Exercise 2.17 and parts (a), (b) and (c) of Exercise 2.18.)

81

2.3. PROPERTY L

2.3 Property L

Some spectral values nof T may not be approximable by spectral values of Tn, even when Tn ! T , as the following example shows.

Example 2.8 Property L does not hold under norm convergence: Let X := ` (ZZ ). For x 2 X , let  6 ;1; (Tx)(k) := x0 (k + 1) ifif kk = = ;1; 2

and for each positive integer n, 8 > <

x(k + 1) if k 6= ;1;

(Tn x)(k) := > x(0) :

n

if k = ;1:

Since kTnx ; Txk = jx(0)j=n for all x 2 X , we see that kTn ; T k = 1=n ! 0. We now show that sp(T ) = f 2 Cj : jj  1g. If jj < 1, consider x 2 X de ned by x (k) := 0 for all k  ;1, x (0) := 1 and x (k) := k for all k  1, and note that Tx = x with x 6= 0. Thus every  satisfying jj < 1 is an eigenvalue and hence a spectral value of T . Since sp(T ) is closed, it follows that f 2 Cj : jj  1g  sp(T ). But since (T )  kT k = 1, we see that sp(T ) is contained in f 2 Cj : jj  1g, and we are through. On the other hand, we claim that sp(Tn ) = f 2 Cj : jj = 1g for each positive integer n. If jj = 1, consider y 2 X de ned by y(k) := 0 for all k 6= ;1 and y(;1) := 1, and note that there is no x 2 X with Tn x ; x = y. Thus every  satisfying jj = 1 is a spectral value of T . Also, since (Tn )  kTnk = 1, 2

f 2 Cj : jj = 1g  sp(Tn )  f 2 Cj : jj  1g: It can be seen that Tn is bijective and for x 2 X , 

; 1) if k 6= 0; (Tn;1 x)(k) = xn(xk(; 1) if k = 0;

82

2. SPECTRAL APPROXIMATION

so that kTn;1k = n. Similarly, for j = 2; 3; : : : and x 2 X , 

0; 1; : : : ; j ; 1; (Tn;j x)(k) = xn(xk(k;;j )j ) ifif kk 6= = 0; 1; : : : ; j ; 1; so that kTn;j k = n for each positive integer j . Hence

(Tn;1 ) = j! lim1 kTn;j k1=j = j! lim1 n1=j = 1:

Thus sp(Tn;1 )  f 2 Cj : jj  1g, sp(Tn )  f 2 Cj : jj  1g, and we are through. It is now clear that if jj < 1, then  2 sp(T ), but there is no n in sp(Tn ) such that n ! . The preceding example shows that the spectrum of an operator can suddenly shrink if the operator is subjected to an arbitrarily small norm perturbation. For examples of very drastic shrinkage, see Exercises 2.9 and 2.10. n We shall prove that (i) if  is an isolated point of sp(T ) and Tn ! T,  or (ii) if  is a nonzero isolated point of sp(T ) and Tn ! T , then there is some n 2 sp(Tn ) for each large n such that n !  (Corollary 2.13). Further, we shall prove that if  is an eigenvalue having nite algebraic multiplicity m, then for each large n, there are a nite number of eigenvalues of Tn near  and the sum of their algebraic multiplicities is m, and that the associated spectral subspaces corresponding to T and Tn are close to each other. In addition, we shall give error estimates for these spectral approximations. With this in mind, we consider a general set-up. Let T 2 BL(X ) and  be a spectral set for T , that is,  is a subset of sp(T ) such that the sets  and sp(T ) n  are also closed in Cj . Then there is a Cauchy contour C 2 C (T; ), that is, C satis es   int(C) and (sp(T ) n )  ext(C). Recall that the spectral projection associated with T and  is given by Z 1 P := P (T; ) := ; 2i R(z ) dz C

and that it does not depend on C 2 C (T; ). Let `(C) denote the length of a Cauchy contour C and

(C) := minfjz j : z 2 Cg:

83

2.3. PROPERTY L

Proposition 2.9

 T. Let  be a spectral set for T , C 2 C (T; ) and Tn !

(a) There is a positive integer n such that for each n  n ; C lies in re(Tn ). Further, if

n := sp(Tn ) \ int(C)

0

0

and

Z

Pn := ; 21 i Rn (z ) dz; C

then n and Pn do not depend on C 2 C (T; ) for each large n. (b) In Theorem 2.6, let E := C. Then for each n  n0, the following estimates hold: `(C)  (C); kP k  `2(C)  1 (C); kPn k   2 2 and kPn ; P k  `2(C)  1 (C)2 (C)kTn ; T k; k(Pn ; P )P k  `2(C)  1 (C)2 (C)k(Tn ; T )P k; k(Pn ; P )Pn k  `2(C)  1 (C)2 (C)k(Tn ; T )Pn k: Further, if 0 2 ext(C), then 1 (C) k(Tn ; T )P k  `2(C)  (C) k(Tn ; T )T k; 2 (C) k(Tn ; T )Pn k  `2(C)  (C) k(Tn ; T )Tnk:

Proof (a) Letting E := C in Theorem 2.6, we see that there exists some n such that C  re(Tn ) for each n  n . Consider another Cauchy contour Ce in C (T; ). Then again there exists some ne such that Ce  re(Tn ) for each n  ne , and we let e : e n := sp(Tn ) \ int(C) We claim that n  e n for all large n. Suppose for a moment that this is not the case. Then there exist positive integers k(1) < k(2) <


and 0

0

0

0

84

2. SPECTRAL APPROXIMATION

e for n = complex numbers k(1) ; k(2) ; : : : such that k(n) 2 k(n) n int(C) 1; 2; : : :. Since k(n) is contained in the compact set int(C) [ C, there are positive integers j (1) < j (2) <


such that the subsequence (k(j(n)) )  T as converges to some  2 int(C) [ C as n ! 1. Since Tk(j(n)) ! n ! 1, we see that  2 sp(T ) by Property U as proved in Corollary 2.7. As C is contained in re(T ), we see that e :  2 sp(T ) \ int(C) =   int(C) e is an open subset of C j containing  and k(j(n)) !  as Since int(C) n ! 1, it follows that  e 6= ;: k(j(1)) ; k(j(2)) ; : : : \ int(C)

This contradicts our choice of k(j(1)) ; k(j(2)) ; : : : Hence n  e n for all e we obtain  e n  n for all large n. Interchanging the roles of C and C, large n. Thus e n = n for all large n. It follows that n is a spectral set for Tn and Ce 2 C (Tn ; n) for all large n, so that Z 1 ; 2i R(ze) dze = P (Tn ; n ) = Pn : e C

(b) Let n  n0. The estimates for kP k and kPnk follow from the de nitions of P and Pn . Next, by the Second Resolvent Identity (Proposition 2.5(a)), Z Pn ; P = ; 21 i [Rn (z ) ; R(z )] dz Z C 1 = 2i Rn (z )(Tn ; T )R(z ) dz ZC 1 = 2i R(z )(Tn ; T )Rn (z ) dz: C Hence the estimate for kPn ; P k also follows easily. As P and R(z ) commute for all z 2 C, Z (Pn ; P )P = 21 i Rn (z )(Tn ; T )PR(z ) dz C and as Pn and Rn (z ) commute for all z 2 C, Z 1 (Pn ; P )Pn = 2i R(z )(Tn ; T )PnRn (z ) dz: C

85

2.3. PROPERTY L

Hence the estimates for k(Pn ; P )P k and k(Pn ; P )Pn k follow. Finally, assume that 0 2 ext(C). Then Z Z P = ; 21 i [TR(z ) ; I ] dzz = ; 2Ti R(z ) dz z; C ZC Z Tn dz Pn = ; 21 i [Tn Rn (z ) ; I ] dz z = ; 2i Rn (z ) z ; C

since

Z

C

C

dz = 0: Hence z Z T ; T ) T R(z ) dz n k(Tn ; T )P k = 2i z C 1 (C)  `2(C)  (C) k(TZn ; T )T k; k(Tn ; T )Pn k = (Tn 2;Ti )Tn Rnz(z ) dz C ` (C)  2 (C)  2 (C) k(Tn ; T )Tnk: (

This completes the proof. In Chapter 3, we shall consider some situations where the rates at which kTn ; T k, k(Tn ; T )T k and k(Tn ; T )Tnk tend to 0 can be estimated. (See Theorem 4.12(a), Example 4.13 and Exercise 4.12.)

Corollary 2.10

Let  be a spectral set for T . With the notation of Proposition 2.9, n n (a) if Tn ! T; then Pn ! P , and

(b) if 0 2=  and Tn ! T , then k(Tn ; T )P k ! 0, k(Tn ; T )Pnk ! 0 and Pn ! P:

Proof (a) Let Tn ! T . The estimate for kPn ; P k given in Proposition 2.9(b) shows that Pn ! P . (b) Let now 0 2=  and Tn ! T . If 0 2= sp(T ), then Tn ! T by Lemma 2.2(a) and hence Pn ! P by (a) above. Thus we are through. n

n

n

n

86

2. SPECTRAL APPROXIMATION

If 0 2 sp(T ), then since C 2 C (T; ) separates the spectral set  from the spectral value 0 of T , we see that 0 2 ext(C); and the estimates for kPn k, k(Tn ; T )P k, k(Tn ; T )Pnk, k(Pn ; P )P k and k(Pn ; P )Pn k given in Proposition 2.9(b) yield the desired results. The following elementary result will be used in the proof of the next theorem.

Lemma 2.11

Let P and Pe be projections in BL(X ) such that (P ; Pe) < 1, Y := R(P ), and Ye := R(Pe). Then the linear map PejY;Ye from Y to Ye is bijective and rank(P ) = rank(Pe).

Proof e = 0. Since (P ; Pe) < 1, we have 1 2 re(P ; Pe), Let y 2 Y with Py that is, the map I ; (P ; Pe) is invertible in BL(X ). In particular, it is e = Py ; Py + 0 = 0. Hence injective. But [I ; (P ; Pe)]y = y ; Py + Py y = 0. Thus the linear map PejY;Ye is injective. Next, consider ye 2 Ye . Again, since (Pe ; P ) < 1, the map I ; (Pe ; P ) is invertible in BL(X ). In particular, it is surjective. Hence there is some x 2 X such that ye = [I ;(Pe;P )]x. But then ye = Peye = Pe[I ;(Pe;P )]x = e ; Px e + PPx e Px = PejY;Ye Px. Thus the map PejY;Ye is surjective. As this map is an isomorphism from Y onto Ye , rank(P ) = dim Y = dim Ye = rank(Pe).

Theorem 2.12

Let  be a spectral set for T . Assume that n  T. (i) Tn ! T or (ii) 0 2=  and Tn ! With the notation of Proposition 2.9, (a) (Pn ; P ) ! 0 and for all large n, rank(Pn ) = rank(P ), where we identify all in nite cardinals, (b)  6= ; if and only if n 6= ; for all large n, and in this case, dist(n ; ) ! 0; so that there are n 2 n; 0n 2  such that n ; 0n ! 0.

87

2.3. PROPERTY L

Proof (a) Since (Pn ; P )  k(Pn ; P ) k =  [k(Pn ; P )Pn k + k(Pn ; P )P k] = ; it follows from Corollary 2.10 that (Pn ; P ) ! 0. Hence (Pn ; P ) < 1 1 2

2 1 2

for all large n, so that rank(Pn ) = rank(P ) by Lemma 2.11. (b) Note that  6= ; if and only if P 6= O, that is, rank(P ) > 0. A similar statement holds for n . Hence by (a) above,  6= ; if and only if n 6= ; for all large n. Let  6= ;. Suppose for a moment that dist(n ; ) 6! 0: Then there is some  > 0 such that dist(k(n) ; )   for some positive integers k(1) < k(2) <


. Since k(n) 6= ;, let k(n) 2 k(n) be such that 1 dist(k(n) ; ) = dist(k(n) ; ). As n[=1 k(n)  int(C) [ C, and int(C) [ C is a compact subset of Cj , there exist positive integers j (1) < j (2) <


such that (k(j(n)) ) converges to some  2 int(C) [ C. By Property U proved in Corollary 2.7,  2 sp(T ). Thus  2 sp(T ) \ (int(C) [ C) = . But as dist(k(j(n)) ; )   for all n, we have  2= . This contradiction shows that dist(n ; ) ! 0: Now since  and n are nonempty compact subsets of Cj , there are n 2 n and 0n 2  such that jn ; 0(n) j = dist(n ; ), so that jn ; 0n j ! 0:

Corollary 2.13

Let  be an isolated point of sp(T ). Assume that n (i) Tn ! T or

 T. (ii)  6= 0 and Tn !

For each positive  < dist(; sp(T ) n fg), let

n := fn 2 sp(Tn) : jn ; j < g: Then for all large n; n 6= ; and if n 2 n, the sequence (n ) converges to . If the algebraic multiplicity of  is nite, then for each large n, every n in n is an eigenvalue of Tn of nite algebraic multiplicity; and the sum of the algebraic multiplicities of the eigenvalues of Tn in n is equal to the algebraic multiplicity of .

88

2. SPECTRAL APPROXIMATION

Proof

In Proposition 2.9, let  := fg and C be the positively oriented circle with center  and radius . Then by Theorem 2.12(b), we obtain n 6= ; for all large n. Let n 2 n . We prove that n ! . Since the sequence (n ) lies in the compact set E := fz 2 Cj : jz ; j  g, it is enough to show that every convergent subsequence of (n ) converges to  itself. Let a subsequence (k(n) ) converge to e 2 Cj . By Property U proved in Corollary 2.7, we see that e 2 sp(T ). But e 2 E and sp(T ) \ E = fg. Hence e = . Thus n ! . Now let the algebraic multiplicity of  be nite. By Theorem 2.12(a), rank P (Tn ; n ) = rank P (T; fg) for all large n, so that n is a spectral set of nite type for Tn . Theorem 1.32 shows that every n in n is an eigenvalue of Tn of nite algebraic multiplicity, and their sum equals the algebraic multiplicity of . The preceding result allows us to say the following: Under norm convergence, Property L holds at each isolated point of the spectrum; and under  -convergence, Property L holds at each nonzero isolated point of the spectrum. It would be interesting to know whether under  -convergence Property L holds at 0 whenever 0 is an isolated point of the spectrum.

Example 2.14 Under the pointwise convergence, Property L does not hold even at a nonzero isolated point of the spectrum: Let X := ` 1 . For x 2 X , let 8 <

x(1) if k = 1, (Tx)(k) := : 2x(k) if k 6= 1 and k is odd, 0

otherwise.

Then T 2 BL(X ) and sp(T ) = f0; 1; 2g. For each positive integer n and x 2 X , let 8 (1) + > > > > 2 () > > <

x x(2n) xk (Tn x)(k) := > x(k) + x(k +x2)(1) > x(4n ; 4) ; n > > > > :0

if k = 1, if k 6= 1 and k is odd, if 2n  k  4n ; 6 and k is even, if k = 4n ; 4, otherwise.

89

2.3. PROPERTY L

For x 2 X , we have 8 > > > <

x(2n) if k = 1, x(k) + x(k + 2) if 2n  k  4n ; 6 and k is even, (Tn x ; Tx)(k) = > x(4n ; 4) ; x(1) > > n if k = 4n ; 4, :

0

otherwise,

so that

n; j +2 X jx(2k)j ! 0 as n ! 1: k(Tn ; T )xk  jx(1) n k n 2

2

1

=

Thus Tn ! T . Also, as k(Tn ; T )Txk = jx(1)j=n, we have k(Tn ; T )T k  1=n ! 0: For each integer n > 1 and x 2 X , consider 8 < x(1) if k = 1, (n x)(k) := : x(k) if 2n  k  4n ; 4 and k is even, 0 otherwise. It can be easily seen that Tn ; n 2 BL(X ) and n2 = n . Let Yn := R(n ) =; spanfe1 ; e2n ; e2n+2 ; : : : ; e4n;4 g and Zn := N (n ), the closure of span fei :
2  i  2n ; 1g [ fe2n+2j+1 : 0  j  n ; 2g [ fek : k > 4n ; 4g . Since the closed subspaces Yn and Zn are invariant under Tn , we see that n Tn = Tn n . Hence by Proposition 1.18, sp(Tn ) = sp(TnjYn ;Yn ) [ sp(TnjZn;Zn ): Consider the nn matrix An representing the operator TnjYn;Yn with respect to the standard basis: 2 3 1 1 0





0 6 0 1 1 0


0 77 6 6 .. . . . . . . . . .. 77 . . . . . .7: An = 666 0


0 1 1 0 77 6 4 0





0 1 15 ;1=n 0


0 0 1 Developing det(An ; z I) by the rst column, we obtain det(A ; z I) = (1 ; z )n ; (;1)n;1 1 : p

1

n

n

90

2. SPECTRAL APPROXIMATION

Hence sp(TnjYn;Yn ) = sp(An )  fz 2 Cj : j1 ; z j = n;1=n g. Also, since Tn x = Tx for every x 2 Zn, we see that sp(TnjZn;Zn ) = sp(TjZn;Zn ) = f0; 2g: Thus j1 ; n j  n;1=n  e;1=e for every n 2 sp(Tn ), and hence the isolated spectral value 1 of T cannot be approximated by any of the spectral values of Tn . Another example of this kind is given in Exercise 2.15. We remark that the situation described in the preceding example is not possible if each Tn is a normal operator on a Hilbert space. (See Exercise 2.16.) We also remark that Corollary 2.13 does not hold if, from the de nition of  -convergence, one of the two conditions k(Tn ; T )T k ! 0, k(Tn ; T )Tnk ! 0 is omitted or if these two conditions are replaced by the condition k(Tn ; T )2 k ! 0. (See Example 2.14, Exercise 2.17 and parts (b), (c) of Exercise 2.18.) We note that under the assumptions of Corollary 2.13, for any eigenvalue of T of nite algebraic multiplicity, there is an eigenvalue n of Tn such that n ! . This statement may not hold if the algebraic multiplicity of  is in nite. For example, let X := ` 2 and (ek ) be its standard 1 1 P P orthonormal basis. For x := x(k)ek 2 X , de ne Tx := x(k)ek 1 P

k=1

k=2

and Tn x := (x(k) + x(k ; 1)=n)ek . Then kTn ; T k = 1=n ! 0; k=2 sp(T ) = f0; 1g, 1 is an eigenvalue of T , but the only eigenvalue of Tn is 0 for n = 1; 2; : : :

2.4 Error Estimates

Let  be a spectral set for T and consider C 2 C (T; ). Assume that  T . By Proposition 2.9(a), there is an integer n such that  := Tn ! 0 n sp(Tn ) \ int(C) is a spectral set for Tn for each n  n0 . Theorem 2.6 shows that

1 (C) := supfkR(z )k : z 2 Cg < 1; 2 (C) := supfkRn(z )k : z 2 C; n  n0 g < 1;

91

2.4. ERROR ESTIMATES

where, as usual, R(z ) := R(T; z ) for z 2 re(T ), and Rn (z ) := R(Tn ; z ) for z 2 re(Tn ). Let P := P (T; ) and for n  n0 , Pn := P (Tn ; n ) denote the spectral projections. Also, let M := M (T; ) and for n  n0 , Mn := M (Tn; n ) denote the spectral subspaces, and

e1 (C) := supfkR(z )jM k : z 2 Cg; e2 (C) := supfkRn(z )jMn k : z 2 C; n  n0 g: Clearly, e1 (C)  1 (C) and e2 (C)  2 (C). In case m := rank P < 1, let ' form an ordered basis for M , 'n form an ordered basis for Mn , and T ' = ' , Tn 'n = 'n n, where ; n 2 Cj mm . Since sp() = sp(TjM;M ) = , sp(n ) = sp(Tn jMn ;Mn ) = n and  [ n  int(C), we have C  re() \ re(n ). De ne

1 (C) := supfkR(; z )k : z 2 Cg; 2;n (C) := supfkR(n; z )k : z 2 Cg: Clearly, 1 (C) is nite and for each large n, 2;n (C) is nite. Recall the notation x := [x1 ; : : : ; xm ] for x1 ; : : : ; xm 2 X and

k x k1 := max fkx k; : : : ; kxm kg 1

from Subsection 1.4.2. We shall use the above-mentioned notation throughout this section. The following result is crucial for the error estimates we shall give.

Proposition 2.15

Let  be a spectral set for T . Assume that n (i) Tn ! T or

Then

 T. (ii) 0 2=  and Tn !

k(Tn ; T )jM k ! 0 and k(Tn ; T )jM k ! 0: n

Also, for all large n,

k(Pn ; P )jM k  `2(C)  e (C) (C)k(Tn ; T )jM k; 1

2

k(Pn ; P )jM k  `2(C)   (C)e (C)k(Tn ; T )jM k: n

1

2

n

92

2. SPECTRAL APPROXIMATION

Further, if rank P is nite, then k( Pn ; P ) ' k1  `2(C)  1 (C)2 (C)k( Tn ; T ) ' k1 ; k( Pn ; P ) 'n k1  `2(C)  1 (C) 2;n (C)k( Tn ; T ) 'n k1 :

Proof If Tn ! T , then k(Tn ; T )jM k  kTn ; T k ! 0 and k(Tn ; T )jM k   T . If 0 2= sp(T ), then by kTn ; T k ! 0. Let now 0 2=  and Tn ! Lemma 2.2(a), Tn ! T . If 0 2 sp(T ), then 0 2 ext(C), and Proposition n

n

n

2.9(b) shows that

k(Tn ; T )jM k  k(Tn ; T )P k ! 0; k(Tn ; T )jM k  k(Tn ; T )Pn k ! 0: n

Now, for each large n, we have Z   (Pn ; P )jM = 21 i Rn (z )(Tn ; T )R(z ) dz C M Z 1 = 2i Rn (z )(Tn ; T )jM R(z )jM dz; C since R(z )(M )  M for all z 2 C. Hence k(Pn ; P )jM k  `2(C)  2 (C)k(Tn ; T )jM ke1 (C): Similarly, as Rn (z )(Mn)  Mn for all z 2 C, we obtain Z (Pn ; P )jMn = 21 i R(z )(Tn ; T )jMn Rn (z )jMn dz; C so that k(Pn ; P )jMn k  `2(C)  1 (C)k(Tn ; T )jMn ke2 (C): Next, for z 2 C, we have R(z ) ' = ' R(; z ) and Rn (z ) 'n = 'n R(n; z ), so that Z 1 ( Pn ; P ) ' = 2i Rn (z ) ( Tn ; T ) ' R(; z ) dz; ZC 1 ( Pn ; P ) 'n = 2i R(z ) ( Tn ; T ) 'n R(n ; z ) dz: C

93

2.4. ERROR ESTIMATES

Hence the desired estimates for k( Pn ; P ) ' k1 and k( Pn ; P ) 'n k1 follow.

Remark 2.16

The preceding proposition allows us to consider the nearness of the spectral subspaces Mn and M . For this purpose we introduce the notion of gap between two closed subspaces Y and Ye of a Banach space X . Let n

o

(Y; Ye ) := sup dist(y; Ye ) : y 2 Y; kyk = 1 ; n

o

gap(Y; Ye ) := max (Y; Ye ) ; (Ye ; Y ) :

Y

6 y

Ye ye

kyk = 1 = kyek

-

The following properties can be proved easily. (i) 0  gap(Y; Ye )  1. (ii) gap(Y; Ye ) = 0 if and only if Y = Ye . (iii) gap(Y; Ye ) = gap(Ye ; Y ). However, the triangle inequality does not hold, as Exercise 2.21 shows. Let P and Pe be projections in BL(X ) such that R(P ) = Y and R(Pe) = Ye . Then for every y 2 Y with kyk = 1, e k = kPy ; Py e k  k(P ; Pe )jY k; dist(y; Ye )  ky ; Py

94

2. SPECTRAL APPROXIMATION

so that (Y; Ye )  k(P ; Pe)jY k. Hence n

o

gap(Y; Ye )  max k(P ; Pe)jY k; k(P ; Pe)jYe k : Under the hypothesis of Proposition 2.15, we have 



gap(M; Mn )  max k(Pn ; P )jM k; k(Pn ; P )jMn k   `2(C)  1 (C)2 (C) max k(Tn ; T )jM k; k(Tn ; T )jMn k  which tends to zero as n ! 1. Let now the spectral set  be of nite type for T . First we consider the case when  consists of a single simple eigenvalue of T .

Theorem 2.17

Let  be a simple eigenvalue of T ,  such that 0 <  < dist(; sp(T ) nfg) and C the positively oriented circle with center  and radius . Assume that n (i) Tn ! T or

 T. (ii)  6= 0 and Tn !

(a) There is a positive integer n such that for each n  n , we have a unique n 2 sp(Tn ) satisfying jn ; j < . Also, n is a simple eigenvalue of Tn for each n  n and n ! . (b) Let ' be an eigenvector of T corresponding to  and for n  n ; 0

0

0

0

let 'n be an eigenvector of Tn corresponding to n . Then for each large n, P'n is an eigenvector of T corresponding to , Pn ' is an eigenvector of Tn corresponding to n , and 

 k ( T ; T ) ' k k ( T ; T ) ' k n n n jn ; j  2 min  (C ) k' k ;  (C ) k'k ; n 1

2

which tends to zero. Also, for all large n,

  (C ) k(T ; T )' k: k'n ; P'n k  dist( n  ;C ) n 1

n



If the sequence (k'n k) is bounded, then k(Tn ; T )'nk ! 0 and k'n ; P'n k ! 0.

95

2.4. ERROR ESTIMATES

For each large n, let cn be the nonzero complex number such that Pn ' = cn 'n . Then kcn'n ; 'k ! 0: Also, if ' n := '=cn , then ' n is the eigenvector of T corresponding to  such that Pn ' n = 'n . We have k' n k  2k'n k for all large n and k'n ; ' n k   (C ) k(Tn ; T )'k : 2  k' n k k'k In fact, if 'n is the eigenvector of Tn corresponding to its (simple) eigenvalue n such that h'n ; 'n i = 1, then cn := h' ; 'n i. ( )

( )

( )

( )

( )

( )

Proof (a) As the algebraic multiplicity of the spectral value  of T is 1, the

results follow from Corollary 2.13. (b) Since for all large n, dim M = 1 = dim Mn, we see that M = N (T ; I ) and Mn = N (Tn ; n I ) by Proposition 1.31(b). Theorem 2.12(a) and Lemma 2.11 show that for all large n; the linear map Qn := PjMn ;M is bijective. In particular, P'n is a nonzero element of N (T ; I ), that is, P'n is an eigenvector of T corresponding to . Also, by Corollary 2.10, k(Pn ; P )Pn k  1=2 for all large n, and hence for x 2 Mn ,

kxk ; kPxk  kx ; Pxk = k(Pn ; P )Pn xk  kx2k :

Thus kQ;n 1k  2 for all large n. Noting that Q;n 1 P'n = 'n , P 2 = P and PT = P; we obtain

n 'n ; 'n = Q;n 1 P (Tn 'n ; PT'n) = Q;n 1 P (Tn ; T )'n : Hence for all large n, kn 'n ; 'n k  2kP k k(Tn ; T )'n k, so that jn ; j  2 1(C ) k(Tnk;' Tk)'n k : n Also, considering n := [n ] 2 Cj 11 , we see that (C ) = sup 1 2

;n 

z2C jn ; z j

96

2. SPECTRAL APPROXIMATION

and hence by Proposition 2.15, we obtain k'n ; P'n k = k(Pn ; P )'n k 1 k(T ; T )' k )  `(C n n 2 1 (C ) zsup 2C jn ; z j  1 (C ) k(T ; T )' k: = dist( n n ; C ) n Since n ! , we have dist(n ; C )  =2 for all large n. If k'n k  c, then by Proposition 2.15, k(Tn ; T )'n k  ck(Tn ; T )jMn k ! 0 and hence k'n ; P'n k ! 0. By interchanging the roles of T and Tn , we see that for all large n, the map Pn jM;Mn : M ! Mn is bijective, Pn ' is an eigenvector of Tn  ;1 corresponding to n ,

Pn jM;Mn

 2, and

jn ; j  2  (C ) k(Tnk;'kT )'k : 2

Since dim Mn = 1, we have Pn ' = cn 'n for some nonzero scalar cn . Also, kcn 'n ; 'k = kPn ' ; P'k = k(Pn ; P )P'k  k(Pn ; P )P k k'k ! 0 by Corollary 2.10. Since ' n := '=cn, we see that Pn ' n = Pn '=cn = 'n . As Pn jM;Mn is injective, ' n is the unique element of M such that Pn ' n = 'n ; and we have k' n k  2k'n k. By considering  := [] 2 Cj 11 , we see that 1 (C ) = sup j ;1 z j ( )

( )

( )

( )

( )

z2C

and hence by Proposition 2.15, we obtain k' ; ' k = 1 k(P ; P )'k n

n

( )

jcn j

n

 2`(Cjc)j  (C ) sup j ;1 z j k(Tn ; T )'k 2

n z2C  2 (C ) = jc j k(Tn ; T )'k n = 2 (C ) k(Tnk;'kT )'k k'(n) k;

97

2.4. ERROR ESTIMATES

so that

k'n ; ' n k   (C ) k(Tn ; T )'k :  k' n k k'k ( )

( )

2

Since the subspaces M , Mn are one dimensional, and since 0 6= ' 2 M , 0 6= 'n 2 Mn , we have k(T ; T ) k = k(Tn ; T )'k ; k(T ; T ) k = k(Tn ; T )'n k ; n

jM

k'k

n

jMn

k'n k

both of which tend to zero by Proposition 2.15. Proposition 1.36(b) shows that n is a simple eigenvalue of Tn and  Pn is the associated spectral projection. By Lemma 1.51, there is an eigenvector 'n of Tn corresponding to n such that h'n ; 'n i = 1. Then cn = hcn 'n ; 'n i = hPn ' ; 'n i = h' ; Pn 'n i = h' ; 'n i: The proof is now complete. We now take up the general case of a spectral set of nite type for T . As one may expect, the general case is much more complex as compared to the case of a single simple eigenvalue of T considered above; but the arguments are similar.

Theorem 2.18 Let  := f ; : : : ; r g, where each j is a spectral value of T of nite type. Let mj be the algebraic multiplicity of j , m := m +


+ mr . Assume 1

1

that

n  T. (i) Tn ! T or (ii) each j 6= 0 and Tn ! (a) There is a positive integer n0 such that for all n  n0, C  re(Tn) and if n := sp(Tn ) \ int(C), then n = fn;1 ; : : : ; n;r(n) g; where each n;j is spectral value of Tn of nite type. Further, if mn;j is the algebraic multiplicity of n;j , then mn;1 +


+mn;r(n) = m. (b) For n  n0, consider the weighted arithmetic means b := m1 1 +


+ mr r

m

98

2. SPECTRAL APPROXIMATION and

m  +


+ mn;r(n) n;r(n) bn := n;1 n;1 m

of elements in  and n , respectively. Then  jbn ; bj  `(C) min  1 (C)k(Tn ; T )jMn k; 2 (C)k(Tn ; T )jM k  ! 0:

(c) For each large n, P 'n forms an ordered basis for M and k 'n ; P 'n k1 `(C)  2  (C)e (C)k(Tn ; T )jM k ! 0: k 'n k1 1

2

n

For each large n, Pn ' forms an ordered basis for Mn and there is an mm nonsingular matrix Cn such that Pn ' = 'n Cn . Then

k 'n Cn ; ' k1 ! 0:

Also, if ' n := ' C;n 1 , then ' n is the unique ordered basis for M such that Pn ' n = 'n . We have k ' n k1  2k 'n k1 for all large n and ( )

( )

( )

( )

k 'n ; ' n k1 `(C) k ' n k1  2 e (C) (C)k(Tn ; T )jM k ! 0: In fact, if 'n forms the ordered basis for R(Pn ) which is adjoint ( )

( )

1

2

to 'n , then Cn := ' ; 'n . (d) For n  n0, let n 2 Mn such that (k nk) is bounded. Then the sequence ( n ) has a convergent subsequence, and every convergent subsequence of ( n ) converges to an element of M .

Proof (a) Since each j is an isolated point of sp(T ),  := f ; : : : ; r g is a 1

spectral set for T . As we have seen in Theorem 1.32, rank(P ) = m, the sum of the algebraic multiplicities of 1 ; : : : ; r . By Theorem 2.12(a), there is a positive integer n0 such that rank(Pn ) = rank(P ) = m for all n  n0 . Hence again by Theorem 1.32, n := sp(Tn ) \ int(C) consists of a nite number of spectral values n;1 ; : : : ; n;r(n) of Tn of nite type, and the sum of their respective algebraic multiplicities mn;1 ; : : : ; mn;r(n) is m.

99

2.4. ERROR ESTIMATES

(b) Again, by Theorem 2.12(a), (Pn ; P ) < 1 for all large n, and hence by Lemma 2.11, the linear map Pn jM;Mn : M ! Mn is bijective. Also, by Corollary 2.10, k(Pn ; P )P k  1=2 for all large n, and so for x 2 M, kxk ; kPn xk  kx ; Pn xk = k(P ; Pn )Pxk  kx2k : Thus k(Pn jM;M ); k  2 for all large n. n

1

Let us de ne A := TjM;M and for each large n, An := Tn jMn ;Mn , A0n := (Pn jM;Mn );1 Tn(Pn jM;Mn ): By the Spectral Decomposition Theorem 1.26, sp(A) =  and the algebraic multiplicity of j 2 sp(A) is mj , j = 1; : : : ; r. Similarly, sp(An ) = n and the algebraic multiplicity of n;j 2 sp(An ) is mn;j , j = 1; : : : ; r(n). The matrices  and n represent the linear maps A and An with respect to the ordered basis ' for M and 'n for Mn , respectively. Let the matrix 0n represent the linear map A0n with respect to the ordered basis ' for M . Since A0n = (PnjM;Mn );1 An (PnjM;Mn ), the matrices n and 0n are similar. Hence it can be seen that sp(0n ) = sp(n ) and that the algebraic multiplicity of n;j as an eigenvalue of 0n is equal to the algebraic multiplicity of n;j as an eigenvalue of n , namely mn;j , j = 1; : : : ; r(n). Thus b := m1 1 +


+ mr r = tr () ;

m m m tr (n ) = tr (0n ) n; 1 n;1 +


+ mn;r (n) n;r (n) bn := = m m m for all large n. Consequently,

0n ; )j jbn ; bj = j tr(m  (0n ; ) = (A0n ; A)  kA0n; Ak = sup k(Pn jM;M ); Pn (Tn ; T )x : x 2 M; kxk  1  k(Pn jM;M ); k kPn k k(Tn ; T )jM k  `(C)   (C)k(Tn ; T )jM k ! 0 n

1

1

n

2

by Proposition 2.15. Interchanging the roles of T and Tn , we obtain jbn ; bj  `(C)  1 (C)k(Tn ; T )jMn k ! 0: Hence  jb ; bj  `(C) min  (C)k(T ; T ) k ;  (C)k(T ; T ) k : n



1

n

jMn

2

n

jM

100

2. SPECTRAL APPROXIMATION

(c) Since the linear map PjMn;M is bijective for all large n, P 'n forms an ordered basis for M . Also, by Proposition 2.15, k 'n ; P 'n k1 = k( Pn ; P ) 'n k1  k(Pn ; P )jM k k 'n k1  `2(C)   (C)e (C)k(Tn ; T )jM k k 'n k1 : n

1

2

n

Hence by Proposition 2.15,

k 'n ; P 'n k1 `(C)  2  (C)e (C)k(Tn ; T )jM k ! 0: k 'n k1 1

2

n

Since the linear map Pn jM;Mn is bijective for all large n, Pn ' forms an ordered basis for Mn . Hence Pn ' = 'n Cn for some nonsingular mm matrix Cn . Then ;




k 'n Cn ; ' k1 = k Pn ' ; P ' k1 = k Pn ; P P ' k1  k(Pn ; P )P k k ' k1 ! 0 by Corollary 2.10. Also, since ' n := ' Cn;1 , ' n is the unique ordered basis for M such that Pn ' n = 'n . We have k ' n k1  2k 'n k1 and ( )

( )

( )

( )

k 'n ; ' n k1 = k( Pn ; P ) ' n k1  k(Pn ; P )jM k k ' n k1 : ( )

( )

( )

Hence by Proposition 2.15,

k 'n ; ' n k1 `(C) k ' n k1  2 e (C) (C)k(Tn ; T )jM k ! 0: Let 'n form the ordered basis for R(Pn ) which is adjoint to 'n . Note that R(Pn ) is the spectral subspace associated with Tn and its ( )

( )

1

2

spectral set n . Then

Cn = 'n Cn ; 'n = Pn ' ; 'n = ' ; Pn 'n = ' ; 'n :

(d) Let ( n ) be a bounded sequence in X with n 2 Mn for all large n. Then as in (c) above, k n ; P n k  `2(C)  1 (C)e2 (C)k(Tn ; T )jMn k k nk ! 0:

101

2.4. ERROR ESTIMATES

Consider a subsequence ( k(n) ) of ( n ). The estimate given above shows that ( k(n) ) converges to 2 X if and only if (P k(n) ) converges to 2 X . In that event, 



P = P nlim !1 k(n) = nlim !1 P k(n) = ; so that 2 M . Since P 2 BL(X ) and rank(P ) is nite, P is a compact operator on X . Therefore the bounded sequence ( n ) does have a subsequence ( k(n) ) such that (P k(n) ) converges in X . We make some remarks about the conclusions of Theorem 2.18. Firstly, the weighted arithmetic mean b of a cluster  of spectral values of T of nite type is well approximated by the weighted arithmetic mean bn of a cluster n of spectral values of Tn of nite type. When  consists of a single multiple spectral value  of T (so that b = ), the arithmetic mean bn of n provides, in general, a better approximation of  as compared to the individual elements of n. In fact, if n := fn;1 ; : : : ; n;r(n) g and ` is the ascent of , then it can be proved that for all large n and j = 1; : : : ; r(n), 



jn;j ; j`  c min k(Tn ; T )jM k ; k(Tn ; T )j k ; n

M

where c is a constant independent of n. See [62] and [30] for the details. Secondly, let ( n ) be a bounded sequence in X with n 2 Mn . Part (d) guarantees that ( n ) has a subsequence which converges to an element of M . If the given sequence ( n ) is also bounded away from zero, that is, k nk   for all large enough n and some constant  > 0, then the limit of this convergent subsequence will be a nonzero element of M , something that we are looking for! Consider the special case where  := fg. Then b =  and for n 2 sp(Tn ), we see that n !  if and only if n 2 n for all large n by Corollary 2.13. Let n 2 n and n be a corresponding generalized eigenvector of Tn of grade at most p, that is, 0 6= n 2 N [(Tn ; n I )p ] : Assume that 0 <   k n k  for all large n. Then there is a subsequence ( k(n) ) which converges to a nonzero element of M . We show that is in fact a generalized eigenvector of T corresponding to  and its grade is at most p, that is, 2 N [(T ; I )p ].

102

2. SPECTRAL APPROXIMATION

Since k(Tn ; T )P k ! 0, Tn x ! Tx for each x 2 M as n ! 1. Now for each j = 1; : : : ; p,

Tnj ; T j = Tnj ; Tnj;1 T + Tnj;1 T ; Tnj;2 T 2 + Tnj;2 T 2 +


+ Tn T j;1 ; T j =

j X i=1

Tnj;i (Tn ; T )T i;1:

As 2 M and M is invariant under T , it follows that Tnj ! T j for each j = 1; : : : ; p. But

kTkj n

( )

k(n) ; T j

k  kTkj n k k ( )

k(n) ;

k + kTkj n ; T j k ! 0; ( )

so that Tkj(n) k(n) ! T j as n ! 1. Also, k(n) !  as n ! 1. Hence ;
p (T ; I )p = nlim T k (n) ; k(n) I k(n) = 0; !1 as desired. It is signi cant to observe that, when n 2 Mn and 0 <   k n k  for some constants  and , the entire sequence ( n ) may not converge, although it has a convergent subsequence. In the following simple example, the sequence (cn n ) does not converge in X for any scalars c1 ; c2 ; : : : except when cn ! 0.

Example 2.19 Only a subsequence of approximate eigenvectors may converge: Let X := Cj  , the linear space of all 21 matrices  with complex  2 1

1 1=n . Then entries, A := I and for each positive integer n, An := 1=n 1  := 1 is the only spectral value of A and its algebraic multiplicity is 2. Also, n;1 := 1 + 1=n and n;2 := 1 ; 1=n are the only spectral values of An , each having algebraic multiplicity equal to 1. Further, >

n;1 := [1; 1]

and

>

n;2 := [1; ;1]

are eigenvectors of An corresponding to n;1 and n;2 , respectively, and Mn = spanf n;1 ; n;2 g. Let n := [1; (;1)n ]> . Then n 2 Mn and the sequence ( n ) is bounded as well as bounded away from zero. The subsequence ( 2n ) of ( n ) converges to the eigenvector [1; 1]> of A. Also, the subsequence ( 2n+1 ) of ( n ) converges to the eigenvector [1; ;1]>

103

2.4. ERROR ESTIMATES

of A. However, if (cn ) is any sequence in Cj which does not converge to 0, then the entire sequence (cn n ) does not converge in X . Before we conclude this section, we seek estimates for k 'n ; P 'n k1 k 'n ; ' n k and k ' k 1 in terms of k( Tn ; T ) 'n k1 and k( Tn ; T ) ' k1 , n 1 respectively. We note that this was easily accomplished when dim M = 1 in Theorem 2.17. For the general case, we need a result proved by Kato in [46] which says that if a sequence of the norms of mm matrices is bounded and if all the eigenvalues of these matrices are bounded away from zero, then the sequence of the norms of the inverses of these matrices is also bounded. This result will also be useful when we consider uniformly well-conditioned bases in Chapter 3. ( )

( )

Lemma 2.20 Let Z 2 Cj mm be a nonsingular matrix. Then km; . (a) kZ; k  kjZdet Zj (b) Let  := minfjj :  2 sp(Z)g. Then 1

2

2

1

m=2

kZ; k  mm kZkm; : 1

1

1

1

Proof (a) Let Z := QP, where Q is a unitary matrix and P is a Hermitian positive de nite matrix. (See Remark 1.3.) Since Q and Q; are unitary, and Z; = P; Q; , we see that kZk = kPk ; kZ; k = kP; k and j det Zj = det P. Hence we can assume without loss of generality that the matrix Z itself is Hermitian positive de nite. Let  


 m be 1

1

1

1

2

1

2

1 > 0; kZk = m ; kZ;1 k = 1 1 2

and j det Zj = 1


m :

2

m; km; : kZ; k = 1 = 





m  (m

)
 = kjZdet Zj m m 1

1

2

2

1

2

1

the eigenvalues of Z, so that

Thus

1

2

1

1

2

1

104

2. SPECTRAL APPROXIMATION

p (b) The inequality in (a), and the inequalities kZ; k  m kZ; k , p kZk  m kZk , and j det Zj = j j


jm j  m imply pm kZkm; p p m; m= ; kZ k  j det Zj  m ( mmkZk ) = mm kZkm; : 1

2

2

1

1

1

1

1

2

1

1

1

1

2

1

1

This completes the proof.

Proposition 2.21

Under the hypotheses of Theorem 2.18, let ' := ['1 ; : : : ; 'm ] and for each large n, 'n := ['n;1 ; : : : ; 'n;m ]. De ne  := j=1min dist('j ; spanf'i : i 6= j; i = 1; : : : ; mg) > 0; ;:::;m n := j=1min dist('n;j ; spanf'n;i : i 6= j; i = 1; : : : ; mg) > 0: ;:::;m Let ' n be as in Theorem 2.18(c). Then for all large n, ( )

k 'n ; ' n k1 `(C) m k ' n k1  2  (C)  (C) k( Tn ; T ) ' k1 ; ( )

1

( )

2

which tends to zero, and

k 'n ; P 'n k1  `2(C)   (C) ;n (C) k( Tn ; T ) 'n k1 : If the sequence (k 'n k1 ) is bounded, then k( Tn ; T ) 'n k1 tends to 1

2

zero. If, in addition, the sequence (n ) is bounded away from zero, then the sequence ( 2;n (C)) is bounded and consequently k 'n ; P 'n k1 tends to zero.

Proof

As Pn ' n = 'n and ' n = ' C;n 1 , Proposition 2.15 shows that for all large n, k 'n ; ' n k1 = k( Pn ; P ) ' C;n 1 k1 ;1  `2(C)  1 (C) 2 (C) k( Tn ; T ) ' k1 kCn k : Let ' n := ['(n);1 ; : : : ; '(n);m ] and c0n;i;j denote the (i; j )th entry of the matrix C;n 1 for 1  i; j  m. Then for each j = 1; : : : ; m, we have '(n);j = c0n;1;j '1 + : : : + c0n;m;j 'm ; ( )

( )

( )

1

( )

105

2.4. ERROR ESTIMATES

so that for i = 1; : : : ; m,

jc0n;i;j j  k'(n);j k  k ' n k1 : ( )

It follows that

kC;n k = j max ;:::;m 1

1

=1

m X i=1

jc0n;i;j j  m k ' n k1 : ( )

Thus

k 'n ; ' n k1 `(C) m k ' n k1  2  (C)  (C) k( Tn ; T ) ' k1 ; where k( Tn ; T ) ' k1  k(Tn ; T )jM k k ' k1 ! 0 by Proposition 2.15. ( )

1

( )

2

Again by Proposition 2.15, we have for all large n,

k 'n ; P 'n k1 = k( Pn ; P ) 'n k1  `2(C)   (C) ;n (C) k( Tn ; T ) 'n k1 : 1

2

Assume that the sequence (k 'n k1 ) is bounded. Then

k( Tn ; T ) 'n k1  k(Tn ; T )jM k k 'n k1 ! 0 n

by Proposition 2.15. Suppose now that the sequence (n ) is bounded away from zero. For each large n, let n;i;j denote the (i; j )th entry of the matrix n for 1  i; j  m. Since Tn 'n = 'n n, we have for each j = 1; : : : ; m,

Tn 'n;j = n;1;j 'n;1 +


+ n;m;j 'n;m ; so that for i = 1; : : : ; m, n jn;i;j j  kTn 'n;j k  k Tn 'n k1 : It follows that

knk := j max ;:::;m 1

=1

m X i=1

jn;i;j j  m kTnk k 'n k1 : n

Hence the sequence   (kn k ) is bounded. As a consequence, the sequence sup kn ; z Ik is bounded. 1

z2C

1

106

2. SPECTRAL APPROXIMATION

By Corollary 1.22, there is a Cauchy contour Ce separating  from (sp(T ) n ) [ C. Let  := dist(Ce ; C) > 0. Since Ce 2 C (T; ), Proposition e for all suciently large n. Thus for 2.9 shows that n = sp(Tn ) \ int(C) each eigenvalue n of n and each z 2 C, we have

jn ; z j  dist(sp(n ); C) = dist(n ; C) >  > 0: By Kato's result (Lemma 2.20), it follows that the sequence ( 2;n (C)) is bounded. Consequently, k 'n ; P 'n k1 tends to zero.

2.5 Exercises

Unless otherwise stated, X denotes a Banach space over Cj , X 6= f0g and T 2 BL(X ). Prove the following assertions.

2.1 (a) If each Tn ; T is compact and Tn ! T , then Tn ! T . (Hint: Let kxn k  1 and yn := (Tj n ; T )xn with j (n)  n , E := fj (n) : n  n g. If the set E is nite, use the compactness of Tm ; T for a suciently large m; and if the set E is in nite, use the inequality k(Tj n ; T )xn k  kTj n ; T k.) (b) If Tn ! T and Tn ! T , then Tn ! T . (Hint: If xn 2 X with kxn k  1 and k(Tk n ; T )xn k   > 0, then without loss of generality (Tk n ; T )xn ! y in X , where kyk  . Let f 2 X  with hy ; f i = kyk. Then   hy ; f i  lim supn! 1 kTk n f ; T f k = 0:) n

cc

0

( )

0

( )

( )

cc

p

n

( )

( )

( )

2.2 If Tn ! T , then k(Tn ; T )R(z)(Tn ; T )k ! 0 for every nonzero z in re(T ). (Hint: R(z ) = (TR(z ) ; I )=z . Compare: Resolvent operator approximation given in [54], Section 14.)

2.3 Even if we have Tn ! T and Un ! U , we may not have T +  n   = T , then Tn + I ! = T + I. Un ! T + U . For example, if Tn ! T but Tn !  T and  TS . Also, even if we have Tn ! Sn ! S , we may not have Tn Sn ! n

For example, if Tn and T are the operators given in Example 2.3 and 1 P S is the right shift operator de ned by Sx := x(k ; 1)ek for x := 1 P

k=1

  T , but T S ! x(k)ek 2 ` p , then Tn ! n = TS .

k=2

107

2.5. EXERCISES

2.4 Let Tn ! T . Then Tn ! U if and only if (T ;U )T = 0 = (U ;T )U . In this case, (T ; U ) = 0. In case Tn ! O, we have Tn ! U if and only n

n

2

if U 2 = O.

2.5 Let dim X = 1 and (Tn) be ap sequence in BL(X ) such that rank(Tn ) is nite for each n and Tn ! I . Then 0 2 sp(Tn ) for each n, but 0 2= sp(I ). 2.6 Let Tn ! T .

(a) If sp(T ) is contained in an open set , then sp(Tn )  for all large n. (Hint: Let E := Cj n in Theorem 2.6.) (b) Upper semicontinuity of the spectrum: sup fdist(; sp(T )) :  2 sp(Tn )g tends to 0. (Hint: Let E := fz 2 Cj : dist(z; sp(T ))  g in Theorem 2.6, where  > 0.) (c) Upper semicontinuity of each spectral set: Let  be a spectral set for T ,  := dist(; sp(T ) n )=2 and n := f 2 sp(Tn ) : dist(; ) < g. Then sup fdist(; ) :  2 ng ! 0. (Hint: Fix 0 <  < . For all large n, n := f 2 sp(Tn ) : dist(; ) < g.)

2.7 Let Tn ! T , n be an eigenvalue of Tn and n ! . Then  2 n

sp(T ), but  need not be an eigenvalue of T . 2.8 Let Tn ! T and z 2 Cj . Then z 2 re(T ) if and only if z 2 re(Tn) for all large n and the sequence (kRn (z )k) is bounded. (Hint: Proposition 2.5 with T , Te replaced by Tn , T respectively and Theorem 2.6 with E := fz g)

2.9 Lack of lower semicontinuity of the spectrum under norm convergence: 1 1 P P Let X := ` and for x := 2 X , Tx := wi x(i)ei , where 2

i=1

wi := e;k ;

+1

if i = 2k (2` + 1) for some k; `  0:

For each positive integer n, let Tn x := 

i=1

1 P

i=1

wn;i x(i)ei+1 , where

i = 2n (2` + 1) for some `  0, wn;i := 0w ifotherwise. i Then each Tn is nilpotent, sp(Tn ) = f0g, kTn ; T k = e;n ! 0, but sp(T )Pcontains fz 2 Cj : jz j  dg for some d > 0 and (T )  e;2r with j +1 r := 1 j =1 j=2 .

108

2. SPECTRAL APPROXIMATION

2.10 Lack of lower semicontinuity of the spectrum under the pointwise convergence: Let X := ` and, for x 2 X , 2

Tx :=

1 X k=1

x(k + 1)ek ; Tn x :=

n X k=1

x(k + 1)ek :

p Then Tn ! T; sp(Tn ) = f0g; but sp(T ) equals f 2 Cj : jj  1g.

2.11 Let  be a spectral set for T such that 0 62 , and Tn ! T . Then, with the notation given in Section 2.4, Tn Pn ; TPn ! O, TnjM ! TjM , Pn ; PPn ! O and Pn jM ! IjM . However, (Tn ) may not converge p

p

p

p

pointwise to T and (Pn ) may not converge pointwise to P on X    . (Hint:  a b 2  1 Use Proposition 2.15 and X := Cj , A := 0 0 and An := a0 0c for all positive integers n, where a 6= 0 and b 6= c.)

2.12 Let Tn ! T as well as Tn ! U . Then Ux = Tx for every x 2 R(T ) \ R(U ). Also, sp(U ) = sp(T ), and  is a spectral set for U if and only if  is a spectral set for T . For such a set , let P := P (T; ) and Q := P (U; ). If 0 62 , then R(Q) = R(P ) = M say, and UjM = TjM . The condition `0 62 ' cannot be omitted from the preceding statement. (Hint: Exercise 2.8 and Exercise 2.11.)

2.13 Let E be a closed subset of re(T ). (a) Let Tn ! T , k(Tn ; T )T k ! 0 and k(Tn ; T )Tnk ! 0. Then there exists some n such that for all n  n , E  re(Tn ) and for every x 2 X , Rn (z )x ! R(z )x uniformly for z 2 E . As a consequence, if  is a spectral set for T , C 2 C (T; ), n := sp(Tn ) \ int(C), P := P (T; ) and Pn := P (Tn; n ), then Pn ! P . (Hint: [Rn (z ) ; R(z )]x = Rn (z )(T ; Tn )R(z )x tends to 0 uniformly for z 2 E . Compare [54], Proposition p

0

0

p

14.1.) n (b) Let Tn ! T . Then there exists some n0 such that for all n  n0 , E  re(Tn ) and kRn(z ) ; R(z )k ! 0 uniformly for z 2 E . As a consequence, kPn ; P k ! 0.

2.14 Let P and Pe be projections in BL(X ) such that kP ; Pek < 1. Let Y := R(P ); Ye := R(Pe). Then PejY;Ye is bijective and k(PejY;Ye ); k is less than or equal to 1=(1 ; kP ; Pek). 1

109

2.5. EXERCISES

2.15 Lack of lower semicontinuity of the spectrum at a nonzero isolated point: Let X := ` and for x 2 X ,  x(1) if k = 1, (Tx)(k) := ; x(k ; 1) + x(k) if k = 6 1, 1

and for each positive integer n, 8 ;x(1) + x(2k) > > > > < x(k ; 1) + x(k ) (Tn x)(k) := > 3x(k) + x(k + 1) > 9x(k ; 1) + x(k) > > : 0

if k = 1, if 1 < k < n, if k = n + 1, if n + 1 < k  2n, if k > 2n.

p Then T; Tn 2 BL(X ),  := ;1 is an isolated point of sp(T ), Tn ! T and sp(Tn ) \ fz 2 Cj : jz + 1j < 1g = ;, so that there is no n 2 sp(Tn ) such that n ! . (Hint: If x(k) := (;1=2)k;1 for all k, then Tx = ;x; and if z 2 Cj , z 6= ;1 and jz ; 1j > 1, then T ; zI is bijective. Also, if Xn := spanfe1; : : : ; e2n g and An is the (2n)(2n) matrix representing TnjXn ;Xn , then det(An ; z I2n ) = ;(1 + z )(1 ; z )2n;1 ; 32n;1.)

2.16 Let X be a Hilbert space and Tn ! T . p

(a) If each Tn is normal and  is an eigenvalue of T , then the lower semicontinuity of the spectrum holds at . (Hint: If Tx = x; x 6= 0, then there exists n 2 sp(Tn ) such that jn ; j  kTnx ; xk by the Krylov-Weinstein Inequality. See Proposition 1.41(c).) (b) If each Tn is selfadjoint, then the lower semicontinuity of the spectrum holds at every spectral value  of T . (Hint: T is selfadjoint,  2 IR. For  > 0, z :=  + i  2 re(Tn ) \ re(T ), kRn (z )k  1= = kR(z )k. If x 2 X with kxk = 1 and kR(z )xk  1=2, then [Rn (z ) ; R(z )]x = Rn (z )(T ; Tn )R(z )x ! 0.)

2.17 For x :=

integer n,

Tn x :=

nX ;1  k=1

1 P

k=1

x(k)ek 2 ` 1 , let Tx := x(1)e1 and for each positive 





x(k) + n1=n x(k + 1) ek + x(n) + (;1)n n(1;n)=n x(1) en:

Then each Tn is a nite rank operator, the sequence (kTnk) is bounded and k(Tn ; T )T k ! 0. However, 2n := 2 2 sp(T2n ) but 2 2= sp(T ). Also, there is no n 2 sp(Tn ) such that n ! 1 although 1 2 sp(T ). (Hint: If Xn := R(Tn ) = spanfe1; : : : ; en g and An is the nn matrix representing the operator TnjXn ;Xn , then det(An ; z In ) = (1 ; z )n ; 1.)

110

2. SPECTRAL APPROXIMATION

2.18 Let X := Cj  with any norm and w; z 2 Cj such that w 6= 0, z= 6 0, w + z 6= 0. Consider         z wz : A := ;zz ;zz ; B := 20z 2z0 ; C := z1 wz ; D := w 0 w (a) Let An := B for all n. Then (kAn k) is bounded, k(An ; A)Ak = 0, n := 2z 2 sp(An ) for all n, n !  := 2z , but  2= sp(A). (Compare Corollary 2.7.) Also,  := 0 2 sp(A), but there is no n 2 sp(An ) such that n ! . (b) Let Bn := A for all n. Then (kBn k) is bounded, k(Bn ; B)Bn k = 0, n := 0 2 sp(Bn ) for all n, n !  := 0, but  2= sp(B). Also,  := 2z 2 sp(B),  = 6 0, but there is no n 2 sp(Bn ) such that n ! . (Compare Corollaries 2.7 and 2.13.) (c) Let Cn := D for all n. Then (kCn k) is bounded, k(Cn ; C) k = 0, n := z 2 sp(Cn ) for all n, n !  := z , but z 2= sp(C). Also,  := w+z = 6 0,  2 sp(C), but there is no n 2 sp(Cn ) such that n ! . 2 1

2

(Compare Corollaries 2.7 and 2.13.) 2.19 Let Tn ! T , where T is compact and sp(Tn) = f0g for each n. Then sp(T ) = f0g. (Hint: Each nonzero  2 sp(T ) is an isolated point of sp(T ). Use Corollary 2.13.) 2.20 (Bouldin) Let  be a spectral set of nite type for T and 0 2= . If p Tn ! T , k(Tn ; T )Tnk ! 0 and kTn(Tn ; T )k ! 0, then the conclusions of Theorem 2.6, Corollary 2.7, Proposition 2.9, Corollary 2.10, Theorem 2.12 and Corollary 2.13 hold. (Hint: If C 2 C (T; ) with 0 2 ext(C), then for z 2 C,   I + R(z )(Tn ; T )T = R(z ) I + Tn ; T (T ; zI );

n z z   I + Tn(Tn ;z T )R(z ) = (Tn ; zI ) I + Tn z; T R(z ): Also, k(Tn ; T )P k ! 0 since P := P (T; ) is compact. Compare [23].) 2.21 Let X := Cj 12 with the 1-norm,p Y1 := f[z; 0] : z 2 Cj g, Y2 := f[z; z ] : z 2 Cj g and Y3 := f[pz; ( 2 ; 1)z ] : z 2 Cj g. Then gap(Y1 ; Y2 ) = 1 and gap(Y1 ; Y3 ) = 2 ; 1 = gap(Y3 ; Y2 ), so that gap(Y1 ; Y2 ) > gap(Y1 ; Y3 ) + gap(Y3 ; Y2 ). 2.22 Let P and Pnccbe projections in BL(X ) with Pn !p P and rank(P ) be nite. Then Pn ! P if and only if k(Pn ; P )Pn k ! 0 if and only if rank(Pn ) = rank(P ) for all large n. In this case, gap(R(Pn ); R(P )) ! 0.

111

2.5. EXERCISES

2.23 With the notation of Theorem 2.18 and Proposition 2.21, k Pn ' ; ' k1 `(C)  2 e (C) (C) k(Tn ; T )jM k ! 0; k ' k1 k Pn ' ; ' k1  `2(C)  (C) (C) k( Tn ; T ) ' k1 ! 0: 1

2

1

2

2.24 Estimates for eigenvectors: Let  be a spectral set for T , C 2 C (T; ) and Tn ! T . With the notation of Proposition 2.9, we have the following. (a) If n 2 n , 'n 2 X and Tn 'n = n 'n for all large n, then

 (C ) k'n ; P'n k  rn := `2(C)  dist( ; C) k(Tn ; T )'n k; 1

n

and if rn < k'n k, then  6= ;. (b) If  2 , ' 2 X and T' = ', then

 (C ) kPn ' ; 'k  ren := `2(C)  dist(; C) k(Tn ; T )'k; 2

and if ren < k'k, then n 6= ;.

2.25 Let  be a spectral value of T of nite type. Suppose that g is the n geometric multiplicity and ` is the ascent of  . Let T ! T , or  6= 0 n  T . For 0 < d < dist(; sp(T ) n fg) and each large n, let and Tn ! sp(Tn ) \ fz 2 Cj : jz ; j < dg = fn;1; : : : ; n;r(n) g: If gn;j is the geometric multiplicity of n;j and `n;j is the ascent of n;j , j = 1; : : : ; r(n), then (a) gn;j  g; but one may not have gn;1 +


+ gn;r(n) = g, (b) `n;1 +


+ `n;r(n)  `, but the equality may not hold.

2.26 Let  be the dominant spectral value of T (that is,  2 sp(T ) and jj > jj for all  2 sp(T );  6= ). Assume that  is nonzero and of  nite type. Let m be the algebraic multiplicity of  and Tn ! T . The m spectral values of Tn (counted according to their algebraic multiplicities) having largest absolute values converge to .

Chapter 3 Improvement of Accuracy

Let us rst review the development of the subject matter so far. In Chapter 1, we have discussed the eigenvalue problem T' = ' for a bounded operator T on a complex Banach space X . In order to obtain an approximate solution to this problem, in Chapter 2 we have considered sequences (Tn ) of bounded operators on X which converge to T in an appropriate manner, and have given error estimates for the solutions of the approximate eigenvalue problem Tn 'n = n 'n . In the next chapter, we shall point out several methods of constructing sequences of nite rank operators Tn which approximate T and show how to reduce the eigenvalue problem Tn 'n = n 'n to a matrix eigenvalue problem An un = n un . Typically, the size of the matrix An increases with n. The cost of the computations is signi cantly aected by the increase in the size of the matrix An . As the applications of the eigenvalue problems become more and more varied, along with a demand for increased accuracy, the conventional methods require a ner and ner discretization. They tend to create problems of ineciencies and instabilities of algorithms while dealing with very large nonsparse matrices. Also, one has to keep track of the accumulation of roundo errors and of the limitations in storage and speed of available computers. As a result, one would like to deal with a matrix eigenvalue problem of optimal size in order to maximize accuracy, minimize computational errors, and respect the storage and time requirements. In many situations, one has no option but to keep the size of the matrix eigenvalue problem moderate. But then the accuracy may not be very high. The purpose of the present chapter is to provide two ways of attaining high precision without having to solve a matrix eigenvalue problem of a very large size. The rst approach is based on an iterative re nement 113

114

3. IMPROVEMENT OF ACCURACY

of a not-so-precise solution of the given eigenvalue problem, and the second is based on a higher order spectral analysis which is equivalent to reformulating the given eigenvalue problem on a suitable product of the original Banach space X . The second approach is known as acceleration. When the desired eigenvalue (or the desired cluster of eigenvalues) of T is not well separated from the rest of the spectrum of T , the iterative re nement approach may cause ill-conditioning. In this situation, the acceleration approach should be preferred. A combination of the the two approaches mentioned above is treated in [32], [15] and [16].

3.1 Iterative Re nement 3.1.1 General Remarks

Let 'e be an approximate solution of the eigenvalue problem T' = '. The Power Method and its variants are the most well-known iterative methods of improving upon 'e. Let 'e 0 = 'e and for k = 1; 2; : : : de ne ( )

'e k = ck T 'e k;1 ; ( )

(

)

where a suitable scalar ck is chosen so as to ensure that k'e k k is neither too large nor too small. For example, if the eigenvalue  we seek is positive, then one may let ck = 1=kT 'e k;1 k; provided T 'e k;1 6= 0. In general, one may consider some fe 2 X  and let ck = 1=h'e k;1 ; fei, provided h'e k;1 ; fei 6= 0. The main limitation of the Power Method is that it can be used to approximate only the dominant eigenvalue of T (if it exists). If T is invertible and has a unique spectral value of the smallest modulus, then the Power Method can be applied to T ;1; and then it is known as the Inverse Power Method. More generally, if  is an isolated spectral value of T and if we can nd a scalar c 2 re(T ) such that j ; cj < j ; cj for every  2 sp(T ) with  6= , then the Power Method can be applied to (T ; cI );1 , which is known as the Inverse Power Method with shift c. The implementation of this method involves solving the equation (T ; cI )x = 'e k;1 and letting 'e k = x=hx ; fei, k = 1; 2; : : : We shall now consider another method for approximating an intermediate eigenvalue  of T , which is analogous to the following iterative ( )

(

)

(

)

(

(

)

(

)

( )

)

115

3.1. ITERATIVE REFINEMENT

re nement technique of nding an approximate solution of the operator equation Ax = y; where A 2 BL(X ) is invertible and y is a given element of X . Suppose that there is an invertible operator Ae in BL(X ) which is an approximation of A and for which it is easier to nd xe 2 X which satis es Aexe = y. Let xe 0 := xe. Consider the residual re 0 := y ; Axe 0 , nd ue 0 2 X such that Aeue 0 = re 0 , and de ne xe 1 := xe 0 + ue 0 . Then xe 1 may be considered as a re nement of the initial solution xe 0 . This procedure can be repeated to obtain the following iterations: ( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

For k = 1; 2; : : : set re k;1 := y ; Axe k;1 ; solve Aeue k;1 = re k;1 ; set xe k := xe k;1 + ue k;1 : (

)

(

(

)

( )

(

(

)

)

)

(

)

It can be proved that if A is invertible and (A;1 (A ; Ae)) < 1; then Ae is also invertible; if in addition (Ae;1 (Ae ; A)) < 1; then the sequence xe k converges to x 2 X , which satis es Ax = y. (See Exercise 3.1.) To nd a similar iteration scheme for solving an eigenvalue problem, let us assume, for the ease of presentation, that  is a simple eigenvalue of T with a corresponding eigenvector ', Te is an approximation of T for which it is easier to nd a simple eigenvalue e with a corresponding eigenvector 'e. Let 'e be the eigenvector of Te corresponding to the complex conjugate of e such that h'e ; 'e i = 1. If Pe denotes the spectral e = hx ; ' projection associated with Te and e, then Px e i' e for x 2 X . 0 0 1 0  e e e Let  = , 'e = 'e and  := hT 'e ; 'e i. Consider the residual re 0 = e 1 'e 0 ; T 'e 0 . Clearly, hre 0 ; 'e i = 0, that is, re 0 2 N (Pe) e )u and hence there is a unique ue 0 2 X such that (Te ; I e 0 = re 0 and hue 0 ; 'e i = 0. De ne 'e 1 = 'e 0 + ue 0 . Then e 1 and 'e 1 may be considered as re nements of the initial approximations e 0 and 'e 0 of  and ', respectively. This procedure can be repeated to obtain the following Elementary Iteration: ( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

For k = 1; 2; : : : set e k := hT 'e k;1 ; 'e i; ek' r e k;1 :=  e k;1 ; T ' e k;1 ; k ; 1 k ; 1 e e )ue = re ; solve (T h;ue I k;1 ; 'e i = 0; set 'e k := 'e k;1 + ue k;1 : ( )

(

(

)

)

( ) (

(

( )

(

)

)

(

(

)

)

(

)

(

)

)

( )

( )

116

3. IMPROVEMENT OF ACCURACY

Note that if Se is the reduced resolvent associated with Te and e, then ue k;1 = Sere k;1 and the above scheme can be written as follows: For k = 1; 2; : : : set e k := hT 'e k;1 ; 'e i; 'e k := 'e k;1 + Se(e k 'e k;1 ; T 'e k;1 ): (

)

(

)

( )

(

( )

(

)

)

( )

(

)

(

)

We observe that if 'e k ! ' in X as k ! 1, then e k !  := hT' ; 'e i in Cj and Se(' ; T') = 0: Also, since h'e k ; 'e i = h'e 0 ; 'e i = 1 for all k = 1; 2; : : : we see that h' ; 'e i = 1. Hence h' ; T' ; 'e i =  ;  = 0, that is, Pe(' ; T') = 0 as well. This shows that ' ; T' = 0: Thus  is an eigenvalue of T with a corresponding eigenvector ' which satis es h' ; 'e i = 1. We shall prove the convergence of the Elementary Iteration in Subsection 3.1.2 by employing mathematical induction. (See [54] and [33] for various re nement methods based on the xed point technique.) We shall also consider the case of a multiple eigenvalue of T and, more generally, the case of a nite cluster of multiple eigenvalues of T . The Elementary Iteration can also be viewed as a Newton-type method of nding roots of the equation F (x) = 0; where F (x) := hTx ; 'e ix ; Tx; x 2 X: If ' is an eigenvector of T corresponding to the eigenvalue  such that h' ; 'e i = 1, then hT' ; 'e i = h' ; 'e i =  and hence F (') = 0. Newton's iteration for nding ' starting with some x 0 in X is given by x k := x k;1 ; (Dk;1 F );1 F (x k;1 ); k = 1; 2; : : : where the operator Dk;1 F denotes the Frechet derivative of F at x k;1 : (Dk;1 F )x = hTx ; 'e ix k;1 + hTx k;1 ; 'e ix ; Tx; x 2 X: For reducing the calculations, if we replace Dk;1 F by D0 F , we obtain the following iteration, known as the Fixed Slope Newton Method: xe k := xe k;1 ; (D0 F );1 F (xe k;1 ); k = 1; 2; : : : Now let x 0 = 'e and note that (D0 F )x = hTx ; 'e i'e + hT 'e ; 'e ix ; Tx; x 2 X: Replacing the operator T by the operator Te in the expression for D0 F , we consider the following modi cation of the previous iteration: 'e k := 'e k;1 ; (De 0 F );1 F ('e k;1 ); k = 1; 2; : : : ( )

( )

( )

( )

( )

( )

(

)

(

)

(

(

)

(

)

( )

(

)

(

)

( )

(

)

(

)

( )

)

117

3.1. ITERATIVE REFINEMENT

where, for x 2 X , e ;' e (De 0 F )x := hTx e i' e + hTe' e; ' e ix ; Tx e )x; e ;' = hTx e i' e ; (Te ; I

as Te'e = e'e and h'e ; 'e i = 1. If x 2 N (Pe), that is, hx ; 'e i = 0, then ehx ; ' e ;' hTx e i = hx ; Te ' e i =  e i = 0, since ' e is an eigenvector of Te e corresponding to the complex conjugate of . Thus, for x 2 N (Pe), we have e )x and hence (D e 0 F );1 x = ;Sx: e (De 0 F )x = ;(Te ; I

By mathematical induction, it can be seen that F ('e k;1 ) 2 N (Pe) for k = 1; 2; : : : Hence for k = 1; 2; : : : we obtain (

)

e (' e (' 'e k = 'e k;1 + SF e k;1 ) = ' e k;1 + SF e k ;1 ) = 'e k;1 + Se(e k 'e k;1 ; T 'e k;1 ); ( )

(

)

(

(

)

( )

)

(

(

)

)

(

(

)

)

as before. (See [9] for a treatment of Newton-type re nement methods.) We shall also consider a combination of the Power Method and the Elementary Iteration, which we call the Double Iteration, rst for the case of a simple eigenvalue of T and then for a cluster of a nite number of multiple eigenvalues of T . (See [6] and [13].) We shall see that the Double Iteration gives improved error estimates. Before giving the convergence results and the error analysis for the above-mentioned iterations, we mention another iterative re nement method which is known as the Rayleigh-Schrodinger Iteration. For s in [0; 1], assume that the operator T (s) := Te + s(T ; Te) has an eigenvalue (s) and a corresponding eigenvector '(s) which have convergent power series expansions:

(s) = e +

1 X j =1

j sj and '(s) = 'e +

1 X j =1

j sj :

(See [58] for conditions under which such expansions are valid.) In particular, for s = 0, we have Te'e = e'e. Assume, for the ease of presentation, that e is a simple eigenvalue of Te and that 'e is the eigenvector of Te corresponding to the complex conjugate of e such that h'e ; 'e i = 1. Suppose further that

h'(s) ; 'e i = 1 for all s 2 ]0; 1]:

118

3. IMPROVEMENT OF ACCURACY

Then h j ; 'e i = 0 for every j = 1; 2; : : : Since T (s)'(s) = (s)'(s) for s 2 [0; 1], we have h

i

Te + s(T ; Te) 'e +

1 X j =1





j sj = e +

1 X j =1



j sj 'e +

1 X j =1

j sj



:

Equating the coecients of sj for j = 1, we obtain e ) 1 = ;(T ; Te)' (Te ; I e + 1 ' e and for j = 2; 3; : : : we obtain e ) j = ;(T ; Te) j ;1 + (Te ; I

j ;1 X i=1

i j;i + j ': e

e ) j;' e ) ' Note that h(Te ; I e i = h j ; (Te ; I e i = h j ; 0i = 0, and   h'e ; 'e i = 1, h j ; 'e i = 0 for j = 1; 2; : : : Hence ;h(T ; Te)'e ; 'e i + 1 = 0, that is,

1 = h(T ; Te)'e ; 'e i and for j = 2; 3; : : : ;h(T ; Te) j;1 ; 'e i + j = 0, that is, j = h(T ; Te) j;1 ; 'e i: For j = 1; 2; : : : j is uniquely determined by the relation obtained by equating the coecients of sj stated earlier. In fact, e e ; (T ; Te)' e]; 1 = S [1 ' and for j = 2; 3 : : : h e

j = S j 'e +

j ;1 X i=1

i

i j;i ; (T ; Te) j;1 :

Let e 0 := e, 'e 0 := 'e, and for k = 1; 2; : : : de ne ( )

( )

e k := e + ( )

k X j =1

j ; 'e k := 'e + ( )

k X j =1

j:

Then the pair (e k ; 'e k ) provides an iterative re nement of the initial e ' eigenpair (; e). This approach can also be extended to the general case ( )

( )

119

3.1. ITERATIVE REFINEMENT

of a cluster of a nite number of multiple eigenvalues of T . (See [64], [56], [57], [49], [3] and [5] for the convergence of the Rayleigh-Schrodinger Iteration along with error estimation.) Because of the necessity of computing an ever-increasing number of coecients, the Rayleigh-Schrodinger Iteration is not preferred to the Elementary Iteration or the Double Iteration discussed before. (See [50] for a comparative performance of these three iterations.)

3.1.2 Re nement Schemes for a Simple Eigenvalue

In this subsection we give estimates for iterative re nements of initial approximations of a simple eigenvalue and of a corresponding eigenvector of T 2 BL(X ). We begin by recollecting some basic approximation results from Chapter 2. These give estimates for the initial approximations.

Lemma 3.1

Let  be a simple eigenvalue of T and ' be a corresponding eigenvector. Assume that n (i) Tn ! T or

 T. (ii)  6= 0 and Tn !

Then for each large enough n, Tn has a unique simple eigenvalue n such that n ! . Let 'n be an eigenvector of Tn corresponding to n and 'n be the eigenvector of Tn corresponding to its (simple) eigenvalue n such that h'n ; 'n i = 1. Then h' ; 'n i 6= 0 for all large n. If we let

' n := h' ;'' i ; n ( )

then for all large n, we have  k ' ; ' k n n  c kTn ; T k max jn ; j; k' k n 

( )

and if  6= 0, then

 k ' ; ' k n n max jn ; j; k' k  c k(Tn ; T )T k; n 

( )

where c is a constant, independent of n.

120

3. IMPROVEMENT OF ACCURACY

Proof

If  > 0 is small enough, then by Theorem 2.17(a), there is a positive integer n0 such that for each n  n0 , we have a unique n 2 sp(Tn ) satisfying jn ; j < . Further, n is a simple eigenvalue of Tn and n ! . If 'n and ' n are as stated, then by Theorem 2.17(b), k' n k  2k'n k and j ; j  2   (C ) k(Tn ; T )'k ; ( )

( )

n

2



k'k

k'n ; ' n k   (C ) k(Tn ; T )'k  k' n k k'k ( )

2

( )

for all large n and for some constants 1 (C ) and 2 (C ), independent of n. Since k(Tn ; T )'k  kTn ; T k k'k, and in case  6= 0, k(Tn ; T )'k = k(Tn ;T )T'=k  k(Tn ;T )T k k'k=jj, the desired estimates for jn ;j and k'n ; ' n k=k'nk follow. ( )

Unless otherwise stated, we shall assume throughout this subsection that  is a simple eigenvalue of T and ' is a corresponding eigenvector. Further, the notation n , 'n and ' n will have the meanings given in the statement of Lemma 3.1. Let Pn and Sn respectively denote the spectral projection and the reduced resolvent associated with Tn and n . ( )

Lemma 3.2 Assume that

n  T. (i) Tn ! T or (ii)  6= 0 and Tn ! (a) The sequences (kPnk) and (kSnk) are bounded. Also, k(Tn ; T )Sn Tnk  ec k(Tn ; T )Tnk and k(Tn ; T )SnT k  ec max fk(Tn ; T )Tnk; k(Tn ; T )T kg for some constant ec, independent of n. (b) (i) The sequence (k' n k) is bounded if and only if the sequence (k'n k) is bounded. (ii) The sequence (k'n k) is bounded if and only if the sequence (k'n k) is bounded away from zero. ( )

121

3.1. ITERATIVE REFINEMENT

Proof (a) Since Tn ! T , the boundedness of the sequence (kPnk) follows from Proposition 2.9(b) if we let  = fg. Next, let C denote the circle with center  and radius , where 0 <  < dist(; sp(T ) n fg), drawn counterclockwise. As n ! , there is a positive integer n such that jn ; j  =2 for all n  n , and then 0

0

Z

1 kSnk = ; 2i Rn (z )  dz; z

n C ` (C)  2 (C)  2 dist( ; C)   =2 (C) 2 = 22 (C); n



where 2 (C) := supfkRn(z )k : z 2 C; n  n0 g < 1 by Theorem 2.6. Thus the sequence (kSn k) is bounded. Since Sn Tn = Tn Sn by Proposition 1.24(a), we have

k(Tn ; T )SnTn k  k(Tn ; T )Tnk kSnk: Finally, we may assume without loss of generality that the circle C does not pass through the origin, that is,  6= jj. Since TnRn (z ) = I + zRn(z ) for all z 2 C by Proposition 1.10(e), Z Z Sn = ; 21 i Rn (z )  dz; z = ; 21 i [Tn Rn (z ) ; I ] z ( dz; z ) n n C C Z h Z i 1 dz dz = ; 2i Tn Rn (z ) z ( ; z ) ; I z ( ; z ) : C

n

C

n

Let  = minfjz j : z 2 Cg. Then  = j jj ; j > 0. Since jn ; z j  =2 for all z 2 C, we see that 2 k(Tn ; T )SnT k  `2(C)   (k(Tn ; T )Tnk2 (C)kT k + k(Tn ; T )T k)  2 (2 (C)kT k + 1) maxfk(Tn ; T )Tnk; k(Tn ; T )T kg:

Hence the result follows if we choose ec such that ec  kSnk for all n  n0 and ec  2(2 (C)kT k + 1)=. (b) (i) Since for all large n, Pn ' n = h' n ; 'n i'n = 'n , we see that ( )

( )

k'n k = kPn ' n k  kPn k k' n k; where the sequence (kPn k) is bounded. ( )

( )

122

3. IMPROVEMENT OF ACCURACY

On the other hand, by Corollary 2.10, kPn ' ; 'k = k(Pn ; P )P'k  k(Pn ; P )P k k'k ! 0;

and so kh' ; 'n i'n k = kPn 'k ! k'k = 6 0. Hence k'2 k  jh' ; 'n ijk'n k and

k' n k =

h' ;'' i

 2k'n k ( )

n

for all large n. Thus the sequence (k' n k) is bounded if and only if the sequence (k'n k) is bounded. (ii) Note that for all large n, kPn k = supfkPn xk : x 2 X; kxk  1g = supfkhx ; 'n i'n k : x 2 X; kxk  1g = k'n k k'n k: Since each Pn is a nonzero projection, we have kPn k  1 for all large n. Also, the sequence (kPn k) is bounded. It follows that the sequence (k'n k) is bounded if and only if the sequence (k'n k) is bounded away from zero. ( )

We shall now analyze the Elementary Iteration with the initial approximate eigenvector 'e = 'n , where n is a xed positive integer for which the conclusions of Lemma 3.1 hold. The successive iterates are then obtained as follows: 8 ' 0 := 'n ; and for k = 1; 2; : : : > > > n ( )

> <

(E )

> > > > :

nk := hT'nk;1 ; 'n i; ( )

(

)

;




'nk := 'nk;1 + Sn nk 'nk;1 ; T'nk;1 : ( )

(

)

( )

(

)

(

)

We establish three important relations about these iterates. For x 2 X , we have hSn x ; 'n i = hSn x ; Pn 'n i = hPn Sn x ; 'n i = h0 ; 'n i = 0: Also, since h'n0 ; 'n i = h'n ; 'n i = 1 and for k = 1; 2; : : : ;
h'nk ; 'n i = h'nk;1 ; 'n i + hSn nk 'nk;1 ; T'nk;1 ; 'n i = h'nk;1 ; 'n i; ( )

( )

(

)

(

)

( )

(

)

(

)

123

3.1. ITERATIVE REFINEMENT

it follows that (E )1 h'nk ; 'n i = 1 for all k = 0; 1; 2; : : : The condition (E )1 is equivalent to `Pn 'nk = 'n for all k = 0; 1; 2; : : :' Next, if x 2 X satis es hx ; 'n i = 0, then hTn x ; 'n i = hx ; Tn'n i = hx ; n 'n i = n hx ; 'n i = 0: Now h' n ; 'n i = 1 by the very de nition of ' n . Hence for k = 1; 2; : : : we have h'nk;1 ; ' n ; 'n i = 1 ; 1 = 0; so that hTn ('nk;1 ; ' n ) ; 'n i = 0; and since ; 'n i = ; hT' n ; 'n i = hhT' ' ; ' i ( )

( )

( )

( )

(

)

( )

(

)

( )

( )

n

we obtain

nk ;  = hT'nk;1 ; 'n i ; hT' n ; 'n i = hT ('nk;1 ; ' n ) ; 'n i: ( )

(

)

(

( )

)

( )

Thus (E )2 nk ;  = h(T ; Tn)('nk;1 ; ' n ) ; 'n i for k = 1; 2; : : : Finally, for k = 1; 2; : : : we have 
'nk = 'nk;1 + Sn nk 'nk;1 ; T'nk;1  = 'nk;1 + Sn (nk ; n )'nk;1 + n 'nk;1  +(Tn ; T )'nk;1 ; Tn 'nk;1   = 'nk;1 + Sn (nk ; n )'nk;1 + (Tn ; T )'nk;1 ;Sn (Tn ; n I )'nk;1 : Now by Proposition 1.24(b), Sn (Tn ; n I )'nk;1 = (I ; Pn )'nk;1 = 'nk;1 ; h'nk;1 ; 'n i'n = 'nk;1 ; 'n , so that   'nk = 'n + Sn (nk ; n )'nk;1 + (Tn ; T )'nk;1 : But since T' n = ' n , we also have ' n = Pn ' n + (I ; Pn )' n = 'n + Sn (Tn ; n I )' n = 'n + Sn [(Tn ; T )' n + ( ; n )' n ] : ( )

(

( )

(

)

(

)

( )

(

( )

)

(

( )

(

(

)

(

)

)

)

(

( )

(

)

(

)

(

(

)

( )

( )

)

(

)

( )

( )

)

)

(

(

)

)

(

(

)

(

)

(

)

( )

( )

( )

( )

( )

( )

)

124 Hence

(E )3

3. IMPROVEMENT OF ACCURACY 8 > > > > > > > > > > > > <

for k = 1; 2; : : : 

'nk ; ' n = Sn (Tn ; T )('nk;1 ; ' n ) ( )

(

( )

)

( )

+(nk ; n )'nk;1 ; ( ; n )' n ( )

> > > > > > > > > > > > :

(



)



( )

= Sn (Tn ; T )('nk;1 ; ' n ) (

)

( )



+(nk ; )'nk;1 + ( ; n )('nk;1 ; ' n ) : The equations (E )1 , (E )2 and (E )3 will play an important role in the proofs of convergence and of error estimation of the Elementary Iteration (E ). ( )

(

)

(

)

( )

Theorem 3.3 Assume that

n  T. (i) Tn ! T or (ii)  6= 0 and Tn ! For all large n, let 'n be so chosen that the sequence (k'n k) is bounded and also bounded away from zero. Let n0 := n , 'n0 := 'n and for k = 1; 2; : : : let nk and 'nk be the iterates of the Elementary Iteration (E ). n (a) If Tn ! T , then there is a positive integer n1 such that for all n  n1 and for k = 0; 1; 2; : : : maxfjnk ; j; k'nk ; ' n kg  ( kTn ; T k)k+1; where is a constant, independent of n and k, and if in fact  6= 0, then maxfjnk ; j; k'nk ; ' n kg  k(Tn ; T )T k( kTn ; T k)k ; where  and are constants, independent of n and k. (b) If  6= 0 and Tn ! T , then there is a positive integer n1 such that for all n  n1 and for all k = 0; 1; 2; : : : maxfjn2k ; j; jn2k+1 ; j; k'n2k ; ' n k; k'n2k+1 ; ' n kg  k(Tn ; T )T k( maxfk(Tn ; T )T k; k(Tn ; T )Tn kg)k ; where  and are constants, independent of n and k. ( )

( )

( )

( )

( )

(

( )

( )

)

(

( )

( )

( )

)

(

)

( )

(

)

( )

125

3.1. ITERATIVE REFINEMENT

Proof

By Lemma 3.2(a), the sequence (kSn k) is bounded. Also, since the sequence (k'n k) is bounded and also bounded away from zero, Lemma 3.2(b) shows that the sequences (k' n k) and (k'n k) are bounded. Further, the sequence (kTn k) is bounded. Hence there are constants , p, q, s, t such that for all large n, k'n k  ; k'n k  p; k' n k  q; kSnk  s and kTn ; T k  t: Let 1 = maxf1; g. n (a) Let Tn ! T . By Lemma 3.1, there is a positive integer n0 , and there is a constant c such that for all n  n0 , maxfjn ; j; k'n ; ' n kg  c 1 kTn ; T k: Let := maxfc 1; p; s(1 + p + pq + c 1 )g: Choose n1  n0 such that kTn ; T k  1= for all n  n1 . Fix n  n1 . We show by induction on k that maxfjnk ; j; k'nk ; ' n kg  ( kTn ; T k)k+1 ; k = 0; 1; 2; : : : Since n0 = n , 'n0 = 'n , maxf1; gc = c 1  and n1  n0 , we see that the desired inequality holds if k = 0. Now assuming that the desired inequality holds for a given k  0, we prove that it holds with k replaced by k + 1. By the equation (E )2 , we have ( )

( )

( )

( )

( )

( )

( )

( )

jnk

( +1)

; j  k'n k kTn ; T k k'nk ; ' n k  pkTn ; T k( kTn ; T k)k  ( kTn ; T k)k ; ( )

( )

+1

+2

since p  . Next, by the equation (E )3 ,

 k'nk+1 ; ' n k = Sn (Tn ; T )('nk ; ' n ) + (nk+1 ; )'nk  +( ; n )('nk ; ' n ) ;  kSnk kTn ; T k k'nk ; ' n k + jnk+1 ; j k'nk k
+j ; n j k'nk ; ' n k : Note that (

)

( )

( )

( )

jnk

( +1)

)

( )

( )

( )

( )

(

( )

( )

(

)

( )

( )

; j  kTn ; T k k'nk ; ' n k k'n k; j ; n j  c kTn ; T k; ( )

( )

1

126

3. IMPROVEMENT OF ACCURACY

and since kTn ; T k  1 for n  n1 , we have

k'nk ; ' n k  ( kTn ; T k)k  1; ( )

+1

( )

k'nk k  k'nk ; ' n k + k' n k  1 + q: ( )

( )

( )

( )

Hence

k'nk

; ' n k  s(1 + p + pq + c )kTn ; T k k'nk ; ' n k  kTn ; T k( kTn ; T k)k  ( kTn ; T k)k ; since s(1 + p + pq + c )  . ( +1)

( )

1

( )

( )

+1

+2

1

Thus the desired inequality holds for k + 1 and the induction is complete. Next, if  6= 0, then by Lemma 3.1 there is a positive integer n0 , and there is a constant c such that for all n  n0 ; maxfjn ; j; k'n ; ' n kg  c 1 k(Tn ; T )T k: ( )

Hence if we let  := c 1 , := maxfp; s(1 + p + pq + kT k)g and choose n1  n0 such that kTn ; T k  1=, kTn ; T k  1= for all n  n1 , then the preceding induction argument shows that for each n  n1 and for all k = 0; 1; 2; : : : maxfjnk ; j; k'nk ; ' n kg  k(Tn ; T )T k( kTn ; T k)k : ( )

( )

( )

(b) Let  6= 0 and Tn ! T . By Lemma 3.1 and Lemma 3.2(a), there is a positive integer n , and there are constants c, ec such that for all nn , maxfjn ; j; k'n ; ' n kg  c kTn ; T k; maxfk(Tn ; T )Sn Tnk; k(Tn ; T )Sn T kg  ecn; where n := maxfk(Tn ; T )Tnk; k(Tn ; T )T kg. Let  := c maxf1; tp; s(t + q)g; := maxfp ; ; pt ; g; 0

0

1

( )

1

1

2

2

3

where 1 := 2ec + 3ts; 2 := s[ 1 (1 + p + pq) + ] and 3 := 2 s[t(1 + p + pq) + ].

127

3.1. ITERATIVE REFINEMENT

Choose n1  n0 such that n  minf1=; 1= ; 1g for all n  n1 . Fix n  n1 . We show by induction on k that for k = 0; 1; 2; : : : maxfjn2k ; j; jn2k+1 ; j; k'n2k ; ' n k; k'n2k+1 ; ' n kg  k(Tn ; T )T k( n)k : Let k = 0. Clearly, since n0 = n and 'n0 = 'n , maxfjn0 ; j; k'n0 ; ' n kg  c 1 k(Tn ; T )T k  k(Tn ; T )T k: Also, by the equation (E )2 , we have jn1 ; j = jh(T ; Tn )('n0 ; ' n ) ; 'n ij  ptc 1 k(Tn ; T )T k  k(Tn ; T )T k and by the equation (E )3 , we have  'n1 ; ' n = Sn (Tn ; T )('n0 ; ' n ) + (n1 ; n )'n0 ;( ; n )' n ]  = Sn (Tn ; T )('n0 ; ' n ) + (n ; )' n ; since Sn 'n0 = Sn 'n = Sn Pn 'n = 0. Hence k'n1 ; ' n k  s(t + q)c 1 k(Tn ; T )T k  (Tn ; T )T k: Thus the desired inequality holds for k = 0. Now assuming that the desired inequality holds for a given k  0, we prove that it holds with k replaced by k + 1. First note that the equation (E )3 gives  'n2k+1 ; ' n = Sn (Tn ; T )('n2k ; ' n ) + (n2k+1 ; )'n2k  + ( ; n )('n2k ; ' n )  = Sn (Tn ; T )('n2k ; ' n ) + (n2k+1 ; )('n2k ; ' n ) + (n2k+1 ; )(' n ; 'n ) +( ; n )('n2k ; ' n ) ; since Sn 'n = 0. As maxfj ; n j; k' n ; 'n kg  k(Tn ; T )T k  n , (

)

(

)

(

)

( )

( )

( )

)

( )

( )

( )

( )

( )

(

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

(

( )

)

(

( )

)

(

(

(

)

(

)

(

( )

)

)

(

)

( )

)

( )

(

)

( )

( )

(

)

( )

( )

we have k(Tn ; T )('n2k+1 ; ' n )k  k(Tn ; T;)Sn(Tn ; T )k k'n2k ; ' n k +kTn ; T k kSn k jn2k+1 ; j k'n2k ; ' n k
+n jn2k+1 ; j + n k'n2k ; ' n k : (

)

(

( )

(

(

)

)

(

(

)

)

)

( )

( )

( )

128

3. IMPROVEMENT OF ACCURACY

Now k(Tn ; T )Sn(Tn ; T )k  k(Tn ; T )SnTn k + k(Tn ; T )Sn T k  2ecn and by the induction hypothesis, we have maxfjn2k+1 ; j; k'n2k ; ' n kg  k(Tn ; T )T k( n)k  k(Tn ; T )T k  n ; since n  1. Hence k(Tn ; T )('n2k+1 ; ' n )k = [2ec + ts( +  + )] nk(Tn ; T )T k( n)k = 1 n k(Tn ; T )T k( n)k ; and by the equation (E )2 , we have jn2k+2 ; j = jh(T ; Tn )('n2k+1 ; ' n ) ; 'n ij  p 1 n k(Tn ; T )T k( n)k  k(Tn ; T )T k( n)k+1 ; as p 1  . Again, by the equation (E )3 , we have ; k'n2k+2 ; ' n k kSn k k(Tn ; T )('n2k+1 ; ' n )k+
jn2k+2 ; j k'n2k+1 k + j ; n j k'n2k+1 ; ' n k : Now k'n2k+1 k  k'n2k+1 ; ' n k + k' n k  k(Tn ; T )T k( n)k + k' n k  n ( n )k + q  1 + q; as n  1 and n  1. Hence we obtain k'n2k+2 ; ' n k  s [ 1 + p 1 (1 + q) + ] n k(Tn ; T )T k( n)k = 2 n k(Tn ; T )T k( n)k  k(Tn ; T )T k( n)k+1 ; since 2  . Further, by the equation (E )2 , we have jn2k+3 ; j = jh(T ; Tn )('n2k+2 ; ' n ) ; 'n ij (

)

(

(

(

(

)

)

( )

)

( )

)

(

)

(

( )

(

(

)

)

(

(

)

)

( )

( )

)

(

( )

( )

( )

(

)

( )

(

)

(

)

( )

 kT ; Tn k k'n ; ' n k k'n k  pt n k(Tn ; T )T k( n)k  k(Tn ; T )T k( n)k ; k

(2 +2)

( )

2

+1

)

( )

129

3.1. ITERATIVE REFINEMENT

as pt 2  . Finally, since

k'n k k  k'n k ; ' n k + k' n k  k(Tn ; T )T k( n)k + k' n k  1 + q; (2 +2)

(2 +2)

( )

( )

+1

the equation (E )3 gives

k'n k

;

( )

; ' n k  kSnk kTn ; T k k'n k ; ' n k +jn k ; j k'n k k
+j ; n j k'n k ; ' n k  s[t + pt (1 + q) +  ]n k(Tn ; T )T k( n)k = n k(Tn ; T )T k( n)k  k(Tn ; T )T k( n)k ; since k(T ; Tn )Tn k  n  1 and  . (2 +3)

(2 +2)

( )

(2 +3)

(2 +2)

2

( )

(2 +2)

( )

2

2

3

+1

3

Thus the desired inequality holds for k + 1 and the induction is complete. n The preceding theorem shows that if Tn ! T , then the sequence of iterates given by the Elementary Iteration (E ) converges `geometrically',  T , then it converges `semigeometrically'. while if  6= 0 and Tn ! In order to improve the rate of convergence of the iterates, especially when Tn ! T and  6= 0, we modify the Elementary Iteration (E ) by inserting a step of `Power Iteration' between its successive steps. This modi ed iteration will be called the Double Iteration. Let  6= 0 and n be a xed positive integer for which the conclusions of Lemma 3.1 hold. Keeping in mind the equation ' = T'  , the successive iterates are de ned as follows: 8 0 > n := 'n ; and for k = 1; 2; : : : > ( )

(D)

> > > > > > > > > > > > <

nk := hT nk;1 ; 'n i, and if nk 6= 0; ( )

(

)

k;1 k;1 := T n ; ' n > nk > > > > > > > nk := hT'nk;1 ; 'n i; > > > (

)

(

( )

)

( )

( )

> > > :

(

)

(k) (k ;1 ) + Sn ((nk) '(nk;1) ; T'(nk;1) ): n := 'n

130

3. IMPROVEMENT OF ACCURACY

Whenever these iterates are well de ned, we obtain the following relations, as in the case of the Elementary Iteration: (D)1 h nk ; 'n i = 1 for k = 0; 1; 2; : : : and also ( )

h'nk ; 'n i = hT

(D)1;1

( )

(k)  n ; 'n i = 1 for k = 0; 1; 2; : : : (k+1) n

Next, (D)2 nk ;  = h(T ; Tn)( nk;1 ; ' n ) ; 'n i for k = 1; 2; : : : and also (D)2;2 nk ;  = h(T ; Tn )('nk;1 ; ' n ) ; 'n i for k = 1; 2; : : : Further, 8 for k = 1; 2; : : : > > > ( )

(

( )

(D)3

> < > > > > :

and also

(D)3;3

)

(

( )

)

( )

 (k) (k;1) ; '(n) ) + ((nk) ; )'(nk;1) n ; '(n) = Sn (Tn ; T )('n 

+( ; n )('nk;1 ; ' n ) (

8 > > > > > > > > > > > > < > > > > > > > > > > > > :

)

( )

for k = 0; 1; 2; : : :  (k) n ; '(n) (nk+1)   (k)  (k+1) = T n (k;+1')(n) + ( ; n(k+1)) '(n) ; n  n

'nk ; ' n = T ( )

( )



provided nk+1 6= 0: (

)

Theorem 3.4  Let  = 6 0 and Tn ! T . For all large n, let 'n be so chosen that the sequence (k'n k) is bounded and also bounded away from zero. Let n = n and n = 'n . Then there is a positive integer n such that for each xed n  n , all the iterates nk and nk of the Double (0 )

(0)

( )

1

( )

Iteration (D) are well de ned, and for k = 0; 1; 2; : : : they satisfy maxfjnk ; j; k nk ; ' n kg  ( k(Tn ; T )T k)k+1; ( )

( )

( )

1

131

3.1. ITERATIVE REFINEMENT where is a constant, independent of n and k.

Proof

Let ; p; q; s; t and 1 be the constants stated at the beginning of the  T , Lemma 3.1 shows that proof of Theorem 3.3. Since  6= 0 and Tn ! there is a positive integer n0 , and there is a constant c such that for all n  n0 , maxfjn ; j; k'n ; ' n kg  c 1 k(Tn ; T )T k: ( )

Let

:= maxfc 1 ; pc 1 ; 1 ; 2 ; p 2 g;   2kT k (p + pq) + c kT k. where 1 := j2j 1 + tpq and 2 := 1 s 1 + 1 jj jj Since  6= 0 and k(Tn ; T )T k ! 0, choose n1 such that k(Tn ; T )T k  1= and k(Tn ; T )T k  2jt j . Fix n  n1 . We show by induction on k that for k = 0; 1; 2; : : : the iterates nk and nk+1 are well de ned and 

( )

(

)

(i) k nk ; ' n k  ( k(Tn ; T )T k)k+1, (ii) jnk+1 ; j  kTn ; T k( k(Tn ; T )T k)k+1, ( )

(

( )

)

(iii) jnk+1 j  j2j and (

)

(iv) jnk+1 ; j  ( k(Tn ; T )T k)k+2. Since n0 = 'n and c 1  , we have k n0 ; ' n k  k(Tn ; T )T k. By the equation (D)2 , it follows that (

)

( )

( )

( )

jn ; j = jh(T ; Tn )( n ; ' n ) ; 'n ij  pc kTn ; T kk(Tn ; T )T k  t k(Tn ; T )T k  j2j ; ( 1)

(0)

( )

1

since pc 1  and k(Tn ; T )T k  2jt j . As a consequence, jn1 j  jj ; jn1 ; j  jj=2. In particular, n1 6= 0 and the iterates 'n0 and n1 are well de ned. By the equations (D)2;2 and (D)3;3 , we have ( )

( )

( )

( )

jn ; j = jh(T ; Tn )('n ; ' n ) ; 'n ij (1)

( 0)

( )

( )

132

3. IMPROVEMENT OF ACCURACY

= h(T ; Tn )T



(0)

n

; ' n + ( ; n )' n  ; ' i n (1)

( )

( )

 n1   2  k(Tn ; T )T k jj + 2jtpq j2 pc 1 k(Tn ; T )T k  ( k(Tn ; T )T k)2; 

n1

( )

( )



since pc 1  and 1 = j2j 1 + tpq jj  : Thus the desired inequalities hold for k = 0. Now assuming that the iterates nk and nk+1 are well de ned (so that nk 6= 0 and nk+1 6= 0) and that the inequalities (i) to (iv) hold for a given k  0, we prove that the iterates nk+1 and nk+2 are well de ned and the inequalities (i) to (iv) hold with k replaced by k + 1. Since nk+1 is well de ned, so is nk+1 . By the equation (D)3 , we have ( )

( )

(

(

)

)

(

(

)

(

)

(

)

)

 (k+1) ; '(n) = Sn (Tn ; T )('(nk) ; '(n) ) n  +((nk+1) ; )'(nk) + ( ; n )('(nk) ; '(n) ) ;

and by the equations (D)3;3 and (D)2 , we obtain

 k k(Tn ; T )('nk ; ' n )k =

(Tn ; T )T n k; ' n n  k + ( ; n )' n

( )

( )

( )

( )

( +1)

( +1)

 nk

( )

( +1)





k  k(Tn ; T )T k j2j + 2jtpq j k n ;'n k  k(Tn ; T )T k( k(Tn ; T )T k)k : ( )

2

( )

+1

1

As a result, we also obtain by the equation (D)2;2 ,

jnk

( +1)

; j = jh(T ; Tn )('nk ; ' n ) ; 'n ij  p k(Tn ; T )T k( k(Tn ; T )T k)k : ( )

1

( )

+1

Further, k ; k'nk k = kT k n k  2kjTj k k jn j ( )

( )

( +1)

(k)  
2 kT k n ; 'n k + k'n k  jj (1 + q );

133

3.1. ITERATIVE REFINEMENT

since k nk ; ' n k  ( k(Tn ; T )T k)k+1  1. Again, by the equations (D)3;3 and (D)2 , we see that ( )

( )

j ; n j k'nk ; ' n k    c k(Tn ; T )T k kT k j2j + 2 j+jpq k nk ; ' n k  c kT k k(Tn ; T )T k( k(Tn ; T )T k)k : ( )

( )

1

2

1

( )

( )

+1

1

From the preceding four estimates, we obtain k nk+1 ; ' n k = 2 k(Tn ; T )T k( k(Tn ; T )T k)k+1  ( k(Tn ; T )T k)k+2; since 2  . This proves the inequality (i) with k replaced by k + 1. Next, by the equation (D)2 , jnk+2 ; j = jh(T ; Tn)( nk+1 ; ' n ) ; 'n ij  pkTn ; T k 2k(Tn ; T )T k( k(Tn ; T )T k)k+1  kTn ; T k( k(Tn ; T )T k)k+2; since p 2  . This proves the inequality (ii) with k replaced by k + 1. Also, we have (

(

)

( )

)

(

)

( )

jnk ; j  t k(Tn ; T )T k  j2j ; because k(Tn ; T )T k  1 and k(Tn ; T )T k  2jt j . As a consequence, jnk j  jj;jnk ; j  jj=2. This proves the inequality (iii) with k replaced by k + 1. Since nk = 6 0, the iterate nk is well de ned. ( +2)

( +2)

( +2)

( +2)

( +2)

Finally, by the equations (D)2;2 and (D)3;3 , we have jnk+2 ; j = jh(T ; Tn )('nk+1 ; ' n ) ; 'n ij   k+2  k(T ; Tn)T k j2j + 2jtpq j2 ( k(Tn ; T )T k) p (

)

(

)

( )

= p 2 k(Tn ; T )T k( k(Tn ; T )T k)k+2  ( k(Tn ; T )T k)k+3; since p 2  . This proves the inequality (iv) with k replaced by k + 1. Thus the induction argument is complete. Since the inequalities (i) and (iv) hold for all k = 0; 1; 2; : : : and since n0 = n , we see that the conclusion of the theorem holds for all k = 0; 1; 2; : : : ( )

134

3. IMPROVEMENT OF ACCURACY

3.1.3 Re nement Schemes for a Cluster of Eigenvalues

In this subsection we extend the considerations of the previous subsection to treat the case of an eigenvalue of T of nite algebraic multiplicity and, more generally, the case of a cluster of a nite number of such eigenvalues. First, we recollect some results from Chapter 2 and develop them further. For a xed positive integer m, we consider the Banach space X := X 1m = f[x1 ; : : : ; xm ] : xj 2 X for j = 1; : : : ; mg with the norm

k[x ; : : : ; xm ]k1 = maxfkxj k : j = 1; : : : ; mg; 1

and use the notation and results given in Subsection 1.4.2.

Lemma 3.5 Let  = f ; : : : r g be a spectral set of nite type for T , P := P (T; ), rank P = m, and let ' = [' ; : : : 'm ] form a basis for R(P ). Assume 1

1

that

n  T. (i) Tn ! T , or (ii) 0 62  and Tn !

Then for all large n, rank Pn = m. Consider

m  +


+ mn;r(n)n;r(n) b := m1 1 +
m

+ mr r and bn := n;1 n;1 ; m the weighted arithmetic means of the elements in  and n respectively, where mj is the algebraic multiplicity of the eigenvalue j of T , j = 1; : : : ; r, and mn;j is the algebraic multiplicity of the eigenvalue n;j of Tn , j = 1; : : : ; r(n). Then bn ! b. Let 'n := ['n;1 ; : : : 'n;m ] form an ordered basis for R(Pn ), and 'n = ['n;1 ; : : : 'n;m ] form the corresponding adjoint basis for R(Pn ). The mm matrix Cn := ' ; 'n is nonsingular for all large n and

k 'n Cn ; ' k1 ! 0: Also, if ' n := ' Cn;1 , then ( )

n

max jbn ; bj;

k 'n ; ' n k1 o  ckTn ; T k; k 'n k1 ( )

135

3.1. ITERATIVE REFINEMENT and if 0 62 , then n

max jbn ; bj;

k'n ; ' n k1 o  ck(Tn ; T )T k; k 'n k1 ( )

where c is a constant, independent of n.

Proof Let C 2 C (T; ). By Theorem 2.18(a), there is a positive integer n such that for each n  n , n := int(C) \ sp(Tn ) = fn; ; : : : n;r n g 0

0

1

( )

and rank Pn (Tn ; n) = rank P = m. Further, if b and bn are as stated, then for all large n, jb ; bj  `(C)  (C)k(T ; T ) k; n



n

2

jR(P )

where 2 (C) is a constant, independent of n, as we have seen in Theorem 2.18(b). Clearly, k(Tn ; T )jR(P )k  kTn ; T k. Also, k(Tn ; T )jR(P )k  k(Tn ; T )P k. Assume now that 0 62 . Then by Corollary 1.22, we may assume that 0 2 ext(C). Hence Proposition 2.9(b) shows that for all large n,

 (C) k(T ; T )T k; k(Tn ; T )P k  `2(C)  (C) n 1

where 1 (C) is a constant, independent of n. Hence the desired estimates for jbn ; bj follow. Finally, Theorem 2.18(c) shows that the mm matrix Cn is nonsingular for all large n, k 'n Cn ; ' k1 ! 0 and

k 'n ; ' n k1 `(C) k ' n k1  2  (C) (C)k(Tn ; T )jR P k; ( )

1

( )

2

( )

where ' n := ' Cn;1 satis es k ' n k1  2k 'n k1 . Thus the desired estimates for k 'n ; ' n k1 =k 'n k1 follow. ( )

( )

( )

Unless otherwise stated, we shall assume throughout this subsection that  is a spectral set of nite type for T .

136

3. IMPROVEMENT OF ACCURACY

Let P := P (T; ), rank P = m; ' := ['1 ; : : : 'm ] form an ordered basis for R(P ), and ' := ['1 ; : : : 'm ] form the corresponding adjoint basis for R(P  ). Then, as we have seen in Theorem 1.52,

P x = ' x ; '

T ' = ' ; where  := T ' ; ' :

and

Also, the notation b, n , bn , Pn , 'n , 'n , Cn and ' n will have the meanings given in the statement of Lemma 3.5. In addition, for each large n, de ne ( )

n := Tn 'n ; 'n ;  n := T ' n ; 'n ; ( )

( )

Sn := S 'n ; the block reduced resolvent corresponding to Tn; n and 'n ;

n;j := dist('n;j ; spanf'n;i : i 6= j; i = 1; : : : ; mg) for j = 1; : : : ; m; n := minfn;1; : : : n;mg: Note that

Tn 'n = 'n n; Tn 'n = 'n n ; T ' n = ' n  n ( )

and

( )

( )

'n ; 'n = Im = ' n ; 'n : ( )

Lemma 3.6 Assume that

n  T. (i) Tn ! T , or (ii) 0 62  and Tn !

(a) (i) The sequence (k ' n k1) is bounded if and only if the sequence (k 'n k1) is bounded. (ii) The sequence (k 'n k ) is bounded if and only if the sequence ( )

1

(n ) is bounded away from zero.

(b) Assume that the sequence (k 'n k1) is bounded and that the sequence (n ) is bounded away from zero. Then

137

3.1. ITERATIVE REFINEMENT

(i) for all large n,

kn ;  n k  ckTn ; T k ( )

1

and if 0 62 ,

kn ;  n k  ck(Tn ; T )T k; ( )

1

where c is a constant, independent of n, (ii) the sequence (kSn k) is bounded,

k( Tn ; T )Sn Tn k  eck(Tn ; T )Tnk and

k( Tn ; T )Sn T k  ec maxfk(Tn ; T )Tnk; k(Tn ; T )T kg; where ec is a constant, independent of n.

Proof (a) (i) Since for all large n,

Pn ' n = 'n ' n ; 'n = 'n ' C;n 1 ; 'n = 'n ' ; 'n C;n 1 = 'n ; ( )

( )

we see that

k 'n k1  k Pn k k ' n k1 = kPn k k ' n k1 ; where the sequence (kPn k) is bounded by Proposition 2.9(b). On the ( )

( )

other hand, as we have seen in the proof of Theorem 2.18(b),



PnjM;Mn

;1

2

for all large n, where M := R(P ) and Mn := R(Pn ). Hence



k ' n k1 =

PnjM;Mn ( )

;1

'n

 2k 'n k1 1

for all large n. It follows that the sequence (k ' n k1) is bounded if and only if the sequence (k 'n k1) is bounded. ( )

138

3. IMPROVEMENT OF ACCURACY

(ii) Let the sequence (n ) be bounded away from 0. For all large n, we have Pn ' = 'n 'n ; '  : Let Dn := 'n ; ' . Then Dn Cn = 'n Cn ; ' ! ' ; ' = Im ;

since 'n Cn ! ' by Lemma 3.5. Hence for all large n, the matrices Dn and Cn are nonsingular and kC;n 1 D;n 1 ; Ik ! 0. Now 'n = ( Pn ' )(Dn );1 and, by Proposition 1.49, 1

k 'n k  k Pn ' k k(D;n ) k1  k Pn k k ' k kD;n k ; where k Pn k = kPnk  kPn k and the sequence (kPn k) is bounded by 1

1

1

1

1

1

Proposition 2.9(b). Also,

kD;n k = kCn C;n D;n k  kCn k kC;n D;n k ; where the sequence (kC;n D;n k ) tends to 1. Thus, to conclude that the sequence (k 'n k ) is bounded, it is enough to show that the sequence (kCn k ) is bounded. Noting that Cn = [h'j ; 'n;i i] and 1

1

1

1

1

1

1

1

1

1

1

1

1

1

kCnk = j max ;:::;m 1

=1

m nX i=1

o

jh'j ; 'n;i ij ;

we need only prove that for each xed pair (i; j ), 1  i; j  m, the sequence (jh'i ; 'n;j ij) is bounded. So x i; j , 1  i; j  m. For all large n, we have

Pn 'i = h'i ; 'n;1 i'n;1 +


+ h'i ; 'n;m i'n;m = h'i ; 'n;j i'n;j + xn;j for some xn;j 2 Xn;j := spanf'n;` : ` 6= j; ` = 1; : : : ; mg, so that kPn 'i k  jh'i ; 'n;j ij dist('n;j ; Xn;j ) = jh'i ; 'n;j ijn;j  jh'i ; 'n;j ijn : Hence jh'i ; 'n;j ij  kPn 'i k=n  kPn k k'i k=n, where the sequence (kPn k) is bounded and the sequence (n ) is bounded away from 0. Thus the sequence (jh'i ; 'n;j ij) is bounded.

139

3.1. ITERATIVE REFINEMENT

Conversely, assume that the sequence (k 'n k ) is bounded. To prove that the sequence (n ) is bounded away from zero, it is enough to show that for each j = 1; : : : ; m, the sequence (n;j ) is bounded away from zero. So x j , 1  j  m and let Xn;j = spanf'n;i : i 6= j; i = 1; : : : ; mg as before. Since h'n;j ; 'n;j i = 1 and hxn;j ; 'n;j i = 0 for all xn;j 2 Xn;j , we have 1

1 = h'n;j ; xn;j ; 'n;j i  k'n;j ; xn;j k k'n;j k:

Taking in mum over all xn;j 2 Xn;j , we obtain

1  n;j k'n;j k  n;j k 'n k : 1

As the sequence (k'n k1 ) is bounded, we see that sequence (n;j ) is bounded away from zero. (b) By (a) above, the sequences (k ' n k1 ) and (k 'n k ) are bounded. (i) We have by Proposition 1.46, ( )

1

kn ;  n k = k Tn 'n ; T ' n ; 'n k  k Tn 'n ; T ' n k1 k 'n k : ( )

( )

1

1

( )

1

We note that

k Tn 'n ; T ' n k1  k Tn ( 'n ; ' n )k1 + k( Tn ; T ) P ' n k1  kTnk k 'n ; ' n k1 + k(Tn ; T )P k k ' n k1 : Now k 'n ; ' n k1  ckTn ; T k k 'n k1 by Lemma 3.5. Also, jj(Tn ; T )P k  kTn ; T k kP k, while if 0 62 , then k 'n ; ' n k1  ck(Tn ; T )T k k 'n k1 and k(Tn ; T )P k  ck(Tn ; T )T k as in the proof of Lemma ( )

( )

( )

( )

( )

( )

( )

3.5, where c is a constant, independent of n. Hence the desired estimates for jjn ;  n k follow. (ii) For all n greater than or equal to some n0 and all x 2 X , we have Z 1 Sn x = ; 2i Rn (z ) x Rn (z ) 'n ; 'n dz: C Hence by Propositions 1.49 and 1.46, 2  kSn k  `2(C)  2 (C) k 'n k1 k 'n k ; where 2 (C) = supfkRn (z )k : z 2 C; n  n0 g < 1 by Theorem 2.6. Thus the sequence (kSn k) is bounded. ( )

1

1

140

3. IMPROVEMENT OF ACCURACY

As the operators Sn and Tn commute, we have

k( Tn ; T )Sn Tn k  k( Tn ; T ) Tn k kSn k = k(Tn ; T )Tn k kSn k: As a result, the sequence (k( Tn ; T )Sn Tn k) is bounded.

Finally, we may assume without loss of generality that the Cauchy contour C does not pass through 0, so that

 := minfjz j : z 2 Cg > 0: Now for all large n and all x 2 X , we have

Z 1 ( Tn ; T )Sn T x = ; 2i ( Tn ; T ) Rn (z ) T x Rn (z ) 'n ; 'n dz: C By Proposition 1.10(e), we have for all z 2 C, ;
(T ; T )R (z )T = 1 (T ; T )T R (z )T ; (T ; T )T :

n

Hence

n

z

n

n n

n

k( Tn ; T )Sn T k  `2(C)  (k(Tn ; T )Tnk (C)kT k +k(T ; Tn )T k) (C)k 'n k1 k 'n k : 2

2

1

Thus the sequence (k( Tn ; T )Sn T k) is also bounded. We remark that for each large n, 'n forms an ordered basis for Mn :=

R(Pn ) and 'n ; 'n = Im , so that the matrix n = Tn 'n ; 'n rep-

resents the operator TnjMn;Mn with respect to 'n . Similarly, ' forms an ordered basis for M := R(P ) and since the matrix Cn = ' ; 'n is nonsingular for each large n, ' n = ' C;n 1 forms an ordered basis for M . As ' n ; 'n = ' ; 'n C;n 1 = Im , we see that the matrix  n = T ' n ; 'n represents the operator TjM;M with respect to ' n . Hence ( )

( )

( )

( )

( )

bn = m1 tr n; b = m1 tr  n ; jbn ; bj = m1 j tr(n ;  n )j  m1 kn ;  n k : ( )

( )

( )

1

In the iteration schemes for the cluster  of eigenvalues of T , we shall use the bases 'n , ' n and the matrices n ,  n for a xed n, as in the ( )

( )

141

3.1. ITERATIVE REFINEMENT

case of a simple eigenvalue treated in Subsection 3.1.2. If nk denotes the kth iterate of n , then bnk := m1 tr nk will be the approximation of b that we seek. Fix a positive integer n for which the conclusions of Lemma 3.5 hold. In analogy with the Elementary Iteration (E ) for a simple eigenvalue, we consider the Elementary Iteration ( E ) for a cluster of eigenvalues as follows: 8 0 > > 'n := 'n ; and for k = 1; 2; : : : ( )

( )

( )

( )

(E )

> > < > > > > :

nk := T 'n k;1 ; 'n ; ( )

(

)

;




'n k := 'n k;1 + Sn 'n k;1 nk ; T 'n k;1 : ( )

(

)

(

)

( )

(

)

We recall that by Lemma 1.51,

Pn x = 'n x ; 'n ; x 2 X ;

Pn f = 'n 'n ; f ; f 2 X  :

Let Zn : X ! X be de ned by

Zn x = x n; x 2 X : Then by Proposition 1.56, the operators Tn , Pn , Sn and Zn commute, and for all x 2 X ,

Sn ( Tn x ; x n) = x ; Pn x ; Sn Pn x = 0 : These considerations allow us to establish the following three relations in exactly the same manner as we established the relations (E )1 ; (E )2 ; (E )3 in the previous subsection. We merely replace T , Tn , Pn , Sn , , n , nk , ' n , 'n and 'n by T , Tn , Pn , Sn ,  n , n , nk , ' n , 'n , and 'n respectively, and the scalar product h
;
i by the Gram matrix
;
. We note that for z 2 Cj and x 2 X , the element zx of X is to be replaced by the element x Z of X , where x 2 X and Z 2 Cj mm . Thus we have ( )

( )

( )

( E )1

( )

( )

'n k ; 'n = Im and hence Pn 'n k = 'n for k = 0; 1; 2; : : : ( )

( )

( E )2 nk ;  n = ( T ; Tn )( 'n k;1 ; ' n ); 'n for k = 1; 2; : : : ( )

( )

(

)

( )

142 and

( E )3

3. IMPROVEMENT OF ACCURACY 8 > > > > > > > > > > > > > > > > > > > > <

for k = 1; 2; : : : h

'n k ; ' n = Sn ( Tn ; T )( 'n k;1 ; ' n ) ( )

(

( )

)

( )

+ 'n k;1 (nk ; n ) ; ' n ( n ; n ) (

)

( )

( )

h

> > > > > > > > > > > > > > > > > > > > :

i

( )

= Sn ( Tn ; T )( 'n k;1 ; ' n ) (

)

( )

+ 'n k;1 (nk ;  n ) (

)

( )

( )

i

+( 'n k;1 ; ' n )( n ; n ) : (

)

( )

( )

Using the above-mentioned relations, we obtain the following analogue of Theorem 3.3.

Theorem 3.7 Assume that

n  T. (i) Tn ! T , or (ii) 0 62  and Tn ! For each large n, let 'n be so chosen that the sequence (k 'n k1 ) is bounded and the sequence (n ) is bounded away from zero. Let n0 := n , 'n 0 := 'n and for k = 1; 2; : : : let nk and 'n k be the iterates of the Elementary Iteration ( E ). Further, let b k = 1 tr  k for k = 1; 2; : : : ( )

( )

( )

( )

n

m

( )

( )

n

(a) If Tn ! T , then there is a positive integer n such that for all n  n and for k = 0; 1; 2; : : : maxfjbnk ; bj; k 'n k ; ' n kg  ( kTn ; T k)k ; where is a constant, independent of n and k, and if in fact 0 62 , n

1

1

( )

( )

( )

+1

then

maxfjbnk ; bj; k 'n k ; ' n kg  k(Tn ; T )T k( kTn ; T k)k ; where  and are constants, independent of n and k. ( )

( )

( )

143

3.1. ITERATIVE REFINEMENT

(b) If 0 62  and Tn ! T , then there is a positive integer n such that for all n  n and for all k = 0; 1; 2; : : : maxfjbn k ; bj; jbn k ; bj; k 'n k ; ' n k; k 'n k ; ' n kg  k(Tn ; T )T k( maxfk(Tn ; T )T k; k(Tn ; T )Tn kg)k ; 1

1

(2 )

(2 +1)

(2 )

(2 +1)

( )

( )

where  and are constants, independent of n and k.

Proof As jbnk ; bj  knk ;  n k for all large n and all k = 0; 1; 2; : : : it is enough to obtain the desired bounds for knk ;  n k in place of jbnk ; bj. By Lemma 3.6(b)(ii), the sequence (kSn k) is bounded. Also, since the sequence (k 'n k1 ) is bounded and the sequence (n ) is bounded away from zero, Lemma 3.6(a) shows that the sequences (k ' n k1 ) and (k 'n k ) are bounded. Further, the sequence (k Tn k) is ( )

( )

( )

1

( )

( )

1

( )

( )

1

bounded. Hence there are constants , p, q, s, t such that for all large n,

k 'n k1  ; k 'n k  p; k ' n k1  q; kSn k  s and k Tn ; T k  t: ( )

1

Let 1 = maxf1; g. n (a) Let Tn ! T . By Lemma 3.5 and by Lemma 3.6(b)(i), there is a positive integer n0 and there is a constant c such that maxfkn ;  n k ; k 'n ; ' n k1 g  c 1 kTn ; T k: ( )

( )

1

Let := maxfc 1 ; p; s(1 + p + pq + c 1 )g and choose n1  n0 such that kTn ; T k  1= for all n  n1 . Fix n  n1 . It can be shown by induction on k that maxfknk ;  n k ; k 'n k ; ' n k1 g  ( kTn ; T k)k+1; k = 0; 1; 2; : : : ( )

( )

( )

1

( )

exactly as in the proof of Theorem 3.3(a) by employing the relations ( E )2 and ( E )3 . For achieving this, we merely replace the norm k
k on X and the absolute value j
j on Cj by the norm k
k1 on X and the norm k
k on Cj mm respectively. Next, if 0 62 , then by Lemma 3.5 and by Lemma 3.6(b)(i), there is a positive integer n0 and there is a constant c such that 1

maxfkn ;  n k ; k 'n ; ' n k1 g  c 1 k(Tn ; T )T k: ( )

1

( )

144

3. IMPROVEMENT OF ACCURACY

Hence if we let  := c 1 ; := maxfp; s(1 + p + pq + kT k)g and choose n1  n0 such that kTn ; T k  1=; kTn ; T k  1= for all n  n1 , then for each xed n  n1 and all k = 0; 1; 2; : : : we obtain o n max knk ;  n k ; k 'n k ; ' n k1  k(Tn ; T )T k( kTn ; T k)k ; as before. (b) Let 0 62  and Tn ! T . By Lemma 3.5 and Lemma 3.6(b), there is a positive integer n0 and there are constants c, ec such that for all n  n0 , maxfkn ;  n k; k 'n ; ' n k1 g  c 1 k(Tn ; T )T k; maxfk( Tn ; T )Sn Tn k; k( Tn ; T )Sn T kg  ecn ; where n := maxfk(Tn ; T )Tnk; k(Tn ; T )T kg. Then there are constants  and such that if n  n0 and if n  minf1=; 1= ; 1g, then for k = 0; 1; 2; : : : n max kn2k ;  n k ; kn2k+1 ;  n k ; k 'n 2k ; ' n k1 ; ( )

( )

( )

1

( )

( )

(

)

( )

( )

k 'n k

(2 +1)

(

1

)

o

( )

(

1

)

( )

; ' n k1  k(Tn ; T )T k( n)k ; ( )

exactly as in the proof of Theorem 3.3(b). We now consider an analogue of the Double Iteration (D). Let  be a spectral set for T such that 0 62  and rank P (T; ) = m < 1. Fix a positive integer n for which the conclusions of Lemma 3.5 hold. Keeping in mind the equation ' n = T ' n ( n );1 , the successive iterates are de ned as follows: 8 n 0 := 'n ; and for k = 1; 2; : : : > > ( )

( )

( )

( )

(D)

> > > > > > > > > > < > > > > > > > > > > > > :

nk := T n k;1 ; 'n ; and if nk is nonsingular, ( )

(

( )

)

'n k;1 := T n k;1 (nk );1 ; (

)

(

)

( )

nk := T 'n k;1 ; 'n ; ( )

(

)

n(k) := 'n(k;1) + Sn ( 'n(k;1) (nk) ;

T 'n k;1 ): (

)

Whenever the iterates are well de ned, we obtain the following relations as in the case of the Elementary Iteration ( E ). ( D )1 n k ; 'n = Im for k = 0; 1; 2; : : : ( )

145

3.1. ITERATIVE REFINEMENT

and also ( D )1;1

'n k ; 'n = T n k ; 'n (nk+1 );1 = Im for k = 0; 1; 2; : : : ( )

(

( )

)

Next,

( D )2 nk ;  n = ( T ; Tn )( n k;1 ; ' n ); 'n and also ( D )2;2 nk ;  n = ( T ; Tn )( 'n k;1 ; ' n ); 'n ( )

( )

(

( )

Further,

( D )3

8 > > > > > > > > > > <

8 > > > > > > > > > > > > > > > < > > > > > > > > > > > > > > > :

)

for k = 1; 2; : : :

( )

)

for k = 1; 2; : : :

( )

for k = 1; 2; : : : n(k) ;

h

' n = Sn ( Tn ; T )( 'n k;1 ; ' n ) (

( )

)

( )

+ 'n k;1 (nk ;  n )

> > > > > > > > > > :

and also

( D )3;3

(

( )

(

)

( )

( )

i

+( 'n k;1 ; ' n )( n ; n ) (

)

( )

( )

for k = 0; 1; 2; : : :

'n k ; ' n = T ( )

h

( )

n(k) ((nk+1) );1 ;

h

i ' n ;n1 ( )

( )

= T ( n k ; ' n )(nk+1 );1 ( )

( )

(

)

i

+ ' n (nk+1 );1 ( n ; nk+1 );n1 ; ( )

(

)

(

( )

)

( )

provided nk+1 is nonsingular. (

)

Theorem 3.8  Let 0 62  and Tn ! T . For each large n, let 'n be so chosen that the sequence (k'n k1 ) is bounded and the sequence (n ) is bounded away from zero. Let n0 = n and n 0 = 'n . Then there is a positive integer n1 such that for each xed n  n1 , all the iterates nk and n k of the ( )

( )

( )

( )

146

3. IMPROVEMENT OF ACCURACY

Double Iteration ( D ) are well de ned; and if we let b k := 1 tr  k ; ( )

n

( )

n

m

then for all k = 0; 1; 2; : : : maxfjbnk ; bj; k n k ; ' n k1 g  ( k(Tn ; T )T k)k+1; where is a constant, independent of n and k. ( )

( )

( )

Proof As jbnk ; j  knk ;  n k for all large n and all k = 1; 2; : : : it is enough to obtain the desired bounds for knk ;  n k in place of jbnk ; j. Let k 'n k1  for all large n and := maxf1; g.  T , Lemma 3.5 and Lemma 3.6(b)(i) show that Since 0 62  and Tn ! there is a positive integer n and there is a positive constant c such that for all n  n , maxfkn ;  n k ; k 'n ; ' n k1 g  c k(Tn ; T )T k: ( )

( )

( )

1

( )

( )

( )

1

1

0

0

( )

1

( )

1

By Lemma 3.6(a), there are constants p and q such that k 'n k  p and k ' n k1  q: Let M denote the spectral subspace associated with T and . Since sp(TjM;M ) =  and 0 62 , we note that the operator TjM;M is invertible in BL(M ) and hence the operator TM := TjM;M is invertible in BL( M ). Now for all large n, TM ' n = T ' n = ' n  n ; ( )

1

( )

( )

( )

( )

so that ' n = ( TM );1 ' n  n and ( )

( )

( )

Im = ' n ; 'n = ( TM );1 ' n  n ; 'n = ( TM );1 ' n ; 'n  n : Thus for all large n, the mm matrix  n is nonsingular and ( )

( ) ( )

( )

( )

( )

k((n) );1 k1 = k ( TM );1 '(n) ; 'n k1  k( TM );1 k k '(n) k1 k 'n k1  pqk(TM );1 k:

147

3.1. ITERATIVE REFINEMENT

Let  := pqk(TM );1 k. Then for all large n,

k( n ); k  : ( )

1

By Lemma 3.6(b)(ii), there is a constant s such that kSn k  s for all large n. Also, let kTn ; T k  t for all n. Consider := maxfc 1 ; pc 1 ; 1 ; 2 ; p 2 g; where 1 := 2(1 + tpq) and 2 := 1 s(1 + 2kT k(p + pq) + c 1 kT k). 1 , Choose n1  n0 such that k(Tn ; T )T k  1= ; k(Tn ; T )T k  2t and x n  n1 . It can be shown by induction on k and by employing the relations ( D )2 , ( D )2;2 , ( D )3 and ( D )3;3 that for k = 0; 1; 2; : : : the iterates n k and nk+1 are well de ned and (i) k n k ; ' n k1  ( k(Tn ; T )T k)k+1, (ii) knk+1 ;  n k  kTn ; T k( k(Tn ; T )T k)k+1, (iii) nk+1 is nonsingular and k(nk+1 );1 k  2, (iv) knk+1 ;  n k  ( k(Tn ; T )T k)k+2. The induction argument is very similar to the one we gave in the proof of Theorem 3.4. For achieving this, we merely replace T , Tn, Sn , , n , nk , nk , ' n , 'nk , nk and 'n by T , Tn , Sn ,  n , n , nk , nk , ' n , 'n k , n k and 'n  respectively, and the scalar product h
;
i by the Gram matrix
;
. Also, we note that if for some k = 1; 2; : : : ( )

( )

(

(

( )

( )

( )

)

( )

)

( )

1

)

(

( )

(

(

)

( )

( )

( )

)

1

( )

( )

( )

( )

( )

( )

knk ;  n k  21  2k( 1 ); k ; n ( )

( )

1

( )

1

then the matrix nk is nonsingular and k(nk );1 k  2. As we remarked in the proof of Theorem 3.7(a), we replace the norm k
k and X and the absolute value j
j on Cj by the norm k
k1 on X and the norm k
k on Cj mm respectively. Since the inequalities (i) and (iv) hold for all k = 0; 1; 2; : : : and since n0 = n, the desired estimates hold for all k = 0; 1; 2; : : : ( )

( )

1

( )

In Subsection 5.1.2 we shall show how the sequence ( 'n ) mentioned in Theorems 3.7 and 3.8 can be chosen in practice in such a way that (k 'n k1 ) is bounded and (n ) is bounded away from zero.

148

3. IMPROVEMENT OF ACCURACY

3.2 Acceleration 3.2.1 Motivation

In this section, we consider another approach toward improving the accuracy of an approximate solution of the eigenvalue problem. Let T 2 BL(X ). Suppose we wish to nd ' 6= 0 in X and  6= 0 in Cj such that T' = ', but instead we have found 'e 6= 0 in X and e 6= 0 in Cj such that Te'e = e'e, where Te 2 BL(X ) is `near' T . e to both sides of the equation ' = T' and rewrite it Let us add T' as follows: e ' ; (T ; Te)' = T': If (T ; Te) < jj, then by Theorem 1.13(d), the operator I ; (T ; Te) is invertible in BL(X ) and so "

e T' = 1 I ; T ; T

e)];1 e

' = [I ; (T ; T 2

1 T ; Te X = 1 4  k=0

!k 3

!#;1

e T'

T':

5e

!k

1 X

T ; Te after a nite The idea is to truncate the in nite series  k=0 number of terms, say q terms, and attempt to nd 'e q 6= 0 in X and e q 6= 0 in Cj such that [ ]

[ ]

2

3

!k q;1 X e T ; T 1 q 4 5 Te' eq : 'e = e q  k=0 e q [ ]

[ ]

[ ]

[ ]

The preceding equation can be rewritten as follows: (e[q] )q e[q]



' =

q;1 X

k=0 := e[q] ,

(e q )q;1;k (T ; Te)k Te'e q : [ ]

[ ]

If q = 1 and we let e 'e := 'e q ; then the preceding equation e e reduces to the equation 'e = T 'e, that is, to the ordinary eigenequation for the approximation Te of T . This equation is linear in the scalar e. [ ]

149

3.2. ACCELERATION

For a general positive integer q, the above equation leads to a polynomial eigenvalue problem of order q. Solving this problem is a part of higher order spectral approximation. It was rst considered in detail by Dellwo and Friedman in [31], and it was systematically developed in the framework of a product space by Alam, Kulkarni and Limaye in [13]. We observe that the polynomial eigenequation can be recast as follows: 2 6 6 6 6 6 6 [ q ] e 6 6 6 6 6 6 6 4



'e q

[ ]

'e q e q

[ ]

[ ]

.. .

'e q

[ ]

3 2 7 6 7 6 7 6 7 6 7 6 7 6 7 6 7=6 7 6 7 6 7 6 7 6 7 6 5 4




(T ; Te)q; Te

Te (T ; Te)Te


I

1






O






O

O

I

O






O

.. .

...

...

...

.. .

32

'e q

[ ]

76 76 76 76 76 76 76 76 76 76 76 76 76 54

'e q e q

[ ]

[ ]

.. .

'e q

[ ]

3 7 7 7 7 7 7 7 7 7 7 7 7 7 5

:

O


O I O (e q )q;1 (e q )q;1 In a similar manner, the ordinary eigenequation ' = T' with  6= 0 can be recast as follows: [ ]

[ ]

2



6 6 6 6 6 6 6 6 6 6 6 4

' '  .. .

'

q;1

3

2

7 7 7 7 7 7 7 7 7 7 7 5

6 6 6 6 6 6 6 6 6 6 6 4

32

T

O











O

I

O











O 77 66

= O .. .

I

...

O

...






O 77 66 . . 76 .

O






O

...

I

76 76 76 76

.. 7 6 76

O

54

' '  .

'

q;1

3 7 7 7 7 7 7 7 7 7 7 7 5

:

These representations lead us to consider the linear space of all columns of length q with entries in X (as against the linear space of all rows of a given length with entries in X , which we have studied in Subsection 1.4.2). Let X be a complex Banach space, q  2 an integer and 



X q := X q1 = [x1 ; : : : ; xq ]> : xi 2 X for 1  i  q ; [ ]

where

2

x1

3

[x1 ; : : : ; xq ]> := 64 ... 75 :

xq

150

3. IMPROVEMENT OF ACCURACY

For x := [x1 ; : : : ; xq ]> 2 X q , de ne [ ]

kxk1 := maxfkx k; : : : ; kxq kg: Then k
k1 is a complete norm on X q . Given T; Te 2 BL(X ), we de ne T q and Te q in BL(X q ) as follows: For x = [x ; : : : ; xq ]> , let T q x := [Tx ; x ; : : : ; xq; ]> 1

[ ]

[ ]

[ ]

[ ]

1

[ ]

and

Te q x = [ ]

1

"q;1 X

k=0

1

1

e k+1 ; x1 ; : : : ; xq;1 (T ; Te)k Tx

#>

:

The operators T q and Te q can be represented by qq matrices as follows: [ ]

2

[ ]

3

T

O











O

I

O











T q := O

O 77

I

O






O 77 ;

6 6 6 6 6 6 6 6 6 6 6 4

[ ]

2 6 6 6 6 6 6 6 6 6 6 6 6 6 4

.. .

...

...

...

7 7 7 7

.. 77 .7 5

O


O I O 3 Te (T ; Te)Te





(T ; Te)q;1 Te I

O











O

Te q := O

I

O






O

.. .

...

...

...

.. .

O






O

I

O

[ ]

7 7 7 7 7 7 7 7 7 7 7 7 7 5

:

Note that T q depends only on T , while Te q depends on both T and Te. When the integer q is xed, we write X , T and Te for X q , T q and Te q , respectively. j Fix an integer q  2. For x = [x1 ; : : : ; xq ]> 2 X and 0 6=  2 C; k we have T x = x if and only if Tx1 = x1 and xk+1 = x1 = for [ ]

[ ]

[ ]

[ ]

[ ]

151

3.2. ACCELERATION

k = 1; : : : ; q ; 1: Also, for x = [x1 ; : : : ; xq ]> 2 X and 0 6= e 2 Cj , qP ;1 eq x1 and e 1 =  we have Te x = ex if and only if eq;1;k (T ; Te)k Tx k=0 xk+1 = x1 =ek for k = 1; : : : ; q ; 1. For ' 6= 0 in X and  6= 0 in Cj , let h i> ' := '; ' ; : : : ; q';1 2 X : Then, as we have remarked above,

T'' = ' if and only if T' = ': Similarly, for 'e 6= 0 in X and e 6= 0 in Cj , let   'e 'e > 2 X : 'e := '; e ;:::; e eq;1

Then, as we have mentioned earlier,

Te 'e = e'e if and only if eq 'e =

q ;1 X k=0

eq;1;k (T ; Te)k Te': e

Thus our polynomial eigenvalue problem in X can be lifted to an ordinary (that is, linear) eigenvalue problem in the column space X . It is easy to see that T , Te 2 BL(X ), and if k
k denotes the operator norm on BL(X ) as well, then n

kT k = maxf1; kT kg; kT k  max 1; e

q;1 X k=0

o

k(T ; Te)k Tek :

We shall now show that the nonzero spectral values of T are the same as the nonzero spectral value of T , and their multiplicities are not changed. On the other hand, the nonzero spectral values of Te can be very dierent from those of Te. As we shall see later, the nonzero spectral values of Te would, in fact, provide a better approximation of the nonzero spectral values of T . Also, even if Te is of nite rank, Te may not be of nite rank. However, the computation of the nonzero spectral values of Te can be reduced to a matrix eigenvalue problem, as we shall see in Section 5.3.

152

3. IMPROVEMENT OF ACCURACY

Let I denote the identity operator on X . Consider z 6= 0 in Cj . It can be easily seen that T ; z I is invertible in BL(X ) if and only if T ; zI is invertible in BL(X ), and in that case 2

R(T ; z )

6 6 6 6 6 6 6 6 = 666 6 6 6 6 6 6 4

R(T; z )

O











O

R(T; z ) z

; zI

O






O

.. .

.. .

...

...

.. .

.. .

.. .

...

...

O






; zI

I R(T; z ) ; I z q ;1 z q;1 ; z q;2

3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 5

:

In order to obtain a representation for a spectral projection associated with T , we introduce the following notation. Let  be a spectral set for T such that 0 62 , and let C 2 C (T; ) be such that 0 2 ext(C). De ne Z 1 Jk (T; ) := ; 2i R(T; z ) dz z k for k = 0; 1; : : : ; q ; 1: C It is easy to see that Jk (T; ) does not depend on C 2 C (T; ), provided 0 2 ext(C). Clearly J0 (T; ) = P (T; ): We shall denote Jk (T; ), simply by Jk when the context is clear just as we denote P (T; ) by P . By the standard techniques of contour integration, it follows that

PJk = Jk P = Jk for k = 1; : : : ; q ; 1: (Compare the proof of Proposition 1.24(c).) Also, by employing the identity TR(T; z ) = I + zR(T; z ) = R(T; z )T for z 2 re(T ) (1.10(e)), it can be seen that

TJk = Jk T = Jk;1 for k = 1; : : : ; q ; 1: We consider a map J (T; ) : X ! X de ned by

J (T; )x = [J0 x; J1 x; : : : ; Jq;1 x]> : Again, we shall denote J (T; ) simply by J when the context is clear.

153

3.2. ACCELERATION

Proposition 3.9 (a) If 0 6=  2 Cj , then  is an eigenvalue of T if and only if  is an eigenvalue of T , and then the geometric multiplicities of  are the h ' i> ' same. In fact, the map ' 7;! '; ;


; q;1 from N (T ; I )   to N (T ; I) is a bijection.

(b) sp(T ) nf0g = sp(T ) nf0g. Also, if 0 62   Cj , then  is a spectral set for T if and only if  is a spectral set for T . In that case, there is C 2 C (T; ) such that 0 2 ext(C), and every such C belongs to C (T ; ). If P := P (T; ) and P = P (T ; ), then 2

P=

6 6 6 6 6 6 6 6 6 4

P

O


O

J1

O


O

.. .

.. .

.. .

Jq;1 O


O

3 7 7 7 7 7 7 7 7 7 5

:

Also, rank P = rank P . In fact, the map J gives a bijection from R(P ) to R(P ). In particular, if  6= 0, then  is an isolated point of sp(T ) if and only if  is an isolated point of sp(T ); and then its algebraic multiplicities are the same.

Proof (a) The statement is easy to verify. (b) Let 0 =6 z 2 Cj . As we have noted before, z 2 re(T ) if and only if z 2 re(T ). Hence sp(T ) n f0g = sp(T ) n f0g. Letting Y = X , A = T and E = f0g in Proposition 1.29, we see that  is a spectral set for T if and only if  is a spectral set for T . In that case, there is C 2 C (T; ) such that 0 2 ext(C) and every such C belongs to C (T ; ). Now the matrix representation of R(T ; z ), z 2 C, gives the following

154

3. IMPROVEMENT OF ACCURACY

Z 1 matrix representation of P = ; 2i R(T ; z )dz: C 2 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 4

;1 Z R(T; z )dz 2i

O











O

;1 Z R(T; z ) dz 2 i z

I Z dz 2i C z

O






O

.. .

.. .

...

...

.. .

.. .

.. .

...

...

O

;1 Z R(T; z ) dz 2i z q;

I Z dz 2i C z q;1

I Z dz 2i C z q;2

C

C

C

1

3 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 5

:

Z


2I i dzz C

Z Z Z ; 1 dz ; 1 But 2i R(T; z )dz = P , 2i R(T; z ) z k = Jk and dz z k = 0 for > 1  k  q ; 1 as 0 2 ext(C). Thus for x = [x ; : : : ; xq ] 2 X , we have C

C

C

1

P x = [Px1 ; J1 x1 ; : : : ; Jq;1 x1 ]> ; so that x 2 R(P ) if and only if x1 2 R(P ) and x2 = J1 x1 ; : : : ; xq = Jq;1 x1 . The desired results now follow easily. The preceding proposition shows that the spectral considerations for

T and for T are the same, if we exclude 0 from these considerations. (See also Exercise 3.6.) As a result, given an approximation Te of T , we may approximate the nonzero spectral values of T by the nonzero spectral values of Te . In doing so, the accuracy of approximation is improved. The following result is crucial in this respect.

Proposition 3.10

Let  be a spectral set for T such that 0 62 , C 2 C (T; ) with 0 2 ext(C)

155

3.2. ACCELERATION and P := P (T ; ). Then 2

T ; T )P =

(e

6 6 6 6 6 6 6 6 6 4

;(T ; Te)q Jq; O


O 1

O

O


O

.. .

.. .

.. .

3 7 7 7 7 7 7 7 7 7 5

:

O


O In particular, if 0 6=  2 i , ' is an eigenvector of T corresponding to h ' >  and ' = ';  ;


; q';1 , then O

"

e q (Te ; T )' = ; (T ;q;T1) ' ; 0;


; 0

#>

:

Proof

By Proposition 3.9(b), we have 2

qP ;1

6 6 6 = 666 6 6 4

k=1

e 6 (T ; T )P +

(Te ; T )P

O

3

e k O


O (T ; Te)k TJ 7 7

O


O 77

.. .

.. .

7 7

.. 77 .7 5

O O


O Now since TJk = Jk;1 for k = 1; : : : ; q ; 1 and J0 = P , we have q ;1 X k=1

e k = (T ; Te)k TJ

= =

q ;1 X k=1 q ;1 X k=1 q ;1 X k=1

(T ; Te)k [T ; (T ; Te)]Jk (T ; Te)k TJk ;

q;1 X

k=1 q;1 X

(T ; Te)k Jk;1 ;

= (T ; T P e)

(T ; Te)k+1 Jk (T ; Te)k+1 Jk

k=1 ; (T ; T Jq;1 : e )q

:

156

3. IMPROVEMENT OF ACCURACY

Hence the desired expression for (Te ; T )P hfollows. i> Next, let 0 6=  2 , T' = ' and ' = '; ' ;


; q';1 : Then ' is an eigenvector of T corresponding to  and i>

h

' = ;(T ; Te)q Jq;1 '; 0; : : : ; 0 : (Te ; T )' = (Te ; T )P' Since R(T; z )' = '=( ; z ) for each z 2 C, we have Z ' dz = ' Jq;1 ' = ; 21 i  ; z z q;1 q;1 C by Cauchy's Integral Formula. Hence the desired expression for (Te ;T )' follows.

3.2.2 Higher Order Spectral Approximation

For a xed integer q  2, we consider a sequence (Tn ) in BL(X ) which approximates T 2 BL(X ) in a manner which we shall presently specify. This mode of convergence is even weaker than  -convergence. For each pair (T , Tn), n = 1; 2; : : : we consider the operators T , Tn de ned on the linear space X of columns of length q with entries in X . We shall show that the nonzero spectral values of nite type of T are approximated by the nonzero spectral values of nite type of Tn with sharper error estimates as compared to the error estimates obtained in Section 2.4. A similar statement holds for a basis of a spectral subspace corresponding to a spectral set of nite type for T , provided such a set does not contain 0. Throughout this subsection we assume that T 2 BL(X ) and (Tn ) is a  O, that is, sequence in BL(X ) such that (Tn ; T ) ! (kTnk) is a bounded sequence and k(Tn ; T )2k ! 0 as n ! 1:  T. This assumption is in general weaker than the assumption: Tn ! 2  1 j For example, consider the 1-norm on , and let T 2 BL(X )   X := C 1 1 be de ned by the matrix A := 1 1 with respect to the standard basis for X . For n = 1; 2; : : : let Tn 2 BL(X ) be de ned by the matrix An := 10 11 with respect to the standard basis for X . Then 



n = O: (An ; A)2 = O but (An ; A)A = 01 01 !

157

3.2. ACCELERATION

For n = 1; 2; : : : replace Te by Tn in the de nition of Te and let 2 6 6 6 6 6 6 6 6 6 6 6 6 6 4

Tn (T ; Tn )Tn





(T ; Tn )q;1 Tn I

O









O

Tn := O

I

O






O

.. .

...

... ...

.. .

O






O

O

I

3 7 7 7 7 7 7 7 7 7 7 7 7 7 5

:

Then it follows that 2

Tn ; T =

6 6 6 6 6 6 6 6 6 4 2

(Tn ; T )T =

6 6 6 6 6 6 6 6 6 4

;(T ; Tn ) (T ; Tn)Tn


(T ; Tn )q; Tn 3 1

O

O

.. .

.. .

O

O






7 7 7 7 7 7 7 7 7 5

O .. .






O

;

;(T ; Tn ) (T ; Tn) Tn


(T ; Tn )q; Tn O 3 2

2

O

O

.. .

.. .

O

O

and

2

(Tn ; T )Tn =

6 6 6 6 6 6 6 6 6 4

1









7 7

O

O 77

.. .

.. 777 .7

O

O

7

O




O ;(T ; Tn)q Tn 3

O




O

O

.. .

.. .

.. .

O




O

O

7 7 7 7 7 7 7 7 7 5

:

5

;

158

3. IMPROVEMENT OF ACCURACY

Proposition 3.11  T . In fact, there is a positive integer Let (Tn ; T ) ! O. Then Tn ! n such that for n  n , k(Tn ; T ) k  1=2 and then kTn k  maxf1; 2(kTnk + k(T ; Tn)Tn k)g k(Tn ; T )T k  [1 + 2(kTnk + k(T ; Tn )Tn k)] k(Tn ; T ) k; k(Tn ; T )Tn k = k(Tn ; T )q Tn k  maxfkTnk; k(Tn ; T )Tnkg k(Tn ; T ) k: 0

2

0

2

2

Proof Since k(Tn ;T ) k ! 0, there is a positive integer n such that for n  n , k(Tn ; T ) k  1=2. For positive integers n and k, we have 8 if k is even, < k(T ; Tn ) kk= kTn k k k(T ; Tn ) Tn k  : k(T ; Tn) k k; = k(T ; Tn)Tn k if k is odd: Hence if n  n ; we obtain 2

0

2

2

2

2 (

1 X

k=0

0

k(T ; Tn )k Tnk = 

1 X j =0

1) 2

k(T ; Tn) j Tnk +

1 X j =0

0

2



1 X j =0

k(T ; Tn ) j Tnk 2 +1

k(T ; Tn ) kj (kTn k + k(T ; Tn )Tn k) 2

 2(kTn k + k(T ; Tn )Tn k):

Hence for n  n0 , we have

n

kTn k  max 1;

q ;1 X k=0

o

k(T ; Tn )k Tn k

 maxf1; 2(kTnk + k(T ; Tn )Tn k)g: Since the sequence (kTnk) is bounded, it follows that the sequence (kTn k) is also bounded. Next, let n  n . We note that 0

k(Tn ; T )T k  k(T ; Tn ) k + 2

q;1 X

k=2

k(T ; Tn )k Tn k

 k(T ; Tn ) k + k(T ; Tn ) k 2



2

1 X k=0 

k(T ; Tn )k Tnk

 1 + 2(kTnk + k(T ; Tn )Tn k) k(T ; Tn ) k: 2

159

3.2. ACCELERATION

Also, since k(T ; Tn )2 k  1, we have

k(Tn ; T )Tn k = k(T ; Tn)q Tn k 8 <

:

k(T ; Tn ) kq= kTn k 2

if q is even,

2

k(T ; Tn ) k q; = k(T ; Tn)Tn k if q is odd  maxfkTnk; k(Tn ; T )Tnkgk(T ; Tn ) k: Since the sequence (kTn k) is bounded and k(T ; Tn ) k ! 0, it follows  T: that k(Tn ; T )T k ! 0 and k(Tn ; T )Tn k ! 0: Thus Tn ! 2 (

1) 2

2

2

Consider now a spectral set  for T such that 0 62 . By Proposition 3.9(b),  is a spectral set for T as well. We shall use the following notation throughout this section:

P := P (T; ); M := R(P ); P = P (T ; ); M = R(P ): For C 2 C (T; ), we let `(C) := the length of C; (C) := minfjz j : z 2 Cg; 1 (C) := sup kR(T; z )k; 1;q (C) := sup kR(T ; z )k: z 2C z 2C

 T (Proposition 3.11), the theory of spectral approximation Since Tn ! developed in Chapter 2 becomes applicable for the operator T and the sequence (Tn ).

Theorem 3.12

Let  be a spectral set for T such that 0 62 , and consider C 2 C (T; )  O: with 0 2 ext(C). Fix an integer q  2. Assume that (Tn ; T ) !

(a) There is a positive integer n ;q such that C  re(Tn ) for all n  0

n0;q and 2;q (C) := supfkR(Tn ; z )k : z 2 C; n  n0;q g < 1: Further, for each large n, if we let nq := sp(Tn ) \ int(C) and [ ]

Pn := ; 21 i

Z

C

R(Tn ; z )dz;

160

3. IMPROVEMENT OF ACCURACY

then nq and Pn do not depend on C 2 C (T; ), provided 0 2 ext(C). (b) Let Mn = R(Pn) for all large n. Then 2;q (C) q k(Tn ; T )jMn k  `(C) 2(C) k(Tn ; T ) Tn k and 1 (C) q k(Tn ; T )jM k  2`(C) [(C)]q;1 k(Tn ; T ) jM k 1 (C)]2 k(T ; T )q T k:  [`4(C) 2 [(C)]q n [ ]

(c) Suppose that  := f ; : : : ; r g, where each j is a spectral value of T of nite type. If mj is the algebraic multiplicity of j and m := m +


+ mr , then for all large n, rank Pn = m and q nq = fn;q ; : : : ; n;r n;q g; 1

1

[ ]

[ ]

1

[ ]

(

)

q is a spectral value of T of nite type. Further, if where each n;j n q] is the algebraic multiplicity of [q] for j = 1; : : : ; r(n; q ), then m[n;j n;j q] m[n;q]1 +


+ m[n;r = m, and the weighted arithmetic means (n;q ) [ ]

b := m1 1 +
m

+ mn n and q m q  q +


+ mn;r q (n;q ) n;r (n;q ) bnq := n;1 n;1 m [ ]

[ ]

[ ]

[ ]

[ ]

satisfy

jbnq ; bj  `(C)  minf ;q (C)k(Tn ; T )jMn k;  ;q (C)k(Tn ; T )jM kg: [ ]

1

2

Proof (a) By Proposition 3.9(b),  is a spectral set for T and C 2 C (T ; ). Also, by Proposition 3.11, Tn ! T . Hence the desired conclusions follow from Theorem 2.6 and Proposition 2.9(a). (b) By Proposition 2.9(b), we have for all large n, 2;q (C) k(Tn ; T )jMn k  k(Tn ; T )Pn k  `(C) 2(C) k(Tn ; T )Tn k;

161

3.2. ACCELERATION

where k(Tn ; T )Tn k = k(Tn ; T )q Tn k, as stated in Proposition 3.11. Let now x = [x1 ; : : : ; xq ]> 2 M : Then by Proposition 3.9(b), x1 2 M , that is, Px1 = x1 . Since P commutes with R(z ) for each z 2 C, we have 

PJq;1 x1 = ; 21 i



Z C

R(z ) zdz q;1 Px1 = Jq;1 x1 ;

so that Jq;1 x1 2 M: Hence letting Te := Tn in Proposition 3.10, we have

k(Tn ; T )xk1 = k(Tn ; T )P xk1 = k(T ; Tn)q Jq; x k  k(T ; Tn )qjM k kJq; x k; 1

1

1 1

1 (C) where kJq;1 x1 k  2`(C) [(C)]q;1 kx1 k: As kx1 k  kxk1 , we obtain  (C) k(T ; T )q k; k(Tn ; T )jM k  2`(C) n jM [(C)]q; 1

1

where

 (C) q k(T ; Tn )qjM k  k(T ; Tn )q P k  `(C) 2(C) k(Tn ; T ) T k; 1

since P = ; 2Ti

Z C

R(z ) dz z.

(c) By Proposition 3.9(b), rank P = rank P = m. Hence the desired

conclusion follows from Theorem 2.18(a) and (b).

The preceding theorem shows that by employing a spectral analysis of order q  2, we may obtain error estimates for a spectral approximation in terms of k(Tn ; T )jM k = O(k(Tn ; T )q T k) and k(Tn ; T )jMn k = O(k(Tn ; T )q Tn k): If we increase the order of spectral analysis to q + 1 or to q + 2, then the error estimates improve in the following manner. n Let Tn ! T . Since k(Tn ; T )q+1 T k  kTn ; T k k(Tn ; T )q T k and k(Tn ; T )q+1 Tn k  kTn ; T k k(Tn ; T )q Tnk; we have k(Tn ; T )q+1 T k = o(k(Tn ; T )q T k) and k(Tn ; T )q+1 Tn k = o(k(Tn ; T )q Tn k): Thus the error estimates improve in a `geometric' manner as q increases.  O. Since k(T ; T )q+2 T k  k(T ; T )2 k k(T ; Let (Tn ; T ) ! n n n T )q T k and k(Tn ; T )q+2 Tnk  k(Tn ; T )2 k k(Tn ; T )q Tn k); we have k(Tn ; T )q+2 T k = o(k(Tn ; T )q T k) and k(Tn ; T )q+2 Tn k = o(k(Tn ;

162

3. IMPROVEMENT OF ACCURACY

T )q Tn k): Thus the error estimates improve in a `semigeometric' manner as q increases.

Theorem 4.12(b) describes a situation in which we can estimate the rate at which the norms k(Tn ; T )q T k and k(Tn ; T )q Tn k tend to zero.

3.2.3 Simple Eigenvalue and Cluster of Eigenvalues

We rst prove an accelerated approximation result for a nonzero simple eigenvalue of T which is analogous to Theorem 2.17. Recall the notation 1;q (C) and 2;q (C) employed in Theorem 3.12.

Theorem 3.13 Let  be a nonzero simple eigenvalue of T ,  such that 0 <  < minfjj, dist(; sp(T ) n fg)g and C the positively oriented circle with center   O and x an integer q  2. and radius . Assume that (Tn ; T ) ! (a) There is a positive integer n ;q such that for each n  n ;q , we have a unique nq 2 sp(Tn ) satisfying jnq ; j < . Also, nq is a simple eigenvalue of Tn for each n  n ;q and nq !  as n ! 1. (b) Let ' be an eigenvector of T corresponding to  and for n  n ;q , let 'nq 2 X be the rst component of an eigenvector 'n of Tn cor0

[ ]

0

[ ]

[ ]

[ ]

0

0

[ ]

responding to nq . Then for each large n, P'nq is an eigenvector of T corresponding to ,  q q q 'k  k ( T ; T )  n 1;q (C ) k(T ; Tn ) Tn 'n k q jn ; j  2 min ; 2;q (C ) q [ ]

[ ]

[ ]

[ ]

and

jj ; 

k'k

k'n k [ ]

  ;q (C ) 1 k(T ; T )q T ' q k k'nq ; P'nq k  dist( n n n  q ; C ) j q jq [ ]

1

[ ]

n

[ ]

[ ]

n

[ ]



 2(jj ;q;(C)q) k(T ; Tn)q Tn 'nq k: 1

[ ]

> i> q q (c) Let ' := '; ' ; : : : ; q';1 , 'n := 'nq ; 'nq ; : : : ( 'q n)q;1 and n n Pn := P (Tn ; fnq g) for n  n0;q . qLet cn;q be the complexq number such that Pn' = cn;q' n and ' n := '=cn;q . Then ' n is an eigenvector of T corresponding to  and k'nq ; ' qn k  2;q (C ) k(Tn ; T )q 'k  2;q (C ) k(T ; T )q T k: n jjq;1 k'k jjq k' qn k 

h

[ ]

[ ]

[ ]

[ ]



[ ]

[ ]

[ ] ( )

[ ] ( )

[ ]

[ ] ( )

[ ] ( )

163

3.2. ACCELERATION

Proof (a) Let  = fg in Theorem 3.12. Since  is a nonzero simple eigenvalue of T; that is, m := rank P (T; ) = 1; the results follow immediately. (b) Since Tn ! T by Proposition 3.11 and since  is a nonzero simple eigenvalue of T by Proposition 3.9(b), Theorem 2.17(b) shows that if 'n is an eigenvector of Tn corresponding to its simple eigenvalue nq , then 'n is an eigenvector of T corresponding to , for each large n, P'   k ( T ; T ) ' k k ( T ; T ) ' k n n n 1 1 q ;  (C ) j ; j  2 min  (C ) [ ]

;q

[ ]

n

1

and



;q

k'n k1



2

k'k1

  ;q (C ) k(T ; T )' k : k' n ; P''n k1  dist( n 1 q ;C ) n 1

n  > q [q ] ' n [q ] 'n Now ' n = 'n ; [q] ; : : : ; [q] q;1 for some '[nq] 2 X and n (n ) [q ] P''n = [P'n ; J1 '[nq] ; : : : ; Jq;1 '[nq] ]> 6= 0: Since Jk x = Jk Px for all x 2 X and k = 1; : : : ; q ; 1, we see that for each large n, P'[nq] 6= 0, and as R(P ) = N (T ; I ), P'[nq] is an eigenvector [ ]



[ ]

of T corresponding to . Next,  > q Tn ' q ( T ; T ) T ' 1 ( T ; T ) n n n n n (Tn ; T )' n = = q ; ; 0; : : : ; 0 ; nq n (nq )q;1 [ ]

[ ]

[ ]

[ ]

so that k(Tn ; T )' n k1 = k(T ; Tn)q Tn 'nq k=jnq jq : Also, k'n k1 = k'nq k maxf1; 1=jnq jq;1 g: Hence k(T ; Tn )q Tn'nq k k(T ; Tn)q Tn 'nq k : k(Tn ; T )' n k1 =  q q q k'n k1 jn j maxfjn jq;1 ; 1gk'n k jnq j k'nq k Since jnq j  jj ; jnq ; j > jj ; ; it follows that q q j q ; j  2 1;q (C ) k(T ; Tn ) Tn'n k : [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

n

jj ;  k'nq k Also, for all large n, since nq 2 int(C= ) by Theorem 3.12(a), we have dist(nq ; C )  =2 and hence   ;q (C ) k(T ; Tn )q Tn'nq k k'nq ; P'nq k  k' n ; P''n k1  dist( q ;C ) j q jq [ ]

[ ]

[ ]

[ ]

[ ]

2

[ ]

1

n

[ ]



 2(j1j ;q;(C)q) k(T ; Tn)q Tn 'nq k; [ ]

n

[ ]

164

3. IMPROVEMENT OF ACCURACY

as desired. Finally, letting Te = Tn in Proposition 3.10, we have  > q' ( T ; T ) n ' (T ; T ) = ; ; 0; : : : ; 0 : n

Hence

q;1

k(Tn ; T )' k1 = k(T ; Tn )q 'k q ; k' k1 jj k'k maxf1; 1=jjq; g k(T ; Tn)q 'k  k(T ; Tn )q 'k = max fjjq; ; 1gk'k k'k 1

1

1

and

q

jnq ; j  2  ;q (C ) k(T ;k'Tkn ) 'k : (c) As in Theorem 2.17(b), for each large n, Pn' is an eigenvector of Tn corresponding to nq ; and since R(Pn ) = N (T ; nq I), there is a nonzero complex number cn;q such that Pn' n = cn;q'n . Let ' n := '=cn;q . Then since 'nq ; ' qn is the rst component of ' n ; ' n , we [ ]

2

[ ]

[ ]

have

[ ]

( )

[ ] ( )

( )

k'nq ; ' qn k  k' n ; ' n k1 : [ ]

Also,

[ ] ( )

( )

 k' n k1 = kjc' k1j = jkc 'kj max 1; jj1q; n;q n;q ( )

Hence

 1





= k' qn k max 1; jj1q;1 : [ ] ( )

k'nq ; ' qn k  k'n ; ' n k1 max jjq; ; 1 : k' n k1 jjq; k'nq k [ ] ( )

[ ]

[ ]

Now by Theorem 2.17(c),

( )

1

1

( )

k'n ; ' n k1   (C ) k(Tn ; T )' k1 ;q  k' n k1 k'k1 k(T ; Tn)q 'k ; =  ;q (C ) max fjjq; ; 1gk'k ( )

( )

2

2

1

as we have seen in (b) above. Thus k'nq ; ' qn k   (C ) k(T ; Tn )q 'k =  (C ) k(T ; Tn )q T'k 2;q  2;q  jjq;1 k'k jjq k'k k' qn k  2;q (C ) k(T ; T )q T k; [ ] ( )

[ ]

[ ] ( )

jjq

as desired.

n

165

3.2. ACCELERATION

In order to extend the preceding result to the case of a nonzero multiple eigenvalue of T (and more generally, to the case of a nite number of such eigenvalues), we consider the linear space of all rows of a given length with entries in X q , where q is an integer  2. We shall generalize Theorem 3.13 and obtain an accelerated approximation result for a spectral set of nite type for T which can be compared with Proposition 2.21. Let m be a xed positive integer. In Subsection 1.4.2, we have introduced the product space X 1m . If X := X 1m and x = [x1 ; : : : ; xm ] 2 X ; we had considered k x k1 = maxfkxj k1 : 1  j  mg. In analogy with these considerations, let [ ]

X q := (X q )1m = f[x1 ; : : : ; xm ] : xj 2 X q ; 1  j  mg; [ ]

[ ]

[ ]

and for x = [x1 ; : : : ; xm ] 2 X q , [ ]

k x k1 := maxfkxj k1 : 1  j  mg: Then X q is a Banach space over Cj . Note that X q = X qm . Whenever we x an integer q, we write X for X q , just as we wrote X for Xq. If F : X ! X , then its natural extension to X is denoted by F . Thus F [x ; : : : ; xm ] = [F x ; : : : ; F xm ] for x = [x ; : : : ; xm ] 2 X . Identifying X m2 = (3X q ) m with (X m )q , we note that X [ ]

[ ]

[ ]

[ ]

1

1

1

x1

consists of all x = 64 ...

xq

1

1 1

7 5

1

1

; where x i 2 X for 1  i  q, and

k x k1 = maxfk x i k1 : 1  i  qg: Let F : X ! X be given by 2

F1;1


F1;q

3

.. 75 ; F = 64 ... . Fq;1


Fq;q where3Fi;k : X ! X for i; k = 1; : : : ; q. Consider x = [x1 ; : : : ; xm ] =

2 6 4

x1 .. .

xq

7 5

= [xk;` ], 1  k  q, 1  `  m, where xj = [x1;j ; : : : ; xq;j ]> for

166

3. IMPROVEMENT OF ACCURACY

j = 1; : : : ; m and x i = [xi;1 ; : : : ; xi;m ] for i = 1; : : : ; q. Then we have 2

Fx=

q P

6 k=1 6 6 6 6 q 4P

k=1

F1;k xk;1


.. .

Fq;k xk;1




q P

k=1 q P k=1

3

F1;k xk;m 7 .. .

Fq;k xk;m

7 7 7 7 5

2

F1;1


F1;q 3 2 x 1 3

= 64 ...

.. .

Fq;1


Fq;q

7 6 5 4

.. .

xq

7 5

;

where Fi;k : X ! X is the natural extension of Fi;k : In particular, if T 2 BL(X ); and 2 6 6 6 6 6 6 6 6 6 6 6 4

T

O







O3

I

O







O

I

O

T := O

as before, then

2

T

O

I

O

Tx= O

I

6 6 6 6 6 6 6 6 6 6 6 4

.. .

...

.. .

I

O

O


O







O 32





O O

.. .

... ...

O




O

If Z 2 Cj mm , we de ne




O

... ...




O ...

.. .

I

O

2

x 1Z

7 7 76 76 76 76 76 76 74 7 7 5

x1 .. . .. .

xq

7 7 7 7 7 7 7 7 7 7 7 5

3

2

3

7 7 7 7 7 7 5

6 . = 66 .. 6 . 4 ..

7 7 7 7 5

T x1 6 x1 7 6 7 x q ;1

:

3

x Z := 64 ... 75 : x qZ Let  be a spectral set for T such that 0 62 , and let C 2 C (T; ) such that 0 2 ext(C): Consider the map J = J (T; ) from X to X introduced just before stating Proposition 3.9: For x 2 X , Jx = [J0 x; J1 x; : : : ; Jq;1 x]> ;

167

3.2. ACCELERATION

Z ; 1 where J = P := P (T; ) and Jk x = 2i R(T; z ) dz z k , k = 0; : : : ; q ; 1. Let J : X ! X denote the natural extension of J to X . Thus J x = [Jx ; : : : ; Jxm ] for x = [x ; : : : ; xm ] 2 X : Clearly, 2 3 2 Px 3 Px


Pxm 0

C

1

1

J x = 64

.. .

1

.. .

Jq;1 x1


Jq;1 xm

7 5

= 64

.. .

Jq;1 x

7 5

:

For Z 2 Cj mm , we have 2

P ( x Z)

3

2

3

( P x )Z 6 7 6 7 . .. . J ( x Z) = 4 5=4 5 = ( J x )Z: . . Jq;1 ( x Z) ( Jq;1 x )Z

Proposition 3.14 Consider T 2 BL(X ) and Z 2 Cj mm . (a) Let  be a spectral set for T such that 0 62  and J := J (T; ). Then

T J = J T: In particular, if x 2 X and T x = x Z, then T ( J x ) = ( J x )Z: 2

x1

(b) Let x = 64 ...

xq

3 7 5

2 X . Then P x = J x . Also, T x = x Z if 1

and only if T x 1 = x 1 Z and x k+1 Zk = x 1 for k = 1; : : : ; q ; 1:

Proof (a) Let x 2 X . Since TP = PT and Jk T = Jk; for k = 1; : : : ; q ; 1; we obtain

TJx=T

1

2 6 6 6 4

Px J1 x .. .

Jq;1 x

3

2

TPx3 6 P x 7 6 7

7 7 7 5

= 64

.. .

Jq;2 x

7 5

2 6

= 664

PTx J1 T x .. .

Jq;1 T x

3 7 7 7 5

= J T x:

168

3. IMPROVEMENT OF ACCURACY

Thus T J = J T : If T x = x Z; then

T ( J x ) = J ( T x ) = J ( x Z) = ( J x )Z: (b) Recalling the matrix representation of P given in Proposition

3.9(b), we have

2 6

P O


O 3 2 x13 J1 O


O 77 66 ... 77

P x = 664 .. . Jq;1 Also, we note that

.. .

.. .

O


O 2

7 6 5 4

xq

7 5

2 6

= 664

3

T x1 6 x1 7 6 7

3

P x1 J1 x 1 .. .

Jq;1 x 1 2

7 7 7 5

= J x 1:

3

x 1Z 6 x 2Z 7 6 7

T x = 64 .. 75 and x Z = 64 .. 75 : . . x q ;1 x qZ Hence T x = x Z if and only if T x 1 = x 1 Z; x 1 = x 2 Z; : : : ; x q;1 = x q Z; that is, T x 1 = x 1 Z and x k+1 Zk = x k Zk;1 =


= x 1 for k = 1; : : : ; q ; 1: Let  be a spectral set of nite type for T such that 0 62 , and let the rank of the associated spectral projection P be m. Let ' := ['1 ; : : : ; 'm ] form an ordered basis for the spectral subspace M := R(P ). Then T ' = '  for some  2 Cj mm . Since sp() =  and 0 62 ; we see that the matrix  is nonsingular. Fix an integer q  2: By Proposition 3.9(b),  is a spectral set for T , the rank of the associated spectral projection P is m, and 2

' := J ' = [J'1 ; : : : ; J'm ] =

6 6 6 6 6 4

P' J1 ' .. .

Jq;1 '

3 7 7 7 7 7 5

forms an ordered basis for the associated spectral subspace M := R(P ). Also, by Proposition 3.14(a),

T ' = T ( J ' ) = ( J ' ) = ' ;

169

3.2. ACCELERATION

and by Proposition 3.14(b), Jk ' k = P ' = ' for k = 1; : : : ; q ; 1: Thus 3 2

'=

6 6 6 6 6 4

'

' ;1 .. .

' ;q+1

7 7 7 7 7 5

:

 O and consider C 2 C (T; ) such that 0 2 Assume that (Tn ; T ) ! ext(C): Then by Theorem 3.12(c), there is a positive integer n0;q such that for each n  n0;q ; nq := sp(Tn ) \ int(C) is a spectral set for Tn and the associated spectral projection Pn is of rank m. Let ' n form a basis for the associated spectral subspace Mn := R(Pn ). Then Tn 'n = ' n nq for some nq 2 Cj mm : Since sp(nq ) = nq  int(C)2and 0 32 ext(C); [ ]

[ ]

[ ]

[ ]

[ ]

'n;q

[ ] 1

we see that the matrix nq is nonsingular. Let 'n = 64 ... [ ]

q 'n;q

7 5

:

[ ]

Since

Tn ' n

2

6 6 6 6 6 6 = 666 6 6 6 6 4

Tn ( T ; Tn ) Tn






I

O






O

I

.. .

O




( T ; Tn )q; Tn 32 'n;q 1






O

O






O

...

...

...

.. .






O

I

O

and

2

' n nq

[ ]

6 6 = 666 4

'n;q nq [ ] 1

[ ]

.. . q q 'n;q n [ ]

[ ]

3 7 7 7 7 7 5

;

76 76 76 76 76 76 76 76 76 76 76 76 74 5

[ ] 1

.. . .. . .. .

q 'n;q [ ]

3 7 7 7 7 7 7 7 7 7 7 7 7 5

170

3. IMPROVEMENT OF ACCURACY

q we have 'n;q = 'n;q nq , 'n;q = 'n;q nq ; : : : ; 'n;q ; q Hence 'n;k = 'n;q (nq );k for k = 1; : : : ; q ; 1. Write in order to simplify the notation slightly. Then [ ] 1

[ ]

+1

[ ] 2

[ ] 2

[ ]

[ ] 1

[ ] 3

[ ]

[ ]

[ ]

2

'n =

6 6 6 6 6 4

q q : = 'n;q n 'nq := 'n;q [ ]

[ ]

[ ] 1

[ ]

3

'nq

[ ]

'nq (nq );1 [ ]

1

[ ]

.. .

'nq (nq );q+1

7 7 7 7 7 5

:

[ ]

[ ]

As in Section 2.4, consider

1 (C) := supfkR(; z )k : z 2 Cg; 2;n;q (C) := supfkR(nq ; z )k : z 2 Cg: [ ]

We now obtain improved error estimates for approximations of the ordered basis ' for M .

Theorem 3.15

Under the hypotheses of Theorem 3.12(c) and with the notation stated above, we have the following:

(a) For each large n, P 'nq forms an ordered basis for the spectral subspace M of T and

[ ]

k 'nq ; P 'nq k1 [ ]

[ ]

q ; q; q q q ;  `2(C)   ;q (C) ;n;q (C)k(n ) k k( T ; Tn ) Tn 'n (n ) k1 : 1

[ ]

2

1

1

[ ]

[ ]

1

1

If the sequence k 'nq k1 is bounded, then [ ]

k( T ; Tn )q Tn 'nq k1 ! 0: Let ' n = [' n; ; : : : ; ' n;m ] and n;q := j min dist(' n;j ; spanf'n;i : i 6= j; i = 1; : : : ; mg): ;:::;m [ ]

1

=1

If the sequence k 'n k1 is bounded and the sequence (n;q ) is bounded away from zero, then the sequences ( 2;n;q (C)) and (k(nq );1 k ) are bounded, and in particular, k 'nq ; P 'nq k1 = O(k(T ; Tn)q Tn k) as n ! 1: [ ]

[ ]

[ ]

1

171

3.2. ACCELERATION

(b) For each large n; Pn' forms an ordered basis for Mn and Pn ' = ' n Cn;q for some nonsingular mm matrix Cn;q . If ' qn := ' C;n;q1 ; [ ] ( )

then ' qn forms an ordered basis for M and [ ] ( )

k 'nq ; ' qn k1 `(C) m q ;q k ' qn k1  2 q (C) ;q (C) n;q k( T ; Tn ) '  k1 ; [ ] ( )

[ ]

1

[ ] ( )

+1

2

where

q := j=1min dist(J'j ; spanfJ'i : i 6= j; i = 1; : : : ; mg); ;:::;m n k J1 ' qn k1 k Jq;1 ' qn k1 o

n;q := max 1; ; : : : ; k ' qn k1 k ' qn k1  maxfkCn;q ;k C;n;q1 k : k = 0; : : : ; q ; 1g: [ ] ( )

[ ] ( )

[ ] ( )

[ ] ( )

1

In particular,

k 'nq ; ' qn k1 q k ' qn k1 = O(k(T ; Tn ) T k) as n ! 1: [ ] ( )

[ ]

[ ] ( )

Proof (a) Since Tn ! T by Proposition 3.11, Theorem 2.18(c) shows that

for each large n; P ' n forms an ordered basis for M : Now by Proposition 3.14(b), we have 2 6

P ' n = J 'nq = 664 [ ]

P 'nq J1 'nq

3

[ ] [ ]

.. .

Jq;1 'nq

7 7 7 5

:

[ ]

We show that P 'nq forms an ordered basis for the m-dimensional subspace M := R(P ): Let P 'nq = [x1 ; : : : ; xm ] and c1 x1 +


+cm xm = 0 for some c1 ; : : : ; cm in Cj . Then for each k = 21; : : : ; q ; 1, Jk 'nq 3= [ ]

[ ]

[ ]

Jk P 'nq = [Jk x1 ; : : : ; Jk xm ], so that P ' n = [ ]

6 6 6 4

x1


xm J1 x1


J1 xm .. .

.. .

Jq;1 x1


Jq;1 xm

7 7 7 5

:

172

3. IMPROVEMENT OF ACCURACY

Let xj := [xj ; J1 xj ; : : : ; Jq;1 xj ]> for j = 1; : : : ; m: Then c1 x1 +


+ cm xm = [0; J1 0; : : : ; Jq;1 0]> = 0. Since [x1 ; : : : ; xm ] forms a basis for M , we have c1 =


= cm = 0; as required. Applying Proposition 2.21 to T and Tn in place of T and Tn , we obtain k ' n ; P ' n k1  `2(C)  1;q (C) 2;n;q (C)k( Tn ; T ) ' n k1 : Clearly,

k 'nq ; P 'nq k1  k 'n ; P ' n k1 : Also, since Tn 'n = ' n nq , we have 2 32 'nq O


O ;( T ; Tn )q Tn [ ]

[ ]

[ ]

3

[ ]

( Tn ; T ) '

6 6 6 n =6 6 4 2 6 6 =666 4

O


O .. .

76 76 76 76 76 54

O

.. .

.. .

'nq ([nq] );1 [ ]

.. .

O O


O 'nq (nq );q+1 3 ;( T ; Tn )q Tn 'nq (nq );q [ ]

[ ]

7 7 7 [q] ;1 7(n ) 7 5

[ ]

[ ]

7 7 7 7 7 5

O .. .

O

;

so that k( Tn ; T ) ' n k1  k( T ; Tn )q Tn 'nq (nq );1 k1 k(nq );1 kq;1 : Hence [ ]

[ ]

[ ]

1

k 'nq ; P 'nq k1 q ; q; q q q ;  `2(C)   ;q (C) ;n;q (C)k(n ) k k( T ; Tn ) Tn 'n (n ) k1 ; [ ]

[ ]

1

[ ]

2

1

1

1

[ ]

[ ]

1

as desired. If k 'nq k1  , then [ ]

k( T ; Tn )q Tn 'nq k1  k(T ; Tn )q Tn k ! 0: [ ]

q denote the (i; j )th entry of the matrix  q for 1  i; j  m: Let n;i;j n Since Tn 'n = ' n nq , we have for each j = 1; : : : ; m; [ ]

[ ]

[ ]

q ' ; Tn' n;j = n;q ;j ' n;1 +


+ n;m;j n;m [ ] 1

[ ]

173

3.2. ACCELERATION

so that for i = 1; : : : ; m; q j  kT ' k  k T ' k : n;q jn;i;j n n;j 1 n n 1 [ ]

It follows that

knq k = j max ;:::;m [ ]

1

=1

m X i=1

q j  m kT k k ' k ; jn;i;j n 1 n 1  [ ]

n;q

where (kTn k1 ) is a bounded sequence. Assume now that the sequence (k ' n k1 ) is bounded and the sequence (n;q ) is bounded away from zero. Then the sequence (k q k ) is bounded. Also, since sp(nq ) = nq  int(C) and 0 2 ext(C); we see that [ ]

[ ]

1

[ ]

jnq j  (C) > 0 [ ]

for every eigenvalue nq of nq . Hence by Kato's result (2.20), the sequence (k(nq );1 k ) is bounded. Further, we note that the sequence  sup knq ; z Ik is bounded. By Corollary 1.22, there is a Cauchy z2C contour Ce separating  from (sp(T ) n ) and also from C [ f0g. Let e Theorem 3.12(a)  := dist(Ce ; C) > 0. Since Ce 2 C (T; ) and 0 2 ext(C), q e shows that n = sp(Tn ) \ int(C) for all suciently large n. Thus for each eigenvalue nq of nq and each z 2 C; we have [ ]

[ ]

[ ]

[ ]

1

1

[ ]

[ ]

[ ]

jnq ; z j  dist(sp(nq ); C) = dist(nq ; C) >  > 0: [ ]

[ ]

[ ]

Again by Kato's result, it follows that the sequence ( 2;n;q (C)) is bounded. Consequently,

k 'nq ; P 'nq k1 = O(k(T ; Tn)q Tn k): [ ]

[ ]

(b) As in (a) above, Theorem 2.18 shows that for each large n, Pn ' forms an ordered basis for Mn . Since ' n also forms an ordered basis for Mn , there is a nonsingular mm matrix Cn;q such that Pn ' n = 'n Cn;q : De ne 2 6

' C;n;q1

6 ;1 '(n) := ' C;n;q1 = 666 J1 '. Cn;q 4

..

Jq;1 ' C;n;q1

3 7 7 7 7 7 5

174

3. IMPROVEMENT OF ACCURACY

and ' qn := ' C;n;q1 . As ' forms an ordered basis for M and the matrix Cn;q is nonsingular, it follows that ' qn also forms an ordered basis for M. Applying Proposition 2.21 to T and Tn in place of T and Tn , and noting that T ' = '  with ' = [J'1 ; : : : ; J'm ], we obtain k ' n ; ' n k1 `(C) m k ' n k1  2 q 1 (C)2;q (C)k( Tn ; T ) ' k1 ; where q = j=1min dist(J'j ; spanfJ'i : i 6= j; i = 1; : : : ; mg): ;:::;m Clearly, [ ] ( )

[ ] ( )

( )

( )

k 'nq ; ' qn k1  k 'n ; ' n k1 : [ ] ( )

[ ]

( )

Also, since ( Jk ' )C;n;q1 = Jk ( ' C;n;q1 ) = Jk '[(qn]) for k = 1; : : : ; q ; 1; we have

n

o

k ' n k1 = max k ' qn k1 ; k J ' qn k1 ; : : : ; k Jq; ' qn k1 = k ' qn k1 n;q ; n k J ' qn k1 k Jq; ' qn k1 o where n;q := max 1; k ' qn k1 ; : : : ; k ' qn k1 . Further, since for k = 0; : : : ; q ; 1; [ ] ( )

( )

1

[ ] ( )

1

[ ] ( )

[ ] ( )

1

[ ] ( )

1

[ ] ( )

[ ] ( )

[ ] ( )

Jk ' qn = ( Jk ' )C;n;q1 = '  ;k C;n;q1 = ' C;n;q1 (Cn;q ;k C;n;q1 ) = ' qn (Cn;q  ;k C;n;q1 ); [ ] ( )

[ ] ( )

we have





n;q  max kCn;q ;k C;n;q1 k : k = 0; : : : ; q ; 1 : 1

Next, ( Tn ; T ) ' = ( Tn ; T ) P ' 2 ;( T ; Tn )q Jq;1 O


O 3 6

= 66 4

O .. .

O


O 77 .. .

.. .

O


O O q ;( T ; Tn ) Jq;1 ' 3 6 7 O 7 = 664 7: .. 5 . O 2

2

6 6 7 6 5 4

'

J1 ' .. .

Jq;1 '

3 7 7 7 5

175

3.2. ACCELERATION

Hence k( Tn ; T ) ' k1 = k( T ; Tn )q ' ;q+1 k1 and we obtain

k 'nq ; ' qn k1 `(C) m q ;q k ' qn k1  2 q (C) ;q (C) n;q k( T ; Tn ) '  k1 : But n;q  maxf1; kJ k; : : : ; kJq; kg and k( T ; Tn )q ' ;q k1 = k( T ; Tn )q T ' ;q k1  k(T ; Tn)q T k k ' k1 k; kq ; [ ] ( )

[ ]

1

[ ] ( )

1

+1

2

1

+1

1

so that

1

k 'nq ; ' qn k1 q k ' qn k1 = O(k(T ; Tn) T k) as n ! 1; [ ] ( )

[ ]

[ ] ( )

as desired.

Remark 3.16

We observe that if m = 1,  = fg and C is the circle C of radius  with center  in the preceding theorem, then we obtain Theorem 3.13. To see this, we note the following: `(C)=2 = , 2;n;q = supf1=jnq ; z j : z 2 C g = 1= dist(nq ; C), nq = [nq ], k(nq );1 k = 1=jnq j;  = [], 1 (C) = supf1=j ; z j : z 2 C g = 1=, q = kJ'k1 = k'k maxf1; 1=jj; : : :; 1=jjq;1 g, Cn;q = [cn;q ], Jk ' qn = '=cn;q k for k = 0; : : : ; q ; 1 and n;q = maxf1; 1=jj; : : : ; 1=jjq;1 g: Also, if q = 1 in the preceding theorem, then we obtain Proposition 2.9. To see this we note the following: k( T ; Tn ) Tn 'n ;n 1k1 = k( T ; Tn ) 'n k1 ; q =  and n;q = 1: [ ]

[ ]

[ ]

[ ]

[ ]

1

[ ] ( )

Remark 3.17

 T , then In Remark 2.16, we have noted that if Tn ! gap(M; Mn ) := maxf(M; Mn); (Mn ; M )g  maxfk(Pn ; P )jM k; k(Pn ; P )jMn kg; where, by Proposition 2.15, we have for all large n, k(Pn ; P )jM k  `2(C)  1 (C)2 (C)k(Tn ; T )jM k; k(Pn ; P )jMn k  `2(C)  1 (C)2 (C)k(Tn ; T )kjMn k:

[ ]

176

3. IMPROVEMENT OF ACCURACY

 O; then T !  Similarly, it follows that if (Tn ; T ) ! n T and if

gap(M ; Mn ) := maxf(M ; Mn ); (Mn ; M )g; then for all large n,

(M ; Mn )  k(Pn ; P )jM k  `2(C)  1;q (C)2;q (C)k(Tn ; T )jM k; (Mn ; M )  k(Pn ; P )jMn k  `2(C)  1;q (C)2;q (C)k(Tn ; T )jMn k:

We have seen in Theorem 3.12 that

k(Tn ;T )jM k = O(k(Tn ;T )q T k) and k(Tn ;T )jMn k = O(k(Tn ;T )q Tn k): Let us de ne Mn;q to be the subspace of X consisting of all the rst components of the elements of Mn , and investigate gap(M; Mn;q ) := maxf(M; Mn;q ); (Mn;q ; M )g: To relate (M; Mn;q ) with (M ; Mn ); consider x 2 M with kxk = 1: Then Jx = [x; J1 x; : : : ; Jq;1 x]> 2 M : Now dist(x; Mn;q ) = inf fkx ; xn;q k : xn;q 2 Mn;q g  inf fkJx ; xn k1 : xn 2 Mn g 



Jx ; M :  kJxk1 dist kJx k1 n

Hence

(M; Mn;q ) := supfdist(x; Mn;q ) : x 2 M; kxk = 1g  kJjM k supfdist(x; Mn ) : x 2 M ; kxk1 = 1g = kJjM k(M ; Mn ): It is not easy to nd a bound for (Mn;q ; M ) in terms of (Mn ; M ).

We refer to Theorem 3.5 of [14] for a way out.

3.2.4 Dependence on the Order of the Spectral Analysis

In our development of the acceleration technique, we have so far xed an integer q  2 and considered spectral analysis of order q. It is interesting to investigate whether this analysis depends on the choice of q and, if so, in what manner.

177

3.2. ACCELERATION

 O: It follows from Proposition 3.11 Let T 2 BL(X ) and (Tn ; T ) !  q q that Tn ! T as n ! 1; uniformly in q 2 f2; 3; : : :; g; that is, (i) (kTnq k) is bounded in n and q, and (ii) k(Tnq ; T q )T q k as well as k(Tnq ; T q )Tnq k tend to zero as n ! 1; uniformly in q 2 f2; 3; : : :g: Let E be a closed subset of re(T ) such 0 62 E . De ne [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

(E ) := minfjz j : z 2 E g; 1 (E ) := supfkR(T; z )k : z 2 E g and for q = 2; 3; : : : 1;q (E ) := supfkR(T q ; z )k : z 2 E g: Let us rst consider the case when (E ) > 1: The reason for this restriction will soon be clear. The matrix representation of R(T q ; z ) given earlier shows that for all z 2 E , kR(T q n; z )k z )k + 1 ;


; kR(T; z )k + 1 +


+ 1 o  max kR(T; z )k; kR(jT; zj jz j jz jq;1 jz jq;1 jz j o n  1  1 1 (E ) 1 (E )  max 1 (E ); (E ) + (E ) ; : : : ; [(E )]q;1 + [(E )]q;1 +


+ (1E )  1 (E ) + [(E1)]q;1 +


+ (1E ) [ ]

[ ]

[ ]

< 1 (E ) + (E1) ; 1 :

Let 1;1 (E ) := 1 (E ) + 1=[(E ) ; 1]: Then the sequence (1;q (E )) is bounded by 1;1 (E ). [On the other hand, if (E )  1, then the sequence (1;q (E )) may not be bounded, as the simple example in Exercise 3.11 shows.] The proof of Theorem 2.6 shows that there is a positive integer n0 (independent of q) such that for each n  n0 and each q 2 f2; 3; : : :; g, E  re(Tnq ): As a consequence, we note that if n 2 [fsp(Tnq ) : q = 2; 3; : : :g and n !  as n ! 1; where jj > 1; then  2 sp(T ): (See the proof of Corollary 2.7.) Further, de ne [ ]

[ ]

2;q (E ) := supfkR(Tnq ; z )k : z 2 E; n  n0 g: [ ]

As in the proof of Theorem 2.6, we have

2;q (E )  21;1(E )[1 + t 1 (E )];

178

3. IMPROVEMENT OF ACCURACY

where kT q ; Tnq k  t for all n and q. Let 2;1 (E ) := 21;1 (E )[1 + t 1 (E )]: Then the sequence (2;q (E )) is bounded by 2;1 (E ): Let now  be a nonempty spectral set for T such that 0 62 ; P := P (T; ) and P q := P (T q ; ): Assume rst that minfjj :  2 g > 1: By Proposition 1.29, nd C 2 C (T; ) such that fz 2 Cj : jz j  1g  ext(C): Then (C) > 1 and there is a positive integer n0 such that C  re(Tnq ) for all n  n0 and all q = 2; 3; : : : As we proved in Proposition 2.9, if n  n0 and q = 2; 3; : : : then [ ]

[ ]

[ ]

[ ]

[ ]

nq := sp(Tnq ) \ int(C) and Pnq := P (Tnq ; nq ) do not depend on C 2 C (T; ) provided (C) > 1; and k(Pnq ; P q )P q k as well as k(Pnq ; P q )Pnq k tend to zero as n ! 1; uniformly in q 2 f2; 3; : : :g: As a consequence, there is a positive integer n1 such that for all n  n1 and all q = 2; 3; : : : ; (Pnq ; P q ) < 1, so that rank Pnq = rank P q = rank P and, in particular, nq 6= ;: Since for all q = 2; 3; : : : [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

1;q (C)  1;1 (C) < 1 and 2;q (C)  2;1 (C) < 1; we see that the conclusions of Theorem 3.12 hold true if we replace n0;q by n0 , 1;q (C) by 1;1 (C) and 2;q (C) by 2;1 (C). In case the spectral set  is of nite type, then for all large n and all q = 2; 3; : : : we obtain `(C)2;1 (C) k(T ; T )q T k; jbnq ; bj  `(C)  1;1 (C) n n  2(C) [ ]

so that jbnq ; bj = O(k(Tn ; T )q Tn k) as n; q ! 1; and also [ ]

[`(C) (C)] q jbnq ; bj  `(C)   ;1 (C) 4 [(C)]q k(Tn ; T ) Tn k; [ ]

2

1

2

2

so that jbnq ; bj = O(k(Tn ; T )q T k=[(C)]q as n; q ! 1: Let us now turn to the general case of a spectral set  of T such that 0 62 . Since  is closed, there is a positive number s such that [ ]

minfjj :  2 g > s: Again by Proposition 1.29, nd C 2 C (T; ) such that (C) > s: As we have mentioned before, the sequences (1;q (C)) and (2;q (C)) may not

179

3.2. ACCELERATION

be bounded if s  1: We may, however, multiply the operators T and Tn by the positive number r := 1=s and apply our earlier considerations to  O; r := fr :  2 g is a the operators rT and rTn . Now (rTn ;rT ) ! spectral set for T and minfjj :  2 rg = r minfjj :  2 g > rs = 1: Also, rC 2 C (rT; r); 0 2 ext(rC) and Z Z P (rT; r) = ; 21 i R(rT; w)dw = ; 21 i R(T; z )dz = P (T; ): rC

C

For x := [x1 ; : : : ; xq ]> 2 X q , let [ ]

diag[1; r; : : : ; rq;1 ]x := [x1 ; rx2 ; : : : ; rq;1 xq ]> :

Consider the operators

;
;
r? T q := diag[1; r; : : : ; rq;1 ] ;1 rT q diag[1; r; : : : ; rq;1 ] [ ]

[ ]

and

r? Tnq := diag[1; r; : : : ; rq;1 ] ;1 rTnq diag[1; r; : : : ; rq;1 ] from X q to X q . It can be easily seen that if we replace T by rT , and Tn by rTn in the de nitions of the operators T q and Tnq , then we obtain the operators r? T q and r? Tnq . Since similar operators have ;

[ ]

[ ]




;

[ ]




[ ]

[ ]

[ ]

[ ]

[ ]

the same spectrum, we have

sp(r? T q ) = fr :  2 sp(T q )g and sp(r? Tnq ) = fr :  2 sp(Tnq )g: [ ]

[ ]

Also, we see that

P (r? T q ; r) = 2;1i

Z

[ ]

R(r? T q ; w)dw [ ]

[ ]

Z = 2;1i

[ ]

rC

diag[1; r; : : : ; rq;1 ] ;1 R(rT q ; w) diag[1; r; : : : ; rq;1 ]dw rC   Z ;
; 1 q q ; 1 ;1 q ;1 = diag[1; r; : : : ; r ] 2i C R(T ; z )dz diag[1; r; : : : ; r ]dz: Thus ;




[ ]

[ ]

P (r? T q ; r) = diag[1; r; : : : ; rq;1 ] ;1P (T q ; ) diag[1; r; : : : ; rq;1 ]: Similarly, there is an integer n0 (r) such that for all n  n0 (r), we have ;

[ ]




[ ]

;
P (r? Tnq ; rnq ) = diag[1; r; : : : ; rq;1 ] ;1P (Tnq ; nq ) diag[1; r; : : : ; rq;1 ]: [ ]

[ ]

[ ]

[ ]

180

3. IMPROVEMENT OF ACCURACY

As a result, an element x = [x1 ; : : : ; xq ]> in X q belongs to the range of P (r? Tnq ; rnq ) if and only if P (Tnq ; nq )[x1 ; rx2 ; : : : ; rq;1 xq ]> = [x1 ; rx2 ; : : : ; rq;1 xq ]> ; that is, the element [x1 ; rx2 ; : : : ; rq;1 xq ]> belongs to the range of P (Tnq ; nq ). In particular, nq is a nonzero spectral value of Tnq of algebraic multiplicity m if and only if rnq is a nonzero spectral value of rTnq of algebraic multiplicity m. As in Theorem 3.12(c), let  be a spectral for T of nite type, 0 62  and b denote the weighted arithmetic mean of the spectral values of T belonging to . Then rb is the weighted arithmetic mean of the spectral values of rT belonging to r. Also, it follows from the considerations given above, that if rbnq is the weighted arithmetic mean of the spectral values of r? Tnq belonging to rnq := sp(r? Tnq ]) \ int(rC); then bnq is the weighted arithmetic mean of the spectral values of Tnq belonging to nq := sp(Tnq ) \ int(C). Applying Theorem 3.12(c) to rT and rTn , we obtain for all large n and q = 2; 3; : : : bq (rC) q jrbnq ; rbj  `(rC) aq (rC) `(r2C) (rC) k(rTn ; rT ) rTn k as well as a1 (rC)]2 k(rT ; rT )q rT k; jrbnq ; rbj  `(rC) bq (rC) [`4(rC) n 2 [ (r C)]q where aq (rC) := supfkR(r? T q ; w)k : w 2 rCg; bq (rC) := supfkR(r? Tnq ; w)k : n  n0 (r); w 2 rCg; a1 (rC) := supfkR(rT; w)k : w 2 rCg: [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ] [ ]

Note that `(rC) = r`(C), (rC) = r(C) > rs = 1 and a1 (rC) = 1 (C)=r: As the sequences (aq (rC)) and (bq (rC)) are bounded in q, there is an integer n1 (r) such that for all n  n1 (r) and all q = 2; 3; : : :

jbnq ; bj  sq; minfk(Tn ; T )q Tn k; k(Tn ; T )q T kg; [ ]

1

where  is a constant independent of n and q. We observe that if s > 1; then the estimate jbnq ; bj = O(k(Tn ; T )q Tn k=sq;1) (as n; q ! 1) obtained above is an improvement over the estimate jbnq ; bj = O(k(Tn ; T )q Tn k) (as n; q ! 1) stated earlier, while the estimate jbnq ; bj = O(k(Tn ; T )q T k=sq;1)(as n; q ! [ ]

[ ]

[ ]

181

3.3. EXERCISES

1) obtained above matches with the estimate jbnq ; bj = O(kTn ; T )q T k=[(C)]q )(as n; q ! 1) stated earlier. [ ]

For uniform estimates for the accelerated approximation of an eigenvector of T corresponding to a nonzero simple eigenvalue, see Exercises 3.12 and 3.13. Similar considerations hold for uniform estimates for the accelerated approximation of an ordered basis of the spectral subspace of T corresponding to a spectral set of nite type not containing zero.

3.3 Exercises

Unless otherwise stated, X denotes a Banach space over Cj , X 6= f0g and T 2 BL(X ). Prove the following assertions.

3.1 Let A 2 BL(X ) be invertible and Ae 2 BL(X ) satisfy (A;1(A ; Ae)) < 1. Then Ae is invertible. If, in addition, (Ae;1 (Ae ; A)) < 1, then A;1 =

1h X j =0

ij

Ae;1 (Ae ; A) Ae;1 : ;




Let y 2 X: If Aexe = y, xe(0) := xe and xe(k) := xe(k;1) + Ae;1 y ; Axe(k;1) h ik for k = 1; 2; : : : then xe(k) = Ae;1 (Ae;1 ; A) Ae;1 xe, and hence xe(k) ! x in X and Ax = y: 3.2 Let X be a Hilbert space with an orthonormal basis fe~; e1; e2; : : : ; g and consider a diagonal operator Te 2 BL(X ) given by ehx ; e e :=  Tx eiee +

1 X j =1

`j hx ; ej iej ; x 2 X:

Let Ve := T ; Te. Then the coecients 1 , 1 , 2 in the RayleighSchrodinger series associated with T , Te and e are given by 1 ee X

1 hVe ee ; e ihVe e ; eei hV e ; ej i e ;  = ; X j j : j 2 e e `j ;  j =1 `j ;  j =1 1 hx ; ej i P e = hx ; e e = (Hint: Px eiee and Sx ej for x 2 X .) j =1 `j ; e

1 = hVe ee ; eei;

1 =;

182

3. IMPROVEMENT OF ACCURACY

3.3 The rst iterate 'n1 in the Elementary Iteration is given by 'n1 = 'n ; Sn T'n ; ( )

( )

and the rst iterate n(1) in the Double Iteration is given by 



T'n Sn T hT 2 'n ; 'n i n = hT'n ; ' i + hT'n ; ' i hT'n ; ' i 'n ; T'n ; n n n provided hT'n ; 'n i 6= 0: 3.4 Let Tn 2 BL(X ) for n = 1; 2; : : : (a) Let  be a nonzero simple eigenvalue of T , n and 'n be as in Lemma 3.1, and n1 and 'n1 be the rst iterates in the Elementary iteration (E ). If Tn (T ; Tn )'n = 0, then n1 = n and 'n1 = T'n=n . (b) Let  be a spectral set for T such that 0 62 , n and 'n be as in Lemma 3.5, and n1 and 'n 1 be the rst iterates in the Elementary Iteration ( E ). If Tn ( T ; Tn ) 'n = 0 , then n1 = n and 'n 1 = ( T 'n );n 1 . (Hint: Remarks 1.25, 1.58. Note: These results hold if Tn is the Galerkin approximation of T as de ned in Section 4.1.) 3.5 Let ' , ' , 'n, 'n and Cn be as in Lemma 3.5 and Lemma 3.6. Let Dn = 'n ; ' . Then the sequence (kC;n 1D;n 1 k1) is bounded. Further, the sequence (k 'n k1 ) is bounded if and only if the sequence (kC;n 1 k1 ) is bounded, while the sequence (k 'n k1) is bounded if and only (1)

( )

( )

( )

( )

( )

( )

( )

( )

if the sequence (kD;n 1 k1 ) is bounded.

3.6 Let 0 6=  2 Cj . Then the ascent of  as an eigenvalue of T is equal to the ascent of  as an eigenvalue of T 2 BL(X ). (Hint: If P := P (T; fg), P := P (T ; fg); D = P (T ; I ) and D = P (T ; I), k then for k = 1; 2; : : :Dk = O if and only if D = O.)

3.7 Let q  2. Then N (T ) = f[0; : : : ; 0; x]> 2 X : x 2 X g, so that 0

is an eigenvalue of T even if 0 may not be an eigenvalue of T . If 0 is an eigenvalue of T and ' is a corresponding eigenvector, then ['; 0; : : : ; 0]> is not an eigenvector of T corresponding to 0; but [0; : : : ; 0; ']> is an eigenvector of T corresponding to 0. n n 3.8 If Tn ! T , then Tnq ! T q , uniformly for q = 2; 3; : : : 3.9 Let C be a Cauchy contour Z in re(T ) with 0 2 ext(C). For k = 1 1m and  2 0; 1; 2; : : : consider Jk := ; 2i R(T; z ) dz z k : Let ' 2 X [ ]

[ ]

C

183

3.3. EXERCISES

Cj mm satisfy T ' = ' . Then ( Jk ' )k = J0 ' , where J0 ' = ' if sp()  int(C) and J0 ' = 0 if csp()  ext(C): In particular, if m = 1, ' = ',  = [] and  6= 0, then Jk ' = '=k if  2 int(C) and Jk ' = 0 if  2 ext(C).

3.10 Under the hypotheses and with the notation of Theorem 3.15, let e1;q (C) := supfkR(T q ; z )jM k : z 2 Cg e2;q (C) := supfkR(Tnq ; z )jMn k : z 2 C; n  n0;q g: [ ] [ ]

Then

k 'nq ; P 'nq k1 `(C)    e2;q (C) max 1; k(nq );q+1 1;q (C) q 2 k 'n k1   ` (C)  2;q (C) q  2 (C) k(Tn ; T ) Tn k ; k 'nq ; ' qn k1 `(C) k ' qn k1  2 e1;q (C)2;q (C) n;q [ ]

[ ]

[ ]

[ ]

[ ] ( )

[ ]

[ ] ( )





2 1 k(T ; T )q T k:  `(C)21 (C) [(C)] q n

(Hint: Proposition 2.21, Theorem 3.12 (b), Theorem 3.15.)

3.11 Fix c 2 Cj , let X := Cj and de ne Tx := c x for x 2 X . For q = 1; 2; : : : x = [x1 ; : : : ; xq ]> 2 X q and z 2 Cj n fc; 0g; we have [ ]

 > R(T q ; z )x = c x;1 z ; z (cx;1 z ) ; xz2 ; : : : ; z q;1x(c1 ; z ) ; zxq;2 1 ;


xzq : [ ]

If 0 < jz j < 1 and z 6= c, then kR(T q ; z )k  1=jz jq;1jc ; z j; which tends to 1 as q ! 1. Also, if c 6= 1; then kR(T q ; 1)k  (q ; 1) ; 1=jc ; 1j, which tends to 1 as q ! 1. [ ]

[ ]

3.12 Let  be a nonzero simple eigenvalue of T , and nq , 'nq be as in Theorem 3.13. Then there is some n0 such that for all n  n0 and all [ ]

[ ]

q = 2; 3; : : :P'nq is an eigenvector of T corresponding to , and [ ]

k'nq ; P'nq k  k(T ; nq I )'nq k; [ ]

[ ]

[ ]

where is a constant independent of n and q.

[ ]

184

3. IMPROVEMENT OF ACCURACY

3.13 Let  be a nonzero simple eigenvalue of T and jj > s > 0.  Assume that (Tn ;T ) ! O: Let 0 <  < minfjj;s; dist(; sp(T )nfg)g. Then there is n0 such that for each n  n0 and q = 2; 3; : : : there is a unique nq 2 sp(Tnq ) satisfying jnq ; j < . Also, nq is a simple eigenvalue of Tnq and nq !  as n ! 1, uniformly for q = 2; 3; : : : q [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

With ', 'nq and ' n as in 3.13, we have for each large n,  q Tn ' q k k(T ; Tn )q 'k   k ( T ; T ) n n ; q j ; j  min ; [ ] ( )

[ ]

[ ]

[ ]

n

sq;1 k'k k'nq k  q q q q k'n ; P'n k  (jj ; )q k(T ; Tn ) Tn'n k; k'nq ; ' qn k   k(Tn ; T )q T'k ; jjq k'k k' qn k where  is a constant, independent of n and q. (Hint: Let r = 1=s and apply Theorem 3.13 to rT; rTn .) 3.14 Let 0 6=  2 Cj , 0 6= nq 2 Cj q, x = [x1 ; : : : ; xq ]> 2 X q and r > 0: (a) x is an eigenvector of r? T corresponding to r if and only if Tx1 = x1 and xk+1 = x1 =(r)k for k = 1; : : : ; q ; 1: (b) x is an eigenvector of r? Tnq corresponding to rnq if and only if Pq;1 q q;1;k (T ; Tn )k Tnx1 = (nq )q x1 and xk+1 = x1 =(rnq )k for k=0 (n ) k = 1; : : : ; q ; 1: In particular, x1 2 X is the rst component of an eigenvector of r? T q (resp., r? Tnq ) corresponding to r (resp., rnq ) if and only if x1 is the [ ]

[ ]

[ ]

[ ]

[ ] ( )

[ ]

[ ] ( )

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

rst component of an eigenvector of T q (resp., Tnq ) corresponding to  (resp., nq ).  O. If E is a closed subset of re(T ) such that 0 62 3.15 Let (Tn ; T ) ;! E , then there is a positive integer n0 such that E  re(TSnq1) for all n  n0 and all q = 2; 3; : : : For n = 1; 2; : : : consider (SP )n = q=2 sp(Tnq ): (a) Property U: If n 2 (SP )n and n !  6= 0; then  2 sp(T ): (b) Property L at a nonzero isolated point of sp(T ): Let  be a nonzero isolated point of sp(T ) and 0 <  < minfjj; dist(; sp(T ) n fg)g: If n 2 (SP )n and jn ; j < , then n ! : Also, there is an integer n1 such that for each n  n1 and each q = 2; 3; : : : the set fnq 2 sp(Tnq ) : j ; nq j < g is nonempty. (c) Let  be a spectral set for T , 0 62  and C 2 C (T; ) with 0 2 ext(C). Then there is some n1 such that for all n  n1 and all q = 2; 3; : : : nq := sp(Tnq ) \ int(C) and Pnq do not depend on C. (Compare Proposition 2.9 (a).) [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

Chapter 4 Finite Rank Approximations

Consider a complex Banach space X and a bounded linear operator T on X , that is, T 2 BL(X ). In Chapter 2, we saw how spectral values of nite type and the associated spectral subspaces of T are approximated n if a sequence (Tn ) in BL(X ) `converges' to T ; speci cally if Tn ! T , and  more generally if Tn ! T . We shall see in Chapter 5 that if the rank of Tn is nite, then the spectral computations for Tn can be reduced to solving a matrix eigenvalue problem in a canonical way. For this reason, we now consider various situations in which it is possible ton nd a sequence (Tn ) of bounded nite rank operators such that Tn ! T ,  T . First we treat some nite rank approximations or at least, Tn ! which employ bounded nite rank projections. Next we treat the case of Fredholm integral operators, for which degenerate kernel approximations as well as approximations based on quadrature rules are described. For a weakly singular integral operator T , we present an approximation (TnK ) cc  T but T K ! = T such that TnK = Tn + Un, where Tn is of nite rank, TnK ! n n K = T . In the last section we give localization results along the and Tn ! lines of a result of Brakhage.

4.1 Approximations Based on Projections Let (n ) be a sequence of bounded projections de ned on X , that is, each n is in BL(X ) and n2 = n . De ne

TnP := n T; TnS := Tn and TnG := n Tn: 185

186

4. FINITE RANK APPROXIMATIONS

The bounded operators TnP , TnS and TnG are known as the projection approximation of T , the Sloan approximation of T (cf. [67]) and the Galerkin approximation of T , respectively. All three are nite rank operators if the rank of the projection n is nite. We prove a basic convergence result for these approximations. Recall that we denote the identity operator on X by I .

Theorem 4.1 p Let T 2 BL(X ) and n ! I . Then (a) TnP !p T , TnS !p T and TnG !p T . n  T, TG !  T. (b) If T is a compact operator, then TnP ! T , TnS ! n n n (c) If T is a compact operator and n !p I , then TnS ! T , TnG ! T. Proof

The Uniform Boundedness Principle (Theorem 9.1 of [55] or Theorem 4.7-3 of [48]) shows that the sequence (kn k) is bounded. (a)p Since n !p I , TnP x = n Tx ! Tx for each x 2 X , that is, TnP ! T . Also, since T is continuous and

TnS ; T = T (n ; I ); TnG ; T = (TnP ; T )n + TnS ; T; p p we see that TnS ! T and TnG ! T. (b) Let the operator T be compact. The set E := fTx : x 2 X; kxk  1g is relatively compact in the Banach space X . By the

Banach-Steinhaus Theorem (Corollary 9.2(a) of [55]), the pointwise convergence of the sequence (n ) to I is uniform on the set E , that is,

kTnP ; T k = kn T ; T k = supfk(n ; I )yk : y 2 E g ! 0: n Thus TnP ! T. Since kTnS k  kn k kT k and kTnGk  kn k2kT k for all n, the sequences (TnS ) and (TnG) are bounded in BL(X ). Further, we note that

(TnS ; T )T = T (TnP ; T ); (TnG ; T )T = (TnP ; T )TnP + (TnS ; T )T;

(TnS ; T )TnS = (TnS ; T )Tn; (TnG ; T )TnG = (TnP ; T )TnG:  T and T G !  T. Hence TnS ! n

4.1. APPROXIMATIONS BASED ON PROJECTIONS

187

(c) Let T be compact and n !p I  in addition to n !p I . Since T  is a compact operator on X , we see that kTnS ; T k = k(TnS ; T )k = kn T  ; T k ! 0 by (a) above. Also,

kTnG ; T k = kn (TnS ; T ) + TnP ; T k  kTnS ; T k + kTnP ; T k ! 0: n n Thus TnS ! T and TnG ! T. We remark that if T is compact, (TnS ) and (TnG) in fact converge to T in a collectively compact manner. (See Theorem 4.5 of [25] or Theorem 15.1 of [54].) However, they may not in general converge to T in the norm. (See the comment before Proposition 4.6.) We point out that the Hahn-Banach Extension Theorem (Theorem 7.8 of [55] or Theorem 4.3-2 of [48]) is used while asserting k(TnS ; T )k = kTnS ; T k in the proof of (c) above. For an estimation of the rate at which kTnP ; T k, k(TnS ; T )T k, k(TnS ; T )TnS k, k(TnG ; T )T k and k(TnG ; T )TnGk may tend to zero, see Exercises 4.2 and 4.12. We now give several ways of constructing a sequence (n ) of bounded p projections on X such that n ! I and the rank of each n is nite.

4.1.1 Truncation of a Schauder Expansion

Assume that the Banach space X has a Schauder basis, that is, there are e1 ; e2 ; : : : in X such that kej k = 1 for each positive integer j and for every x 2 X , there are unique scalars c1 (x); c2 (x); : : : for which

x=

1 X j =1

cj (x)ej := nlim !1

n X j =1

cj (x)ej :

As a consequence of the Bounded Inverse Theorem (Theorem 11.1 of [55]), each linear functional x 7! cj (x), x 2 X , is continuous on X (Theorem 11.4 of [55]). For each positive integer n, de ne

n x :=

n X j =1

cj (x)ej ; x 2 X:

188

4. FINITE RANK APPROXIMATIONS

p Clearly, n 2 BL(X ), n2 = n , n ! I and rank n = n. If an operator T 2 BL(X ) is represented by the in nite matrix (ti;j ) with respect to a Schauder basis e1 ; e2 ; : : : that is,

Tej = then

1 X i=1

TnP ej = n Tej

ti;j ei ; j = 1; 2; : : :

=

n X i=1

ti;j ei ; j = 1; 2; : : :

8 1 X > <

= > i=1 ti;j ei if j = 1; : : : ; n; : 0 if j > n;

TnS ej = Tn ej

8 n X > <

TnGej = n Tnej = > i=1 ti;j ei if j = 1; : : : ; n; : 0 if j > n:

Hence the matrix representing the projection approximation TnP of T is obtained by truncating each column of the matrix (ti;j ) at the nth entry and putting all zeros thereafter, while the matrix representing the Sloan approximation TnS of T is obtained by replacing every column after the nth column of the matrix (ti;j ) by a column of all zeros. The matrix representing the Galerkin approximation TnG of T is obtained by carrying out both these operations, so that the entries in the top left nn corner of this matrix are the same as the corresponding entries of the matrix (ti;j ) and all other entries are equal to zero. We now consider a special case. Let X be a separable Hilbert space and let e1 ; e2 ; : : : form an orthonormal basis for X . If h
;
i denotes the inner product on X , then by the Fourier Expansion Theorem (Theorem 22.7 of [55] or Theorem 3.5-2 of [48]), we have

x=

1 X j =1

hx ; ej iej ; x 2 X:

Thus for each positive integer n,

n x =

n X j =1

hx ; ej iej ; x 2 X:

4.1. APPROXIMATIONS BASED ON PROJECTIONS

189

n P

Since hn x ; yi = hx ; ej ihej ; yi = hx ; n yi for all x; y 2 X , we see j =1 that n = n , that is, the projection n is orthogonal. Hence if T is n a compact operator on X , then we obtain not only TnP ! T but also n n TnS ! T and TnG ! T by Theorem 4.1(b),(c).

Example 4.2 Some classical Schauder bases and orthonormal bases: (a) Standard Schauder basis for ` p , 1  p < 1.

For each positive integer j , let ek := (0; : : : ; 0; 1; 0; 0; : : :); where only the kth entry is 1. Then (ek ) forms a Schauder basis for ` p . If p = 2, this is an orthonormal basis. (b) Schauder basis of saw-tooth functions for C 0 ([0; 1]). For t 2 [0; 1], let e0 (t) := t, e1 (t) := 1 ; t, 8 1 > < 2e0 (t) if 0  t  , 2 e2 (t) := > 1 : 2e1 (t) if < t  1. 2 For each positive integer m and j = 1; : : : ; 2m, let e2m +j (t) := e2 (2m t ; j + 1); where we let e2 (t) := 0 for t 2= [0; 1]. The sequence of these saw-tooth functions forms a Schauder basis for C 0 ([0; 1]). (c) Orthonormal basis of Haar functions for L2 ([0; 1]). For t 2 [0; 1], let e0;0 (t) := 1, 8 > > 1 if 0  t < 21 , > > < e1;0 (t) := > ;1 if 1 < t < 1, 2 > > > : 0 if t = 1 . 2 For each positive integer m and j = 1; : : : ; 2m, let 8 > 2m=2 if j 2;m1  t < 22jm;+11 , > < em;j (t) := > ;2m=2 if 2j ; 1  t < j , > 2m+1 2m : 0 otherwise.

190

4. FINITE RANK APPROXIMATIONS

The sequence of these Haar functions forms an orthonormal basis for the space L2 ([0; 1]). (d) Orthonormal basis of trigonometric functions for L2 ([;; ]). For t 2 [;; ], let ej (t) := p1 (cos jt + i sin jt), j = 0; 1; 2; : : : The 2 sequence of these trigonometric functions forms an orthonormal basis for the space L2 ([;; ]). p 1 For t 2 [0; ], let e0 (t) := p , ej (t) := p2 cos jt and eej (t) := p p 2 sin jt. Then each of the sequences (e ) and (ee ) forms an orthonor-



j

j

mal basis for the space L ([0; ]). (e) Orthonormal bases of polynomials for L2w ( ]a; b[ ). For t 2 ]a; b[, let yj (t) := tj for each nonnegative integer j . Applying the Gram-Schmidt Process described in Proposition 1.39 to the linearly independent set fy0 ; y1 ; y2 ; : : :g, we obtain a sequence of orthonormal polynomials (en ). If a := ;1 and b := 1, then p they are known p pas the Legendreppolynomials. Note that e0 (t) = 1= 2, e1 (t) = 3 t= 2 and e2 (t) = 10(3t2 ; 1)=4 for t 2 ];1; 1[. The Legendre polynomials of higher degrees are often found by employing a three-term recurrence relation. If w is a positive continuous function on ]a; b[, one can orthonormalize y0 ; y1 ; y2 ; : : : with respect to the inner product 2

hx ; yiw :=

b

Z

a

x(t)y(t)w(t) dt

de ned on the set L2w ( ]a; b[ ) of all equivalence classesZof Lebesgue meab surable complex-valued functions x on ]a; b[ satisfying jx(t)j2 w(t) dt < a 1. Then the resulting sequence (ej ) of polynomials forms an orthonorp mal basis for L2w ( ]a; b[ ) and the sequence ( w ej ) forms an orthonormal basis for L2 ( ]a; b[ ).

4.1.2 Interpolatory Projections

Let X := C 0 ([a; b]) with the sup norm k
k1 . For positive integers n and r(n), consider the nodes tn;1 ; : : : ; tn;r(n) in [a; b]:

a  tn;1 < tn;2 <


< tn;r(n);1 < tn;r(n)  b

4.1. APPROXIMATIONS BASED ON PROJECTIONS

191

and let tn;0 := a and tn;r(n)+1 := b. Consider functions en;1 ; : : : ; en;r(n) in C 0 ([a; b]) such that

en;j (tn;k ) = j;k ; j; k = 1; : : : ; r(n): We De ne (n x)(t) :=

rX (n) j =1

x(tn;j )en;j (t); x 2 X; t 2 [a; b]:

Then n : X ! X , and clearly n2 = n . Also,

R(n ) = spanfen;1; : : : ; en;r(n)g and the set fen;1; : : : ; en;r(n)g is linearly independent. Hence rank(n ) = r(n). Since for each x 2 X , (n x)(tn;j ) = x(tn;j ); j = 1; : : : ; r(n); that is, n x interpolates x at tn;1 ; : : : ; tn;r(n), we say that n is an interpolatory projection. An approximation based on an interpolatory projection is also known as a collocation approximation.

rP

(n) We show that kn k =

jen;j j

. For x 2 X and t 2 [a; b], we 1 j =1 have

j(n x)(t)j 

rX (n) j =1

jx(tn;j )j jen;j (t)j  kxk1 (n)

rX

 kxk1

rP (n)

j =1

rX (n) j =1

jen;j (t)j

jen;j j

: 1

Hence kn k 

jen;j j

. On the other hand, choose t0 2 [a; b] such 1 j =1 that (n) (n)

rX   rX jen;j j (t0 ) =

jen;j j

j =1

and de ne x0 2 X as follows:

j =1

1

x0 (tn;j ) := sgn en;j (t0 ); j = 1; : : : ; r(n);

192

4. FINITE RANK APPROXIMATIONS

x0 (a) := x(tn;1 ), x0 (b) := x(tn;r(n) ) and x0 is a polynomial of degree  1 on each of the subintervals [a; tn;1]; [tn;1 ; tn;2 ]; : : : ; [tr(n);1; tr(n) ]; [tr(n) ; b]. Then kx0 k1 = 1 and (n x0 )(t0 ) =

rX (n) j =1

rP (n)

x0 (tn;j )en;j (t0 ) =

rX (n) j =1

(n)

rX

jen;j (t0 )j =

j =1

jen;j j

: 1

Hence kn k 

jen;j j

and we are through. 1 j =1 As the following examples will show, we often have r(n) = n.

Example 4.3 Some classical nodes: Some classical choices of nodes tn;1 ; : : : ; tn;n in [0; 1] for which the mesh hn := maxftn;j ; tn;j;1 : j = 1; : : : ; n + 1g ! 0 are as follows. (a) tn;j := nj , j = 1; : : : ; n (b) t := j ; 1 , j = 1; : : : ; n n;j

n

(c) tn;j := 2j2;n 1 , j = 1; : : : ; n 1 , j = 1; : : : ; n for n > 1 (d) tn;j := nj ; p 8;1 j ; (1= 3) > < if j = 1; 3; : : : ; n ; 1, for n = 2; 4; : : : (e) tn;j := > j ; 1n+ (1=p3) : if j = 2; 4; : : : ; n, n

p

The nodes in (e) above are obtained by considering the roots 1= 3 of the second degree Legendre polynomial in [ ; 1; 1] and transfering them to each of the n=2 intervals [2(k ; 1)=n; 2k=n], k = 1; : : : ; n=2, where n is an even positive integer, by a linear change of variable. We now give some speci c choices of functions en;j 2 C 0 ([a; b]), j = 1; : : : ; n, which yield important sequences of interpolatory projections. (i) Piecewise Linear Interpolation: For each positive integer n, de ne en;j in C 0 ([a; b]) as follows: en;j (tn;k ) := j;k ; j; k = 1; : : : ; n; en;1 (a) := 1, en;n (b) := 1, en;j (a) := 0 for j = 2; : : : ; n, en;j (b) := 0 for j = 1; : : : ; n ; 1 and en;j is a polynomial of degree  1 on each

193

4.1. APPROXIMATIONS BASED ON PROJECTIONS

of the subintervals [a; tn;1 ]; [tn;1 ; tn;2]; : : : ; [tn;n;1 ; tn;n ] and [tn;n ; b]. The functions en;1 ; : : : ; en;n are known as the hat functions because of the shape of their graphs. 1

6

1

-

a tn;1 tn;2

the hat function en;1 1

6

b

6

a

-

tn;n;1 tn;n b the hat function en;n

-

a

tn;j;1 tn;j tn;j+1 b the hat function en;j , 1 < j < n Note that en;j (t)  0 for all positive integers n, j = 1; : : : ; n and t 2 [a; b]. If t 2 [tn;j;1 ; tn;j ] for some j = 1; : : : ; n + 1, then en;j;1 (t) + en;j (t) = 1, while en;k (t) = 0 for all k 6= j ; 1; j . In particular, en;1 (t) +


+ en;n (t) = 1 for all t 2 [a; b]: Let tn;0 := a and tn;n+1 := b. We show that if the mesh hn := maxftn;j ; tn;j;1 : j = 1; : : : ; n + 1g ! 0; p then n ! I . Fix x 2 C 0 ([a; b]) and let  > 0. By the uniform continuity of x on [a; b]; there is some  > 0 such that jx(s) ; x(t)j < ; whenever s; t 2 [a; b] and js ; tj < . Since hn ! 0, choose n0 such that hn <  for all n  n0 . Let n  n0 and t 2 [a; b]. If t 2 [tn;j;1 ; tn;j ], then jtn;j ; tj  hn <  and hence jx(tn;j ) ; x(t)j < . Thus

j(n x)(t) ; x(t)j =

n X [ i=1



x(tn;i ) ; x(t)]en;i (t)

 jx(tn;j;1 ) ; x(t)jen;j;1 (t) + jx(tn;j ) ; x(t)jen;j (t) <  [en;j;1 (t) + en;j (t)] = :

p Thus kn x ; xk1 ! 0 for every x 2 C 0 ([a; b]), that is, n ! I.

194

4. FINITE RANK APPROXIMATIONS

If in fact x 2 C 1 ([a; b]) and t 2 [tn;j;1 ; tn;j ] for some j = 1; : : : ; n + 1, then by the Mean Value Theorem, jx(tn;j;1 ) ; x(t)j = jx0 (sn;j )j(t ; tn;j;1 ) and jx(tn;j ) ; x(t)j = jx0 (un;j )j(tn;j ; t) for some sn;j 2 [tn;j;1 ; t] and un;j 2 [t; tn;j ], so that

jn x(t) ; x(t)j  jx0 (sn;j )j(t ; tn;j;1 )en;j;1 (t) +jx0 (un;j )j(tn;j ; t)en;j (t)  kx0 k1 [(t ; tn;j;1 ) + (tn;j ; t)]  kx0 k1 hn :

Thus

kn x ; xk1  kx0 k1 hn for all x 2 C 1 ([a; b]):

It can also be proved that if tn;1 = a and tn;n = b for all n, then kn x ; xk1  18 kx00 k1 (hn )2 for all x 2 C 2 ([a; b]): (See Exercise 4.8.) Piecewise quadratic interpolation is treated in Exercise 4.9. (ii) Cubic Spline Interpolation: For each positive integer n and nodes a = tn;1 <


< tn;n = b in [a; b], let Xn denote the subspace of X := C 0 ([a; b]) consisting of all x 2 C 2 ([a; b]) such that x is a polynomial of degree  3 on each of the n;1 subintervals [tn;1 ; tn;2 ]; : : : ; [tn;n;1 ; tn;n]. Then dim Xn = n +2. [Note: A polynomial of degree  3 on each of the n ; 1 subintervals has 4 degrees of freedom, which are constrained by 3 continuity conditions at each of the n ; 2 points tn;2 ; : : : ; tn;n;1 and 4(n ; 1) ; 3(n ; 2) = n + 2. See Theorem 4.4 of [65].] An element of Xn is known as a cubic spline function with knots at tn;1 ; : : : ; tn;n. It can be shown that for each j = 1; : : : ; n, there is a unique function en;j in Xn such that en;j (tn;k ) = j;k ; k = 1; : : : ; n; and the third derivative of en;j exists at tn;2 and tn;n;1 . The last requirement says that for each en;j , the two points tn;2 and tn;n;1 n P are not really knots. For x 2 X , let n x := x(tn;j )ej ; x 2 X , j =1

as usual. Let hn and ehn denote the maximum and the minimum of ftn;j ; tn;j;1 : j = 2; : : : ; ng, respectively. If there is a constant  such that (hn =ehn)   for all n (a so-called sequence of quasiuniform partip tions) and hn ! 0, then it is known that n ! I . Note that all the

4.1. APPROXIMATIONS BASED ON PROJECTIONS

195

classical choices of nodes yield quasiuniform partitions. Further, it can be shown that kn x ; xk1  kx(4) k1 (hn )4 for all x 2 C 4 ([a; b]); where  is a constant. (See Corollary 6.21 of [65] and pages 55, 56 of [29].) Cubic spline interpolation for the case where a < tn;1 and b > tn;n is considered in Exercise 4.11(b). It is possible to choose a basis for the space Xn of cubic splines such that each function in the basis is nonzero on at most four of the subintervals [a; tn;1 ]; : : : ; [tn;n;1 ; b]. Such functions are known as B-splines and have proved to be well suited for numerical computations. (See [29] for a discussion of B-splines and [70] for spectral computations using cubic splines.) (iii) Lagrange Interpolation: Let a  tn;1 <


< tn;n  b. The Lagrange polynomials pn;1 ; : : : ; pn;n with nodes at tn;1 ; : : : ; tn;n are de ned by n t;t Y n;k pn;j (t) := t ; t ; t 2 [a; b]: k=1 n;j k6=j

n;k

Note that each pn;j is a polynomial of degree n ; 1 and pn;j (tn;k ) = j;k for j; k = 1; : : : ; n. The interpolatory projection n , obtained by letting en;j := pn;j , j = 1; : : : ; n, is known as the Lagrange interpolatory projection. A result of Kharshiladze and Lozinski implies that no matter how the nodes tn;1 ; : : : ; tn;n are chosen, there is some x 2 C 0 ([a; b]) for which the sequence (kn x ; xk1 ) is unbounded. (See page 214 of [28].) p = I . If, however, a := ;1, b := 1 and the nodes tn;1 ; : : : ; tn;n Thus n ! are the roots of the nth Legendre polynomial in [;1; 1], then it can be proved that kn x ; xk ! 0 for each x 2 C 0 ([;1; 1]). (See page 137 of [28].) Using this result of Erdos and Turan, it can be seen that if T is a compact operator on L2 ([;1; 1]) with R(T )  C 0 ([;1; 1]), then n TnP ! T ; and if T is a Fredholm integral operator on C 0 ([;1; 1]) with a  T . (See Exercise 4.13.) continuous kernel, then TnS ! 2

4.1.3 Orthogonal Projections on Subspaces of Piecewise Constant Functions Let X := L2 ([a; b]). For each positive integer n, consider a = tn;0 < tn;1 <


< tn;n;1 < tn;n = b

196

4. FINITE RANK APPROXIMATIONS

and the subspace Xn := fx 2 X : x is constant on [tn;j;1 ; tn;j [; j = 1; : : : ; ng of X . For x 2 X , let Z tn;j

1

(n x)(t) := t

n;j ; tn;j ;1 tn;j;1

x(s) ds; t 2 [tn;j;1 ; tn;j [; j = 1; : : : ; n;

(n x)(b) := (n x)(tn;n;1 ): It is clear that n : X ! X is linear, R(n ) = Xn and if x 2 Xn , then n x = x. Hence n2 = n and rank n = dim Xn = n. Let x 2 X . Then for j = 1; : : : ; n, Z tn;j

tn;j;1

j(n x)(t)j dt = 2



Z tn;j

Z tn;j

1

tn;j;1 tn;j ; tn;j ;1 tn;j;1 Z tn;j 1

Z tn;j h

2

x(s) ds dt i

jx(s)j2 ds dt

tn;j;1 tn;j ; tn;j ;1 tn;j;1 i Z tn;j 1 = jx(s)j2 dt ds tn;j;1 tn;j;1 tn;j ; tn;j ;1 Z tn;j Z tn;j h

=

tn;j;1

jx(s)j2 ds:

Hence

kn xk2 = 2

n Z tn;j X j =1 tn;j;1

j(n x)(t)j2 dt 

n Z tn;j X j =1 tn;j;1

jx(s)j2 ds = kxk2 : 2

Thus kn k  1. It follows that n is an orthogonal projection de ned on the Hilbert space L2 ([a; b]) and that the range Xn of n is contained in the set of all piecewise constant functions. We now show that if hn := maxftn;j ; tn;j;1 : j = 1; : : : ; ng ! 0, then kn x ; xk ! 0 for every x 2 L2([a; b]). Since the sequence (kn k ) is bounded and C 0 ([a; b]) is a dense subset of L2 ([a; b]), it is enough to show that 2

2

2

kn x ; xk ! 0 for every x 2 C 0 ([a; b]): 2

Consider then x 2 C 0 ([a; b]). Let  > 0. As x is uniformly continuous on [a; b], there is  > 0 such that jx(s) ; x(t)j <  whenever s; t 2 [a; b] and js ; tj < . Since hn ! 0, choose n0 such that hn <  for all n  n0 .

4.1. APPROXIMATIONS BASED ON PROJECTIONS

197

Then for all n  n0 and j = 1; : : : ; n, we have Z tn;j

j(n x)(t) ; x(t)j2 dt

tn;j;1 Z tn;j

=

 



1

Z tn;j

tn;j;1 tn;j ; tn;j ;1 tn;j;1 Z tn;j 1

Z tn;j h

tn;j;1 tn;j ; tn;j ;1 tn;j;1

Z tn;j

tn;j;1

2

[x(t) ; x(s)] ds dt i

jx(t) ; x(s)j2 ds dt

2 ds = 2 (tn;j ; tn;j;1 ):

Hence

kn x ; xk = 2 2



n Z tn;j X j =1

n X j =1

tn;j;1

j(n x)(t) ; x(t)j2 dt

2 (tn;j ; tn;j;1 ) = 2 (b ; a)

for all n  n0 . Thus kn x ; xk ! 0 for every x 2 C 0 ([a; b]). We p conclude that n ! I. One can similarly consider orthogonal projections de ned on L2([a; b]) whose ranges are contained in the set of all piecewise polynomials of degree  d, where d is a given positive integer. (See Exercise 4.14.) 2

4.1.4 Finite Element Approximation

If A and B are operators on a Banach space X , then the problem of nding a nonzero element ' of X and a scalar  such that A' = B' is known as the generalized eigenvalue problem. A `weak formulation' of the generalized eigenvalue problem which is valid for several dierential operators can be given as follows. Let X be a Hilbert space with an inner product h
;
i. A sesquilinear functional a(
;
) on X X is a complex-valued function on X X which is linear in the rst variable and conjugate-linear in the second variable. It is said to be bounded if ja(x; y)j  kxk kyk for some  > 0 and all x, y 2 X . Consider bounded sesquilinear functionals a(
;
) and b(
;
) on X X , where a(
;
) is strongly coercive, that is,

< a(x; x)  kxk2 for some  > 0 and all x 2 X:

198

4. FINITE RANK APPROXIMATIONS

Consider the problem of nding a nonzero element ' of X and a scalar  such that a('; y) = b('; y) for all y 2 X: This is known as a weakly posed generalized eigenvalue problem. By the Riesz Representation Theorem for a Hilbert space (Theorem 24.3 of [55] or Theorem 3.8-1 of [48]), there are A 2 BL(X ) and B 2 BL(X ) such that

a(x; y) = hAx ; yi and b(x; y) = hBx ; yi for all x; y 2 X: The strong coercivity of the sesquilinear functional a(
;
) shows that the linear operator A is bounded below and its adjoint A is injective. It follows that the bounded operator A is invertible in BL(X ). Let T := A;1 B . A nite element approximation of the weakly posed generalized eigenvalue problem is obtained by considering, for each positive integer n, a nite dimensional subspace Xn of X and by requiring to nd a nonzero element 'n of Xn and a scalar n such that

a('n ; y) = n b('n ; y) for all y 2 Xn: Let n denote the orthogonal projection de ned on X with R(n ) = Xn . Then the preceding equation can be written as

n A'n = n n B'n ; 0 6= 'n 2 Xn : Then for every x 2 Xn , we have

kxk2  < a(x; x)  ja(x; x)j = jhAx ; xij = jhAx ; n xij = jhn Ax ; xij  kn Axk kxk: De ne An := n AjXn ;Xn . Then An 2 BL(Xn ) is injective. Since the linear space Xn is nite dimensional, An is invertible and kA;n 1 k  1=. Now the earlier equation can be written as

'n = n A;n 1 n B'n = n A;n 1 n AT'n; 0 6= 'n 2 Xn : De ne n x := A;n 1 n Ax for x 2 X . Then n 2 BL(X ), R(n ) = Xn and n2 = A;n 1 (n AjXn A;n 1 )n A = A;n 1 n A = n :

199

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

Thus n is a bounded nite rank projection; and if we let TnP := n T; then our equation becomes

TnP 'n = 1 'n ; 0 6= 'n 2 Xn ; n

provided n 6= 0. This shows that a nite element approximation of a weakly posed generalized eigenvalue problem can be realized as a projection approximation of a bounded operator T . In most applications, the operator T is compact and the nite dimensional subspaces p X1 ; X2 ; : : : of X are so chosen that n ! I . Since n = n ; n + n = p n A;n 1 n A(I ; n ) + n , we see that n ! I and so TnP ! T.

4.2 Approximations of Integral Operators

The approximation procedures discussed in the last section are applicable to any compact operator T on a complex Banach space X . In this section we consider the special case where X is a suitable function space and T is a Fredholm integral operator on X . We develop several approximation procedures which are peculiar to this case. For the sake of simplicity, we consider either X := L2 ([a; b]) with the 2-norm and a function k(
;
) 2 L2 ([a; b][a; b]), or X := C 0 ([a; b]) with the sup norm and a function k(
;
) 2 C 0 ([a; b][a; b]). For x 2 X , let (Tx)(s) :=

b

Z

a

k(s; t)x(t) dt; s 2 [a; b]:

The operator T is known as a Fredholm integral operator with kernel k(
;
). It is easy to see that T 2 BL(X ). In fact, if X := L2([a; b]), then

kT k 

Z Z [a;b][a;b]

jk(s; t)j2 dm(s; t)

1=2

= kk(
;
)k ; 2

and, if X := C 0 ([a; b]), then

kT k  (b ; a) supfjk(s; t)j : s; t 2 [a; b]g = (b ; a)kk(
;
)k1 :

200

4. FINITE RANK APPROXIMATIONS

Further, T is a compact operator. (See Example 17.4(b) of [55] or Theorem 8.7-5 of [48].) One can also consider integral operators on Lp (S ) or C 0 (S ), where S is a suitable subset of IRk , k  1 and 1  p  1. However, we shall restrict ourselves to the two cases mentioned above. We shall denote the 2-norm on L2 ([a; b][a; b]) as well as the sup norm on C 0 ([a; b][a; b]) simply by k
k.

4.2.1 Degenerate Kernel Approximation

A kernel ek(
;
) is said to be degenerate if there are x1 ; : : : ; xr and y1 ; : : : ; yr in X such that

k s; t) := e(

r X j =1

xj (s)yj (t); s; t 2 [a; b]:

If Te is a Fredholm integral operator with a degenerate kernel ek(
;
), then for x 2 X , we have

Tx)(s) =

(e

r X j =1

b

Z

xj (s) yj (t)x(t) dt; s 2 [a; b]; a

so that R(Te)  spanfx1 ; : : : ; xr g and hence Te is a nite rank operator.

Theorem 4.4

Let T be a Fredholm integral operator on X := L2 ([a; b]) or X := C 0 ([a; b]) with kernel k(
;
); and for each positive integer n, let kn (
;
) be a degenerate kernel such that kkn (
;
) ; k(
;
)k ! 0. Consider the degenerate kernel approximation of T given by:

(TnD x)(s) :=

b

Z

a

kn (s; t)x(t) dt; x 2 X; s 2 [a; b]:

Then (TnD ) is a sequence of bounded nite rank operators on X and n TnD ! T.

Proof Note that TnD ; T is a Fredholm integral operator on X with kernel kn (
;
) ; k(
;
). If X := L2 ([a; b]), then kTnD ; T k  kkn (
;
) ; k(
;
)k ! 0 2

2

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

201

and if X := C 0 ([a; b]), then

kTnD ; T k1  (b ; a)kkn (
;
) ; k(
;
)k1 ! 0: Hence the result follows. We now describe various methods for constructing a sequence of degenerate kernels which converges to a given kernel in the norm.

(i) Piecewise Linear Interpolation in the Second Variable:

Let k(
;
) 2 C 0 ([a; b][a; b]), and for a xed s 2 [a; b], consider ks (t) := k(s; t), t 2 [a; b]. For each positive integer n, let a = tn;0  tn;1 <


< tn;n  tn;n+1 = b and let n denote the piecewise linear interpolatory projection described in part (i) of Subsection 4.1.2. De ne

kn (s; t) := (n ks )(t) =

n X j =1

k(s; tn;j )en;j (t); s; t 2 [a; b];

where en;j , j = 1; : : : ; n, are the corresponding hat functions. Thus kn (
;
) is a degenerate kernel obtained by interpolating the kernel k(
;
) in the second variable. We show that kkn (
;
) ; k(
;
)k1 ! 0 if hn := maxftn;j ; tn;j;1 : j = 1; : : : ; n + 1g ! 0. Let  > 0. By the uniform continuity of the function k(
;
) on [a; b][a; b], there exists  > 0 such that jk(s; t) ; k(s; u)j <  whenever s 2 [a; b] and jt ; uj < . Since hn ! 0, choose n0 such that hn <  for all n  n0 . If n  n0 , we have

jkn (s; t) ; k(s; t)j = j(n ks )(t) ; ks (t)j <  p for all s; t 2 [a; b]. (See the proof of n ! I given in part (i) of Subsection 4.1.2.) Thus kkn (
;
) ; k(
;
)k1 ! 0. If we interpolate the kernel k(
;
) in both the variables, we obtain the degenerate kernel

k s; t) = en (

n X i;j =1

k(tn;i ; tn;j )en;i (s)en;j (t); s; t 2 [a; b]:

As before, it can be seen that kekn (
;
) ; k(
;
)k1 ! 0 if hn ! 0. (ii) Bernstein Polynomials in Two Variables: Let k(
;
) belong to C 0 ([0; 1][0; 1]). For each nonnegative integer n, consider the nth

202

4. FINITE RANK APPROXIMATIONS

Bernstein polynomial in two variables given by

kn (s; t) :=

n X

  

  k ni ; nj ni i;j =0

n si (1 ; s)n;i tj (1 ; t)n;j ; s; t 2 [a; b]: j

Then kn (
;
) is a degenerate kernel and kkn (
;
) ; k(
;
)k1 ! 0. (See page 10 of [35].) (iii) Truncation of a Taylor Expansion: Suppose that k(
;
) belongs to C 0 ([a; b][a; b]) and has a uniformly and absolutely convergent Taylor series expansion about some point (s0 ; t0 ) 2 IR2 given by

k(s; t) :=

1 X

i;j =0

ci;j (s ; s0 )i (t ; t0 )j ; s; t 2 [a; b];

where ci;j 2 Cj for i; j = 0; 1; : : : For each positive integer n, let

kn (s; t) :=

n X

i;j =0

ci;j (s ; s0 )i (t ; t0 )j ; s; t 2 [a; b]:

It follows that kkn (
;
) ; k(
;
)k1 ! 0. A simple example of this kind is given by 1 jj X est := sjt! ; s; t 2 [a; b]; j =0

where (s0 ; t0 ) := (0; 0), ci;j := 0 if i 6= j and cj;j := 1=j ! for i; j = 0; 1; : : : (iv) Truncation of a Fourier Expansion: Suppose that k(
;
) belongs to L2 ([a; b][a; b]). Consider an orthonormal basis (en ) for L2 ([a; b]), and for positive integers i and j , let ki;j (s; t) := ei (s)ej (t); s; t 2 [a; b]: Then (ki;j ) is an orthonormal basis for L2 ([a; b][a; b]). (See 22.8(d) of [55].) By the Fourier Expansion Theorem for L2 ([a; b][a; b]) (Theorem 22.7 of [55] or Theorems 3.5-2, 3.6-2 and 3.6-3 of [48]), we have

k(
;
) = where

ci;j :=

Z bZ

a a

1 X

i;j =1 b

ci;j ki;j (
;
);

k(s; t)ki;j (s; t) ds dt;

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

203

and the series converges in L2 ([a; b][a; b]). For each positive integer n, let

kn (s; t) :=

n X

i;j =1

ci;j ki;j (s; t) =

n X

i;j =1

ci;j ei (s)ej (t); s; t 2 [a; b]:

Clearly, kkn (
;
) ; k(
;
)k ! 0. n P If we let n x := hx ; ej iej for x 2 L2 ([a; b]), then it is easy to see j =1 that TnD = n Tn = TnG. Several other degenerate kernel approximations are considered in [66]. 2

4.2.2 Approximations Based on Numerical Integration

Let X := CZ0 ([a; b]) with the sup norm and let Q : X ! Cj be de ned b by Q(x) := x(t) dt, x 2 X . It is clear that Q is a continuous linear a functional on X and kQk = b ; a. A quadrature formula is a linear functional Qe : X ! Cj given by

Q x) := e(

r X j =1

wj x(tj ); x 2 X;

where the nodes t1 ; : : : ; tr satisfy a  t1 <


< tr  b and the weights w1 ; : : : ; wr are complex numbers. It is easy to see that the functional Qe is continuous on X and in fact we have

kQek =

r X j =1

jwj j:

We say that (Qn ) is a convergent sequence of quadrature formul if Qn (x) ! Q(x) for every x 2 X: The following result, known as Polya's Theorem, gives a criterion for a sequence of quadrature formul to be convergent. A sequence of quadrature formul given by

Qn (x) := is convergent if and only if

rX (n) j =1

wn;j x(tn;j ); x 2 X;

204

4. FINITE RANK APPROXIMATIONS

(i) Qn (y) ! Q(y) for every y in a subset E whose span is dense in C 0 ([a; b]), (ii)

rP (n) j =1

jwn;j j   for some constant  and all positive integers n.

The proof depends on the Uniform Boundedness Principle. A convenient choice of a subset E whose span is dense in C 0 ([a; b]) is fy0; y1 ; y2 ; : : :g, where yk (t) := tk , t 2 [a; b]. In any case, if the weights wn;j are all nonnegative and if the constant function y0 is in the set E , the condition (ii) given above is automatically satis ed since rX (n) j =1

jwn;j j =

rX (n) j =1

wn;j = Qn (y0 ) ! Q(y0 ) = b ; a:

We now describe a natural way of approximating a Fredholm integral operator T by employing a sequence (Qn ) of quadrature formul. For a xed s 2 [a; b], consider the function ks (t) := k(s; t), t 2 [a; b]. Let x 2 X . Then ks x 2 X for every xed s 2 [a; b] and (Tx)(s) =

b

Z

a

(ks x)(t) dt = Q(ks x):

We de ne an approximating operator TnN by replacing the functional Q by the quadrature formula Qn in the equation given above. Thus for each positive integer n and all x 2 X , let (TnN x)(s) := Qn (ks x) = =

rX (n) j =1

rX (n) j =1

wn;j (ks x)(tn;j )

wn;j k(s; tn;j )x(tn;j ); s 2 [a; b]:

The operator TnN is known as the Nystrom approximation of T based on the quadrature formula Qn . Let (n ) be a sequence of bounded projections de ned on X and let TnF := n TnN for each positive integer n. The operator TnF is known as the Fredholm approximation of T based on the quadrature formula Qn and the bounded projection n .

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

205

Theorem 4.5

Let X := C 0 ([a; b]) and T be a Fredholm integral operator on X with a continuous kernel.

(a) Let (TnN ) be a Nystrom approximation of T pbased on a convergent  T. sequence of quadrature formul. Then TnN ! T and TnN ! (b) Let, in addition, (n ) be a sequence of bounded projections such p p  T. that n ! I . Then TnF ! T and TnF ! Proof (a) Let x 2 X . Since the sequence (Qn ) of quadrature formul em-

ployed to de ne the Nystrom approximation (TnN ) is convergent, we see that (TnN x)(s) = Qn(ks x) ! Q(ks x) = (Tx)(s) for each s 2 [a; b]. We show that this convergence is uniform for s 2 [a; b]. The subset S := fks x : s 2 [a; b]g of X is uniformly bounded since for all s 2 [a; b],

kks xk1  kks k1 kxk1  kk(
;
)k1 kxk1 : Also, it is uniformly equicontinuous, since the functions x and k(
;
) are uniformly continuous; and for all t; u 2 [a; b], we have

jks (t)x(t) ; ks (u)x(u)j  jks (t)x(t) ; ks (t)x(u)j +jks (t)x(u) ; ks (u)x(u)j  kk(
;
)k1 jx(t) ; x(u)j +kxk1 sup jk(s; t) ; k(s; u)j: s2[a;b]

By Ascoli's Theorem (Theorem 3.10(a) of [55]), the set S is relatively compact; and by the Banach-Steinhaus Theorem (Theorem 9.2(a) of [55]), the pointwise convergence of the sequence (Qn ) of continuous functionals is uniform on S . This means that kTnN x ; Txk1 ! 0. Thus p TnN ! T . In particular, (kTnN k) is bounded. Let E := fTx : x 2 X; kxk1  1g. Since T is a compact operator, the set E is relatively compact in X . Again by the Banach-Steinhaus Theorem, the pointwise convergence of (TnN ) to T is uniform on E , that is, k(TnN ; T )T k = supfk(TnN ; T )yk1 : y 2 E g ! 0:

206

4. FINITE RANK APPROXIMATIONS

Next, the subset

Ee :=

1 [ n=1

fTnN x : x 2 X; kxk1  1g

of X is uniformly bounded since for all x 2 X with kxk1  1, kTnN xk  sup kTnN k < 1. Also, it is uniformly equicontinuous since for all n, all n1 x 2 X with kxk1  1 and all s; u 2 [a; b], we have rX (n) N N j(Tn x)(s) ; (Tn x)(u)j  jwn;j j jk(s; tn;j ) ; k(u; tn;j )jjx(tn;j )j j =1 rX (n)     sup jwn;j j sup jk(s; t) ; k(u; t)j : n1 j =1 t2[a;b]

Note that the function k(
;
) is uniformly continuous on [a; b][a; b] and sup

rX (n)

n1 j =1

jwn;j j < 1

by Polya's Theorem. Again by Ascoli's Theorem, the set Ee is relatively compact; and by the Banach-Steinhaus Theorem, the pointwise convergence of (TnN ) to T is uniform on Ee , so that

k(TnN ; T )TnN k  supfk(TnN ; T )yek1 : ye 2 Eeg ! 0:  T. Thus TnN ! (b) Letp (n ) be a sequence of bounded projections de ned on X such that n ! I . By the Uniform Boundedness Principle, kn k   for some  > 0 and all n. Since TnF ; T = n (TnN ; T ) + n T ; T = n (TnN ; T ) + TnP ; T;

p we see that TnF ! T . In particular, (kTnF k) is bounded. Since the sets E and Ee, introduced in the proof of (a) above, are relatively compact, we have k(TnF ; T )T k = supfk(TnF ; T )yk1 : y 2 E g ! 0;

k(TnF ; T )TnF k = supfk(TnF ; T )nyek1 : ye 2 Eeg ! 0

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

207

p again by the Banach-Steinhaus Theorem, since (TnF ; T )n ! O. Thus  F Tn ! T . cc cc We remark that in fact TnN ! T and TnF ! T . (See Propositions 2.1 and 2.2 of [17], Theorem 4.11 and Corollary 4.12 of [25], or Theorem n n = T and TnF ! = T , unless, of course, T := O. 16.2 of [54].) However, TnN ! This follows from the following result whichn also implies that if n isn an = T and TnG := n Tn ! = T, interpolatory projection, then TnS := Tn ! unless T := O.

Proposition 4.6 Let X := C 0 ([a; b]), T 2 BL(X ) be a Fredholm integral operator with a kernel k(
;
), and (Tn ) be a sequence in BL(X ). Suppose that the following conditions are satis ed.

(i) For each x 2 X and each s 2 [a; b], (Tn x)(s) ! (Tx)(s) as n ! 1. (ii) For each n, there are tn;1 ; : : : ; tn;r(n) in [a; b] such that 1. Tn x = 0 whenever x 2 X and x(tn;1 ) =


= x(tn;r(n) ) = 0, 2. For each s 2 [a; b], there is n (s) > 0 such that rX (n)

Z

k(s; t) dt ! 0 as n ! 1:

j =1jtn;j ;tj<n (s)

Then

lim inf kT ; T k  2kT k: n!1 n

Proof

Let  > 0. Then there exist x0 2 X and s0 2 [a; b] such that kx0 k1  1 and j(Tx0 )(s0 )j > kT k ; : Since, by condition (i), (Tn x0 )(s0 ) ! (Tx0 )(s0 ), choose n0 such that for all n  n0 , we have

j(Tn x0 )(s0 ) ; (Tx0 )(s0 )j < : For each n = 1; 2; : : : consider points tn;1 ; : : : ; tn;r(n) and let n := n (s0 ), as stated in condition (ii). By altering the continuous function x0 on the

208

4. FINITE RANK APPROXIMATIONS

intervals In;j := ]tn;j ; n ; tn;j + n [ \ [a; b], j = 1; : : : ; r(n), construct a function xn 2 C 0 ([a; b]) such that

kxn k1  1; xn (tn;j ) = ;x0 (tn;j ) for j = 1; : : : ; r(n): Then (n) Z rX

j(Txn )(s0 ) ; (Tx0 )(s0 )j =

2

j =1 In;j rX (n) Z



k(s0 ; t)[xn (t) ; x0 (t)] dt

j =1 In;j

jk(s0 ; t)j dt;

so that

(Txn )(s0 ) ! (Tx0 )(s0 ): Then for each n, we have (x0 + xn )(tn;j ) = 0; j = 1; : : : ; r(n), so that Tn (x0 + xn ) = 0 by condition (ii). In particular, (Tn xn )(s0 ) = ;(Tn x0 )(s0 ). Hence

j(Tn xn ; Txn )(s0 )j = j ; (Tn x0 )(s0 ) ; (Txn )(s0 )j ! 2j(Tx0 )(s0 )j: Since kxn k1  1, we see that kTn ; T k  k(Tn ; T )xn k1  j(Tn xn ; Txn )(s0 )j: Thus lim inf kT ; T k  nlim n!1 n !1 j(Tn xn ; Txn)(s0 )j = 2j(Tx0 )(s0 )j  2kT k; 2: As  > 0 is arbitrary, we have

lim inf kT ; T k  2kT k: n!1 n The proof is complete. The special case of Proposition 4.6 (where k(
;
) is a continuous kernel and Tn := TnN ) seems to have prompted Anselone to develop the theory of collectively compact operator approximation and to extend several classical results about norm convergence. (See [17].) As we have seen in  T . Hence Lemma 2.2, if T is compact, then Tn cc ! T implies that Tn ! our results for  -convergence in Chapter 2 further extend the classical results.

209

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

We now give a number of convergent sequences of quadrature formul, which can be employed for constructing Nystrom and Fredholm approximations of an integral operator. Several quadrature formul arise as Riemann sums: For n = 1; 2; : : : consider a partition a = n;0 < n;1 <


< n;r(n) = b of the interval [a; b] and let ehn := maxfn;j ; n;j;1 : j = 1; : : : ; r(n)g. Choose tn;j in [n;j;1 ; n;j ] for j = 1; : : : ; r(n). Letting tn;0 := a, tn;r(n)+1 := b and hn := maxftn;j ; tn;j;1 : j = 1; : : : ; r(n) + 1g, we note that hn  2ehn as well as ehn  2hn since n;j 2 [tn;j ; tn;j+1 ] for j = 0; : : : ; r(n). Let wn;j := n;j ; n;j;1 for j = 1; : : : ; r(n), and for x : [a; b] ! Cj ,

Qn (x) :=

rX (n) j =1

rX (n)

wn;j x(tn;j ) =

j =1

x(tn;j )(n;j ; n;j;1 ): Z

b

Then Qn (x) is a Riemann sum for x. Hence Qn (x) ! x(t) dt for every a Riemann integrable function x on [a; b], provided ehn ! 0, or equivalently hn ! 0, as n ! 1. If x 2 C 0Z([a; b]), then we can describe the rate of convergence of b (Qn (x)) to x(t) dt in terms of the modulus of continuity of x: For a  > 0, let

!(x; ) := maxfjx(s) ; x(t)j : s; t 2 [a; b]; js ; tj  g: By the Mean Value Theorem for Integrals, for each j = 1; : : : ; r(n), there exists sn;j 2 [n;j;1 ; n;j ] such that

Qn (x) ;

b

Z

a



(n)  rX

x(t) dt =

j =1 r (n) X 

=

j =1

x(tn;j )(n;j ; n;j;1 ) ; 

Z n;j



x(t) dt

n;j;1



x(tn;j ) ; x(sn;j ) (n;j ; n;j;1 )

 !(x; h

en)

rX (n) j =1

(n;j ; n;j;1 ) = (b ; a)!(x; ehn ):

The most simple examples of this kind are the Rectangular Rules: Consider a := 0, b := 1 and n;j := j=n for j = 0; : : : ; n. For j = 1; : : : ; n, we

210

4. FINITE RANK APPROXIMATIONS

have wn;j := n;j ; n;j;1 = 1=n. Next, for j = 1; : : : ; n, let tn;j := j=n as in Example 4.3(a), or tn;j := (j ; 1)=n as in Example 4.3(b). Then for x 2 C 0 ([0; 1]), n   X

and

n   X or Qn (x) := n1 x j ;n 1

x nj j =1

Qn (x) := n1

Qn (x) ;

j =1

Z 1 0



x(t) dt  !(x; 1=n):

If in fact x 2 C 1 ([0; 1]), then !(x; 1=n)  kx0 k1 =n and hence

Qn (x) ;

Z 1 0

0

x(t) dt  kxnk1

for n = 1; 2; : : :

Other classical quadrature formul can also be interpreted as Riemann sums, as indicated in Example 4.9.

Proposition 4.7 Let T be a Fredholm integral operator on X := C 0 ([a; b]) with a continuous kernel k(
;
). Consider a convergent sequence (Qn ) of quadrature

formul, and let TnN be the Nystrom approximation of T based on Qn. Assume that for each n, the quadrature formula Qn gives a Riemann sum, that is, there is a partition a = n;0 < n;1 <


< n;r(n) = b of [a; b] such that tn;j 2 [n;j;1 ; n;j ] for j = 1; : : : ; r(n) and Qn (x) := rP (n) wn;j x(tn;j ), where wn;j := n;j ; n;j;1 for j = 1; : : : ; r(n). For j =1 s; t 2 [a; b], de ne

ks;t (u) := k(s; u)k(u; t) for u 2 [a; b]: Then for n = 1; 2; : : : we have

k(TnN ; T )T k ; k(TnN ; T )TnN k  (b ; a) sup !(ks;t ; ehn ); s;t2[a;b]

where !(
;
) denotes the modulus of continuity and ehn := maxfn;j ; n;j;1 : j = 1; : : : ; r(n)g.

211

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

Proof Let x 2 X and s 2 [a; b]. We have (TnN ; T )Tx(s) = = = =

rX (n)

rX (n)

j =1 Z b

j =1

a

wn;j k(s; tn;j )Tx(tn;j ) ;

wn;j k(s; tn;j ) k(tn;j ; t)x(t) dt ;

Z b h rX (n)

a j =1

Z b h rX (n)

a j =1

wn;j k(s; tn;j )k(tn;j ; t) ; wn;j ks;t (tn;j ) ;

b

Z

a

b

Z

a

b

Z

a

b

Z

a

k(s; u)Tx(u) du

k(s; u)

b

hZ

a

i

k(u; t)x(t) dt du i

k(s; u)k(u; t) du x(t) dt i

ks;t (u) du x(t) dt:

Since wn;j = n;j ; n;j;1 for j = 1; : : : ; r(n), we obtain

j(TnN ; T )Tx(s)j  (b ; a)kxk1 sup !(ks;t ; ehn ); t2[a;b]

as noted earlier. Hence k(TnN ; T )T k  (b ; a) sup !(ks;t ; ehn ): s;t2[a;b]

Again, we have (TnN ; T )TnN x(s) =

rX (n)

rX (n)

j =1

`=1

wn;j k(s; tn;j ) b

Z

; k(s; u) =

(n) hrX

a rX (n)

(n) hrX

`=1

j =1

wn;` b

Z

`=1

wn;` k(tn;j ; tn;` )x(tn;` ) i

wn;` k(u; tn;` )x(tn;` ) du

wn;j k(s; tn;j )k(tn;j ; tn;` ) i

; k(s; u)k(u; tn;`) du x(tn;` ) =

a rX (n)

(n) hrX

`=1

j =1

wn;`

Z

b

wn;j ks;tn;` (tn;j ) i

; ks;tn;` (u) du x(tn;` ): a

212

4. FINITE RANK APPROXIMATIONS

As above, we obtain

j(TnN ; T )TnN x(s)j  But

rX (n)

rX (n)

`=1

`=1

wn;` =

(n) rX

`=1



wn;` kxk1

sup

`=1;:::;r(n)

!(ks;tn;` ; ehn ):

(n;` ; n;`;1 ) = b ; a. Hence

k(TnN ; T )TnN k  (b ; a) sup !(ks;t ; ehn ); s;t2[a;b]

as desired.

Example 4.8

Consider the kernel



s  t  1; k(s; t) := s(1(1;;st))t ifif 00   t < s  1: It is continuous on [0; 1][0; 1]. Fix s; t 2 [0; 1] such that s  t. Then for u 2 [0; 1], we have 8 < (1

; s)(1 ; t)u2 if u  s; ks;t (u) := k(s; u)k(u; t) = : s(1 ; t)u(1 ; u) if s < u < t; st(1 ; u)2 if t  u: It can be seen that for  > 0, we have

!(ks;t ; )  4: This inequality holds for t  s as well, since the kernel k(
;
) is symmetric in s and t. Proposition 4.7 now shows that for the Fredholm integral operator T with this kernel, we have

k(TnN ; T )T k ; k(TnN ; T )TnN k  4(b ; a)ehn ; where ehn is as de ned in the proposition. Many quadrature formul arise from interpolatory projections considered in Subsection 4.1.2. Let a  tn;1 < tn;2 <


< tn;r(n)  b

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

213

and en;1 ; : : : ; en;r(n) be in C 0 ([a; b]) such that en;j (tn;k ) = j;k for j; k = 1; : : : ; r(n). Let

n x :=

rX (n) j =1

x(tn;j )en;j ; x 2 C 0 ([a; b]);

and consider the induced quadrature formula Qn given by

Qn(x) :=

a b

Z

where wn;j :=

b

Z

a

(n x)(t) dt =

rX (n) j =1

wn;j x(tn;j ); x 2 C 0 ([a; b]);

en;j (t) dt, j = 1; : : : ; r(n). Since for j = 1; : : : ; r(n),

Qn (en;j ) =

Z

b

(n en;j )(t) dt =

Z

b

en;j (t) dt; a a the quadrature formula Qn is exact on spanfen;1; : : : ; en;r(n)g. Also, for x 2 C 0 ([a; b]) and s 2 [a; b], we have (TnN n x)(s) =

rX (n) j =1

wn;j k(s; tn;j )(n x)(tn;j ) = (TnN x)(s);

since (n x)(tn;j ) = x(tn;j ) for j = 1; : : : ; r(n). Thus TnN n = TnN ; if the quadrature formula Qn is induced by an interpolatory projection n . Assume now that n is another interpolatory projection given by

n (x) :=

rX (n) i=1

x(tn;i )en;i ; x 2 C 0 ([a; b]):

If we employ n to de ne the Fredholm approximation TnF := n TnN , then for x 2 C 0 ([a; b]) and s 2 [a; b], we have (TnF x)(s) = =

rX (n)

(TnN x)(tn;i )en;i (s)

i=1 rX (n) h rX (n) i=1 j =1

i

wn;j k(tn;i ; tn;j )x(tn;j ) en;i (s):

214

4. FINITE RANK APPROXIMATIONS

We note that in using TnN , the kernel k(
;
) of the integral operator T is discretized in only the second variable, while in using TnF := n TnN the kernel is discretized in both the variables. p If the sequence (n ) of interpolatory projection satis es n ! I , then clearly

Qn (x) :=

b

Z

a

(n x)(t) dt !

b

Z

a

x(t) dt = Q(x)

for all x 2 C 0 ([a; b]), that is, the induced sequence of quadrature formul is convergent. It is possible, however, for a sequence of quadrature formul induced by a sequence (n ) of interpolatory projections to be p = I , as Example 4.10 shows. convergent even if n !

Example 4.9 Quadrature formul induced by piecewise linear interpo-

latory projections: Consider nodes tn;1 ; : : : ; tn;r(n) in [a; b]. In this case, the weights are given by

wn;j :=

Z 8a > > > > <

b

en;j (t) dt

tn;1 ; a + tn;2 ;2 tn;1 if j = 1, = > tn;j+1 ; tn;j;1 if j = 2; : : : ; r(n) ; 1, 2 > > t ; t n;r(n);1 if j = r(n). > : b ; tn;r(n) + n;r(n) 2

Observe that the quadrature formula Qn (x) :=

rX (n)

wn;j x(tn;j ) can be j =1 considered as a Riemann sum for x: If n;0 := 0, n;r(n) := 1 and n;j := (n) tn;j + tn;j+1 for j = 1; : : : ; r(n) ; 1, then Q (x) = rX x(tn;j )(n;j ; n 2 j =1 n;j;1 ), where tn;j 2 [n;j;1 ; n;j ] for j = 1; : : : ; r(n). Hence

Qn (x) ;

Z 1 0



x(t) dt  !(x; ehn );

where ehn := maxfn;j ; n;j;1 : j = 1; : : : ; r(n)g.

215

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

We shall now consider some special cases. (a) Compound Mid-Point Rule: Let a := 0, b := 1, tn;j := 2j2;n 1 , j = 1; : : : ; n, as in Example 4.3(c). Then w := 1 for all j = 1; : : : ; n and we obtain n;j

n

Qn (x) := n1 We also have

Qn (x) ;

Z 1 0

n 2 X

 x j2;n 1 ; x 2 C 0 ([0; 1]): j =1

00

x k1 for n = 1; 2 : : : and x 2 C 2 ([0; 1]): x(t) dt  k24 n2

(b) Compound Trapezoidal Rule: 1 , n > 1, j = 1; : : : ; n, as in Example Let a := 0, b := 1, tn;j := nj ; ;1 4.3(d). Then wn;1 := 2(n 1; 1) = wn;n ; wn;j := n ;1 1 for j = 2; : : : ; n ; 1 and we obtain " nX ;1  j ; 1  x(1) # 1 x (0) Qn (x) := n ; 1 2 + x n ; 1 + 2 ; x 2 C 0 ([0; 1]): j =2 We also have

Qn (x) ;

Z 1 0

00 x(t) dt  12(kxn ;k11)2 for n = 1; 2 : : : and x 2 C 2 ([0; 1]):

(c) Compound Gauss Two-Point Rule: Let a := 0, b := 1, n an even positive integer and let the nodes be as in Example 4.3(e). Then wn;j := 1=n for all j = 1; : : : ; n and we obtain for x 2 C 0 ([0; 1]),

Qn (x) := n1 We also have

Qn (x) ;

"

Z 1 0

n X

j =1 j odd

p

p

#

n  j ; 1 + (1= 3)   X x j ; (1n= 3) + x : n j =2 j even 

x k1 for n = 1; 2 : : : and x 2 C 4 ([0; 1]): x(t) dt  k270 n4

(4)

216

4. FINITE RANK APPROXIMATIONS

Some of these quadrature formul will be used in Section 5.4 to illustrate numerical approximations of integral operators. The Compound Simpson Rule is an example of a quadrature formula induced by a piecewise quadratic interpolatory projection. (See Exercise 4.9.) For error estimates of various compound quadrature rules mentioned above, we refer the reader to Section 7.4 of [27].

Example 4.10 Gauss-Legendre Rule: Let a := ;1, b := 1 and let tn;1 : : : ; tn;n be the roots of the Legendre polynomial of degree n. For j = 1; : : : ; n, let pn;j denote the Lagrange polynomial with nodes at tn;1 ;


; tn;n and de ne (n x)(t) :=

n X

x(tn;j )pn;j (t); x 2 C 0 ([;1; 1]):

j =1

Let Qn be the induced quadrature formula. It can be shown that if yk (t) := tk , t 2 [;1; 1], then

Qn (yk ) = and

wn;j :=

Z 1

;1

Z 1

;1

tk dt for k = 0; 1; : : : ; 2n ; 1

pn;j (t) dt =

Z 1

;1

p2n;j (t) dt > 0; j = 1; : : : ; n:

(See Theorem 9.6(b) of [55].) By Polya's Theorem, we see that (Qn ) is p = I , as we a convergent sequence of quadrature formul, although n ! have noted in part (iii) of Subsection 4.1.2. We also have

Qn (x ) ;

Z 1

;1

3 2n+1 (2n) x(t) dt  (n!)(2n2!)4 (2knx+ 1)k1

for n = 1; 2 : : : and x 2 C 2n ([ ; 1; 1]). (See Example 2.12 on page 108 of [22].) Finally, we consider a situation where one can estimate the rate at which k(TnN ; T )q T k and k(TnN ; T )q TnN k tend to zero, where q is a positive integer. For this purpose, we prove a preliminary result.

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

217

Lemma 4.11

Let T be a Fredholm integral operator on X := C 0 ([a; b]) with a continuous kernel k(
;
). Consider a convergent sequence (Qn ) of quadrature formul given by

Qn (x) =

rX (n) j =1

wn;j x(tn;j ) for n = 1; 2; : : : and x 2 X;

and

 := sup

(n) n rX

o

jwn;j j : n = 1; 2 : : : :

j =1 N Let Tn be the Nystrom approximation of

T based on Qn . If for some i+j nonnegative integers i and j , the (i; j )th partial derivative @ j ki exists @t @s at every (s; t) 2 [a; b][a; b], de ne o n @ i+j k ( s; t ) : s; t 2 [ a; b ] : i;j := sup @t j @si Let p be a positive integer.

(a) Assume that for j = 1; : : : ; p, the j th partial derivative of k(
;
) with respect to the rst variable exists and is a continuous function on [a; b][a; b]. Then for j = 1; : : : ; p and for all x 2 C 0 ([a; b]), Tx, TnN x 2 C j ([a; b]), and k(Tx)(j) k1  j;0 (b ; a)kxk1 ; k(TnN x)(j) k1  j;0  kxk1 : (b) Assume that for n = 1; 2; : : : and x 2 C p ([a; b]),

Qn(x) ;

Z

a

b

x(t)dt  ncpp kx(p) k1 ;

where cp is a constant, independent of n and x. Further, assume that for j = 1; : : : ; p, the j th partial derivative of k(
;
) with respect to the second variable exists and is continuous at every (s; t) 2 [a; b][a; b]. Then for all x 2 C p ([a; b]),

k(TnN ; T )xk1  ncpp

p   X j =0

p  kx(j) k : 1 j 0;p;j

218

4. FINITE RANK APPROXIMATIONS If in fact for a positive integer i and each j = 1; : : : ; p, the (i; j )th i+j partial derivative @ j ki of k(
;
) exists and is continuous at every @t @s (s; t) in [a; b][a; b], then Tx, TnN x belong to C i ([a; b]) for all x 2 C p ([a; b]), and

k (TnN ; T )x (i) k1  ncpp 



p   X j =0

p  kx(j) k : 1 j i;p;j

Proof (a) Let x 2 X and s 2 [a; b]. Fix j , 1  j  p. Dierentiating Tx(s) =

b

Z

a

k(s; t)x(t) dt and TnN x(s) =

rX (n) j =1

wn;j k(s; tn;j )x(tn;j )

j times with respect to s, we obtain Z b j (Tx)(j) (s) = @@skj (s; t)x(t)dt; a rX (n) j (TnN x)(j) (s) = wn;` @@skj (s; tn;` )x(tn;` ): `=1 Hence the desired bounds for k(Tx)(j) k1 and k(TnN x)(j) k1 follow easily. Note that j;0 is nite by the continuity of the j th partial derivative of k(
;
) with respect to the rst variable, and  is nite by Polya's Theorem. (b) For a xed s 2 [a; b], let ks(t) = k(s; t); t 2 [a; b]. By assumption, ks 2 C p ([a; b]) for each s 2 [a; b]. Let x 2 C p ([a; b]). Then by Leibnitz's Rule, p   X p k(p;j) x(j) for each s 2 [a; b]: (p) (ks x) = j s Thus for all s 2 [a; b],

j =0

Z b N jTn x(s) ; Tx(s)j = Qn (ks x) ; ks (t)x(t)dt  ncpp k(ks x)(p) k1 a p   X p  kx(j) k  ncpp 0;p;j 1 j =0 j

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

219

by our assumption on the sequence (Qn ) of the quadrature formul. Hence the desired bound for k(TnN ; T )xk1 follows easily. If for a positive integer i and j = 1; : : : ; p, the (i; j )th partial derivative of k(
;
) exists and is continuous on [a; b][a; b], then by considering the i kernel @@ski (
;
) in place of the kernel k(
;
), we obtain the desired bound for k[(TnN ; T )x](i) k1 for each x 2 C p ([a; b]).

Theorem 4.12 Let T be a Fredholm integral operator on X := C 0 ([a; b]) with a continuous kernel k(
;
). Consider a convergent sequence (Qn ) of quadrature formul and TnN be the Nystrom approximation of T based on Qn . Let p be a positive integer and assume that the sequence (Qn ) satis es

Qn (x) ;

b

Z

a

x(t)dt  ncpp kx(p) k1 for n = 1; 2; : : : and x 2 C p ([a; b]);

where cp is constant, independent of n and x. (a) Let for each i = 1; : : : ; p, the ith partial derivatives of k(
;
) with respect to the rst variable as well as the second variable exist and be continuous on [a; b][a; b]. Then   k(T N ; T )T k = O 1 = k(T N ; T )T N k:

n n np (b) Let in ifact for each i; j = 0; 1; : : :; p, the (i; j )th partial deriva+j k @ tive j i of k(
;
) exist and be continuous at every (s; t) 2 @t @s [a; b][a; b]. Then for n; q = 1; 2; : : :  q  q k(TnN ;T )q T k  (b;a) ndpp and k(TnN ;T )q TnN k   ndpp ; where dp is a constant, independent of n and q, while  is a constant, independent of n, q and p. n

Proof Let x 2 X . (a) By Lemma 4.11(a), Tx and TnN x belong to C p([a; b]), and for j =

1; : : : ; p,

k(Tx)(j) k  j;0 (b ; a)kxk1 ; k(TnN x)(j) k  j;0 kxk1 ;

220

4. FINITE RANK APPROXIMATIONS

where the constants j;0 and  are as de ned in Lemma 4.11. Hence by Lemma 4.11(b),

k(TnN ; T )Txk1

p X c p  k(Tx)(j) k p  np 0;p;j 1 j =0 j p   X p   kxk  (b ; a) ncpp 0;p;j j;0 1 j =0 j

and

k(TnN ; T )TnN xk1

 

p   X c p  k(T N x)(j) k p  np 0;p;j n 1 j =0 j p   X c p   kxk : p   np 0;p;j j;0 1 j =0 j

Hence both k(TnN ; T )T k and k(TnN ; T )TnN k are less than or equal to a constant times 1=np, as desired. (b) By Lemma 4.11(a), (Tn ; T )q Tx 2 C p ([a; b]) for all q = 1; 2; : : : We claim that for q = 1; 2; : : : ; n = 1; 2; : : : and i = 0; 1; : : : ; p; q ;1 k[(T N ; T )q Tx](i) k  (b ; a) cp p (cp ep ) kxk ; n

where

p = i=0max ;1;:::;p

p X j =0

1

npq

1

p p   and e = max X p  ; p i=0;1;:::;p i;p;j j i;p;j j;0 j =0 j

 

 

i;j being the constant introduced in Lemma 4.11, i; j = 0; : : : ; p. To prove our claim, we use mathematical induction on q. Let q = 1. By Lemma 4.11(a), we have k(Tx)(j) k1  j;0 (b ; a)kxk1 for j = 0; : : : ; p; and hence by Lemma 4.11(b),

k[(TnN ; T )Tx](i)k1

p X p  k(Tx)(j) k  ncpp i;p;j 1 j =0 j  (b ; a) cnp pp kxk1  

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

221

for n = 1; 2; : : : and i = 0; : : : ; p. Thus our claim holds for q = 1. Assume that our claim holds for some q  1 and let y := (TnN ; T )q Tx: Again, by Lemma 4.11(b),

k[(TnN ; T )q+1 Tx](i) k1 = k[(TnN ; T )y](i) k1 p   X c p  ky(j) k p  np 1 j i;p;j j =0

p   X

p  (b ; a) cp p (cp ep )q;1 kxk i;p;j 1 npq j =0 j p ep )q kxk  (b ; a) cp npp((cq+1) 1 for n = 1; 2; : : : and i = 0; : : : ; p. Thus our claim holds for q + 1 and the

 ncpp

induction is over. In particular, letting i = 0, we have for q = 1; 2; : : : and n = 1; 2; : : : q;1

k(TnN ; T )q Txk1  (b ; a) cp p (cnppq ep )

kxk1 :

We let dp := cp maxf p ; ep g and obtain for each xed q = 1; 2; : : :  q k(TnN ; T )q T k  (b ; a) ndpp

for n = 1; 2; : : :

Analogous estimate for k(TnN ; T )q TnN k can be proved similarly upon replacing b ; a by the constant  introduced in Lemma 4.11.

Example 4.13 Quadrature formul satisfying the hypothesis in Theorem 4.12: Several well-known sequences of quadrature formul satisfy the condition

Qn (x) ;

Z

a

b

x(t)dt  ncpp kx(p) k1 for n = 1; 2; : : : and x 2 C p ([a; b]);

mentioned in Theorem 4.12: the Compound Rectangular Rule satis es it for p = 1, the Compound Mid-Point and the Compound Trapezoidal Rules satisfy it for p = 2, the Compound Simpson and the Compound Gauss Two-Point Rules satisfy it for p = 4. (See Exercise 4.10 and Example 4.9(c).) Further, for any positive integer r, the Compound

222

4. FINITE RANK APPROXIMATIONS

Gauss r Point Rule satis es the above-mentioned condition for p = 2r. (See Section 7.4 of [27].) We note that if p is a positive integer and X := C p ([a; b]) with the norm given by

kxk =

p X j =0

kx(j) k1 ; x 2 X;

and if T is a Fredholm integral operator with a continuous kernel k(
;
) such that all the partial derivatives up to the order 2p exist and are  1  N continuous on [a; b][a; b], then kTn ; T k = O np , where TnN is a Nystrom approximation of T based on a quadrature formula satisfying the condition given in Theorem 4.12. This follows easily from Lemma 4.11.

4.2.3 Weakly Singular Integral Operators

In this subsection we extend the use of the Nystrom approximation to a class of Fredholm integral operators having discontinuous kernels. Let  2 C 0 (]0; 1]) be a nonnegative steadily decreasing function, that is, (r1 )  (r2 ) whenever 0 < r1 < r2  1, such that lim (t) = 1 and

Z 1 0

t!0+

(t) dt < 1. We say that  is weakly singular at 0.

Example 4.14 Weakly singular functions: The functions de ned by

(r) := r; ; r 2 ]0; 1]; where  2 ]0; 1[ is xed, (r) := ; ln r; r 2 ]0; 1]; are weakly singular at 0.

Lemma 4.15

Let a function  be a weakly singular at 0. Then

(a) for any  > 0, there exists  > 0 such that

Z 0



(t) dt <  and

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

(b) for any  2 ]0; 1[, the function s 2 [; 1 ; ] 7! decreasing.

Proof (a) The measure  de ned by (E ) :=

Z

E

223

Z s+

(t) dt is steadily

s;

(t) dt, where E is a

Lebesgue measurable subset of [0; 1], is absolutely continuous with respect to the Lebesgue measure. Hence for all  > 0, there exists  > 0 such that for any interval I in [0; 1] whose length is less than or equal to , (I ) is less than . Z s+ (b) For s 2 [; 1;], de ne g(s) := (t) dt. Then g is a continuously s; dierentiable function, and for s 2 ]; 1;[, g0 (s) = (s+);(s;)  0, since  is steadily decreasing. Let  : ]0; 1] ! IR be a weakly singular function at 0. Throughout this section, we let

k(s; t) := ( js ; tj ); s; t 2 [0; 1]; s 6= t; and consider the integral operator T de ned by (Tx)(s) :=

Z 1 0

k(s; t)x(t) dt; x 2 X; s 2 [0; 1]:

Let  2 ]0; 1] and de ne the truncated function  : [0; 1] ! IR by 

0  t  ,  (t) := ((t)) ifotherwise, This truncated function  induces a kernel k (
;
) de ned by

k (s; t) :=  ( js ; tj ); s; t 2 [0; 1]: If  := 1=n, we write n and kn in place of 1=n and k1=n , respectively. Let X := C 0 ([0; 1]) and n be a positive integer. We de ne the auxiliary Fredholm integral operator Ten on X by

T x)(s) :=

( en

Z 1 0

kn (s; t)x(t) dt; x 2 X; s 2 [0; 1]:

224

4. FINITE RANK APPROXIMATIONS

Since kn (
;
) is a continuous function on [0; 1][0; 1], Ten is a compact operator on X . For each positive integer n, we de ne the set

I (s; 1=n) := ft 2 [0; 1] : 0 < jt ; sj  1=ng: By Lemma 4.15(a), given  > 0, there exists an integer n0 such that for n > n0 , Z Z 1=n ( js ; tj ) dt = 2 (t) dt < 2 0

I (s;1=n)

for all s 2 [0; 1]. Hence, for n > n0 , x 2 X such that kxk1  1 and s 2 [0; 1], Z

j(Ten x)(s) ; (Tx)(s)j =



[(1=n) ; ( js ; tj )]x(t) dt

I (s;1=n)

Z

 2kxk1 ( js ; tj ) dt < ; I (s;1=n)

since  is nonnegative and steadily decreasing on ]0; 1]. In particular, n Tx 2 X for every x 2 X , and Ten ! T . Hence T is compact. (Compare Exercise 4.6.) The approximation of T we want to present is motivated by the following way of rewriting (Tx)(s): For s 2 [0; 1], (Tx)(s) =

Z 1 0

( js ; tj )[x(t) ; x(s)] dt + x(s)

Z 1 0

( js ; tj ) dt:

The continuity of x at s is supposed to mitigate the eect of the weak singularity of  at 0 in the rst integral and to makeZ this integral more 1 amenable for numerical integration than the integral ( js ; tj )x(t) dt. 0 This approach is called the Singularity Subtraction Technique and has been proposed by Kantorovich and Krylov. (See [45] and [18].) The basic idea is to truncate  near 0 and then use a Nystrom approximation of the resulting operator, rewritten as in the singularity subtraction technique. More precisely, let (Qn ) be a convergent sequence of quadrature formul with nodes tn;j such that 0  tn;1 <


< tn;n  1 and weights wn;j  0. We de ne the Kantorovich-Krylov approximation of T as follows.

225

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

For x 2 X and s 2 [0; 1], let

Z 1 n X K (Tn x)(s) := wn;j n ( js ; tn;j j )[x(tn;j ) ; x(s)] + x(s) ( js ; tj ) dt: 0 j =1

We consider the following additional hypothesis on the nodes and the weights of Qn : P ( There exists c > 0 such that wn;j  c(b ; a) tn;j 2I (H ) whenever 0  a < b  1; and I := ]a; b] or I := [a; b[: Note that the nodes and the weights for most of the quadrature formul given in Example 4.9 satisfy hypothesis (H ).

Lemma 4.16

Let a and b be real numbers such that 0  a < b  1. Let y : ]a; b] ! IR be a nonnegative steadily increasing function. Then under the hypothesis (H), for each n = 1; 2; : : : Z b X w y(t )  c(b ; a) y(b) + c y(t) dt: n;j a
n;j

n

a

Similarly, let y : [a; b[ ! IR be a nonnegative steadily decreasing function. Then under the hypothesis (H), for each n = 1; 2; : : : Z b X wn;j y(tn;j )  c(b ; a) y(a) + c y(t) dt:

n

atn;j
a

Proof

De ne In;i := ]a +(i ; 1)(b ; a)=n; a + i(b ; a)=n] for i = 1; : : : ; n. Then X

wn;j y(tn;j ) =

a


n X

X

wn;j y(tn;j ) i=1 tn;j 2In;i  n  i(b ; a) c(b ; a) X n

i=1

y a+

n

nX ;1 = c(b n; a) y(b) + c b ;n a y a + i(b n; a) i=1 Z b  c(b ; a) y(b) + c y(t) dt;

n



a



226

4. FINITE RANK APPROXIMATIONS

since y is steadily increasing. If y is steadily decreasing, the proof is similar.

Lemma 4.17 Let  2 ]0; 1], s 2 [0; 1] and n > 1=. Under the hypothesis (H), we have X



Z

wn;j n ( js ; tn;j j )  4c (t) dt:

js;tn;j j<

0

Proof

Let us write

X

wn;j n ( js ; tn;j j ) =

js;tn;j j<

X

wn;j n (s ; tn;j )

0s;tn;j <

+

X

wn;j n (tn;j ; s):

0tn;j ;s<

To nd an upper bound for the rst sum on the right side, consider a function y de ned by y(t) := n (s ; t) for t 2 ]a; s], where a := maxf0; s ; g. Then y is steadily increasing and Lemma 4.16 gives Z  c wn;j n (s ; tn;j )  n (1=n) + c n (t) dt: 0 0s;tn;j < X

But and so that

c (1=n)  c (1=n) = cZ 1=n  (t) dt n n n 0 Z 1=n 0

n (t) dt 

X



Z 0

n (t) dt 



Z 0

Z

(t) dt;



wn;j n (s ; tn;j )  2c (t) dt:

0s;tn;j <

0

In a completely analogous manner, consider a function y de ned by y(t) := n (t ; s) for t 2 [s; b[, where b := minfs + ; 1g. Then y is steadily decreasing and Lemma 4.16 gives exactly the same upper bound for the second sum.

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS

227

For x 2 X , s 2 [0; 1] and n = 1; 2; : : : de ne (Tn x)(s) :=

n X i=1

wn;i n ( js ; tn;i j )x(tn;i ):

The following result was proved in [52].

Proposition 4.18

Under the hypothesis (H), Tn cc ! T.

Proof

p First we prove that Tn ! T . Let x 2 X , kxk1 = 1 and  > 0. There exists  2 ]0; 1] such that



Z 0

n o (t) dt < 9 min 1; 31c :

For n = 1; 2; : : : we decompose (Tn ; T )x = A1 + A2;n + A3;n ; where, for s 2 [0; 1],

A1 (s) := =

A2;n (s) := A3;n (s) :=

Z 1 0

Z



0

 ( js ; tj )x(t) dt ;

0

( js ; tj )x(t) dt

[ ( js ; tj ) ; ( js ; tj )]x(t) dt;

n X j =1 n X j =1

Z 1

wn;j  ( js ; tn;j j )x(tn;j ) ; wn;j n ( js ; tn;j j )x(tn;j ) ;

Z 1 0

 ( js ; tj )x(t) dt and

n X j =1

wn;j  ( js ; tn;j j )x(tn;j ):

Clearly, for s 2 [0; 1],

jA1 (s)j  () +



Z 0

 ( js ; tj ) dt  3 ( t ) dt < 3 : Z

0

Let Te denote the Fredholm integral operator with continuous kernel k (
;
) and let TenN denote the Nystrom approximation of Te based on the

228

4. FINITE RANK APPROXIMATIONS

sequence (Qn ) of (convergent) quadrature formul. Then by Theorem p e 4.5, TenN ! T . Hence there exists some n0 depending on x such that for all n > n0 and all s 2 [0; 1], jA2;n (s)j  3 :

Also, by Lemma 4.15, for n > [1=] + 1 and all s 2 [0; 1], X jA3;n (s)j  wn;j jn ( js ; tn;j j ) ;  ( js ; tn;j j )j

 

js;tn;j j< X

;




wn;j n ( js ; tn;j j ) + ()

js;tn;j j< X

wn;j n ( js ; tn;j j ) + 2c()

js;tn;j j< 

Z

 6c (t) dt < 3 : 0

Hence for a given  > 0, there exists n1 := maxfn0; [1=] + 1g such that for n > n1 and all s 2 [0; 1], we have j(Tn x)(s) ; (Tx)(s)j < . This p proves that Tn ! T. 1 Next, we prove that the set Ee := n=[n fTnx : x 2 X; kxk1  1g is bounded and equicontinuous in X for some n0 . p Since Tn ! T , the sequence (kTnk) is bounded. As a consequence, 1 the set n[=1fTnx : x 2 X; kxk1  1g is bounded. The equicontinuity is proved as follows: Let  > 0. By Lemma 4.15(a), there exists  2 ]0; 1] such that 0



Z

0

(t) dt < 40 c :

By the uniform continuity of  in [; 1], there exists  > 0 such that for all s; t 2 [; 1], jt ; sj <  implies that j(t) ; (s)j < 2c . Let x 2 X be such that kxk1  1, n > n0 := [1=] + 1 and s; t 2 [0; 1] satisfy jt ; sj < . We have j(Tn x)(t) ; (Tn x)(s)j  B1;n (s; t) + B2;n (s; t) + B3;n (s; t) + B4;n (s; t); where X B1;n (s; t) := wn;i jn ( jt ; tn;i j ) ; n ( js ; tn;i j )j jt;tn;i j< js;tn;i j<

4.2. APPROXIMATIONS OF INTEGRAL OPERATORS



X

wn;i n ( jt ; tn;i j ) +

jt;tn;i j< Z



 8c (t) dt < 5 ;

B2;n (s; t) := =

B3;n (s; t) :=



0

X

X

229

wn;i n ( js ; tn;i j )

js;tn;i j<

wn;i jn ( jt ; tn;i j ) ; n ( js ; tn;i j )j

jt;tn;i j js;tn;i j X

wn;i j( jt ; tn;i j ) ; ( js ; tn;i j )j < 2 ;

jt;tn;i j js;tn;i j X

wn;i jn ( jt ; tn;i j ) ; n ( js ; tn;i j )j

jt;tn;i j< js;tn;i j X

wn;i n ( jt ; tn;i j ) + ()

jt;tn;i j<

wn;i

jt;tn;i j<

 3 ; < 4c (t) dt + 2c()  6c (t) dt < 20 0 0 X B4;n (s; t) := wn;i jn ( jt ; tn;i j ) ; ( js ; tn;i j )j Z



X

Z

jt;tn;i j js;tn;i j<

Z  3 : < 6c (t) dt < 20 0

Hence j(Tn x)(t) ; (Tn x)(s)j < . This shows that the set Ee is relatively cc compact in X . Thus Tn ! T. We remark that Tn and TnK are related by the formula

TnK = Tn + Un ; Un being de ned by (Un x)(s) := x(s)[(Te)(s) ; (Tn e)(s)] for x 2 X and s 2 [0; 1]; where

e(s) := 1 for s 2 [0; 1]:

The following result was proved in [1].

230

Theorem 4.19

4. FINITE RANK APPROXIMATIONS

 T , but T K ! K = T. Under the hypothesis (H), TnK ! n = T and Tn ! n

cc

Proof For x 2 X , we have kUnxk1  kxk1 kTne ; Tek1 , so that kUnxk1  p n kTncce ; Tek1 which tends to 0 since Tn ! T . Thus Un ! O. Since Tn ! T and T is compact, it follows from Lemma 2.2 that TnK := Tn +  T. Un ! cc = O since otherwise Un ; T would be compact and hence Un Also, Un !

would be compact. This is not the case as we now show. If (Tn ; T )e is a (nonzero) constant function, then Un is a (nonzero) scalar multiple of the identity operator, which cannot be compact. Assume now that (Tn ; T )e is not a constant function. Then sp(Un ) is the range of the function (Tn ; T )e, as we have seen in Example 1.17. Since (Tn ; T )e is a nonconstant continuous function, its range is an uncountable set. But the spectrum of a compact operator is countable cc = T. (Remark 1.34). Hence, by Lemma 2.2(d), TnK ! Finally, since T and Tn satisfy then hypotheses of Proposition 4.6, n = T . Again by Lemma 2.2(d), TnK ! = T. Tn ! The preceding result is one of the reasons for considering  -convergence rather than norm convergence or collectively compact convergence in Chapter 2.

4.3 A Posteriori Error Estimates Suppose we wish to nd a nonzero isolated spectral value  of T 2  T , then we have seen in Corollary 2.13 that for each BL(X ). If Tn ! large n, there is some n 2 sp(Tn ) such that n ! . Suppose we have calculated such n for some n. It is important to know how close this n is to . Theorem 2.17 shows that if  is a nonzero simple eigenvalue of T and if  is such that 0 <  < dist(; sp(T ) n fg), then for all large enough n,

231

4.3. A POSTERIORI ERROR ESTIMATES

there is a simple eigenvalue n of Tn which satis es

jn ; j  2 jzmax kR(T; z )k k(Tnk;' Tk)'n k ; ;j  n

=

where 'n is an eigenvector of Tn corresponding to n . Since we may be able to calculate 'n , the factor k(Tn ; T )'n k=k'nk in the preceding estimate is computable. However, it is dicult to estimate the other factor max kR(T; z )k, in general. jz;j= In case X is a Hilbert space and T is a normal operator, then for every z 2 re(T ), the operator R(T; z ) is also normal and hence

kR(T; z )k = (R(T; z )) = dist(z;1sp(T )) by Proposition 1.41(a) and 1.10(c). Further, if we choose  > 0 such that   dist(; sp(T ) n fg)=2, then dist(z; sp(T )) =  for every z satisfying jz ; j = , so that kR(T; z )k = 1=. Thus in this case,

jn ; j  2 k(Tnk;' Tk)'n k n

for all large n. However, it is dicult to say exactly how large the integer n need be for the preceding inequality to hold. Further, the eectiveness of this inequality depends on our being able to calculate an eigenvector 'n of Tn corresponding to n . Let us consider the following question: Let Te 2 BL(X ) be an approximation of T and suppose that a nonzero eigenvalue e of Te is found. How close is e to some eigenvalue of T ? The Krylov-Weinstein Inequality 1.41(c) says that there is some  2 sp(T ) such that e k : kxk = 1g: je ; j  inf fkTx ; x

Again, it is not easy to nd a sharp upper bound for the right side of this inequality. In this respect, we give a result of Brakhage [24] for integral operators which does not require any calculation of an eigenvector of Te corresponding to e. One may refer to Section 3.16 of [22] for other error bounds for computed eigenvalues of integral operators. Since these bounds are available only after the computation of an approximate eigenvalue e, they are called a posteriori error estimates.

232

4. FINITE RANK APPROXIMATIONS

Lemma 4.20

Let X = C 0 ([a; b]) and T be a Fredholm integral operator with a continuous kernel k(
;
). Consider a Nystrom approximation of T given by e (s) := Tx

r X j =1

wj k(s; tj )x(tj ); x 2 X; s 2 [a; b];

where a  t1 <


< tr  b and wj  0 for each j = 1; 2; : : : ; r. Let  :=

r X j =1

wj and for s; t 2 [a; b], de ne

d1 (s; t) := d2 (s; t) :=

r X j =1 r X j =1

wj k(s; tj )k(tj ; t) ; wj k(tj ; s)k(tj ; t) ;

Zb

a Zb a

k(s; u)k(u; t) du; k(u; s)k(u; t) du:

Assume that

jd (s; tj )j  for all s 2 [a; b] and j = 1; : : : ; r; 1

and that

jd (ti ; tj )j  for all i; j = 1; : : : ; r: 2

Then for every x 2 X , we have

(i) (ii)

Zb Zab

a

r

e (s)j ds  (b ; a) P wj jx(tj )j j(Te ; T )Tx 2

r

2

2

j =1

e (u)j du  P wj (jTx e (tj )j ; jx(tj )j ): jTx 2

j =1

2

2

233

4.3. A POSTERIORI ERROR ESTIMATES

Proof Let x 2 X .

(i) For s 2 [a; b], we have e (s) = (Te ; T )Tx

r X

e )(ti ) ; wi k(s; ti )(Tx

Zb

a

e )(u) du k(s; u)(Tx

Zb ri=1 r X X = wi k(s; ti ) wj k(ti ; tj )x(tj ) ; k(s; u) wj k(u; tj )x(tj ) du a i=1 j =1 j =1 Zb r hX r i X = wi k(s; ti )k(ti ; tj ) ; k(s; u)k(u; tj ) du wj x(tj ) a j =1 i=1 r X = d1 (s; tj )wj x(tj ); j =1

r X

so that e (s)j  j(Te ; T )Tx 2

r X j =1

wj jd1 (s; tj )j2

r  X j =1



wj jx(tj )j2 :

r P wj = , we have j =1 Zb r X e (s)j2 ds  2 (b ; a) j(Te ; T )Tx wj jx(tj )j2 : a j =1

Since wj  0 for each j = 1; : : : ; r and

(ii) We note that Zb

a

Z b X r 2 wi k(u; ti )x(ti ) du a i=1 Z b X r r  X  = wi k(u; ti ) x(ti ) w` k(u; t` )x(t` ) du a i=1 `=1 Zb r X r X = wi w` x(ti )x(t` ) k(u; ti )k(u; t`)du: a i=1 `=1

e (u)j du = jTx 2

Similarly, for each j = 1; : : : ; r, we have r X

e (tj )j = jTx 2

i=1

2

wi k(tj ; ti )x(ti )

234

4. FINITE RANK APPROXIMATIONS

= = so that

r X j =1

r X

i=1 r X r X i=1 `=1

Hence

e (tj )j2 ; wj jTx

i=1 `=1

Zb

r  X `=1



w` k(tj ; t` )x(t` )

wi w` x(ti )x(t` )k(tj ; ti )k(tj ; t`);

r X r X

e (tj )j2 = wj jTx

r X

wi k(tj ; ti ) x(ti )

wi w` x(ti )x(t` )

e (u)j du = jTx

r X j =1

wj k(tj ; ti )k(tj ; t` ):

r X r X

wi w` x(ti )x(t` )d2 (ti ; t` ) i=1 `=1 r X 2  wi w` jx(ti )j jx(t` )j jd2 (ti ; t`)j  wi jx(ti )j i=1 `r=1 i=1 r r X X  X 2  wi wi jx(ti )j =  wi jx(ti )j2 ; i=1 i=1 i=1 j =1 r X r X

so that

Zb

a

a

e (u)j du  jTx 2

2

r X j =1





e (tj )j2 ; jx(tj )j2 : wj jTx

as desired.

Proposition 4.21

e ; ; be as in Lemma 4.20. Assume further that Let X; T; T; Zb

a

k(s; u)k(t; u)du =

Zb

a

k(u; s)k(u; t)du

for all s; t 2 [a; b]. Let e be an eigenvalue of Te such that   p jej2 > 12  + 22 + 4 2 (b ; a) : Then there is a nonzero eigenvalue  of T such that "

je ; j  e := e(2b ; a) jj ; 

#1=2

:

235

4.3. A POSTERIORI ERROR ESTIMATES Further, we have the following bounds for the relative error:

e

 j j;j j  e

jej + 

e

e : jej ; e

Proof

Let Y := L2 ([a; b]). For x 2 Y and s 2 [a; b], let

Ux(s) =

Zb

a

k(s; t)x(t)dt:

Then U 2 BL(Y ), R(U )  X and U jX;X = T . Hence a nonzero complex number  is an eigenvalue of U if and only if  is an eigenvalue of T and then N (U ; IY ) = N (T ; I ), where IY is the identity operator on Y . Our assumption on the kernel k(
;
) shows that U U = UU , that is, U is a normal operator on Y . Let e be an eigenvalue of Te satisfying the stated inequality and let 'e be a corresponding eigenvector. Then 'e 2 Y and by the KrylovWeinstein Inequality (Proposition 1.41(c)), there exists  2 sp(U ) such that

je ; j  kU 'ek';ek'ek : e

2

2

But by Lemma 4.20, we have kU 'e ; e'ek2 = k(T ; Te)'ek2 = k(T ; Te)Te'ek2 e 2 k'ek2 k'ek2 2

2

kT 'ek

2

2 2 r P 2 (b ; a) wj j'e(tj )j 2 (b ; a) ; j =1  Pr  = e2 jj ;  wj jTe'e(tj )j2 ; j'e(tj )j2 j =1 since Te'e(tj ) = e'e(tj ) for j = 1; : : : ; r. Hence #1=2 "  ( b ; a ) e : j ; j  e 2 jj ;  2

2

Note that jej2 ;  > 0 because of the assumed lower bound for jej2 . It remains to show that  is an eigenvalue of T . First we claim that  6= 0. For if  = 0, then h i1=2 jej  (b ; a) ; that is, jej4 ; jej2 ; 2 (b ; a)  0:

jej ;  2

236

4. FINITE RANK APPROXIMATIONS

But this is impossible since jej2 is greater than each of the roots of the preceding quadratic polynomial in jej2 . Thus  6= 0. Being a nonzero spectral value of the compact operator U on Y , we see that  is an eigenvalue of Y and hence of T . Further, since jej > e we have 0 < jej;e  jj  jej+e. Consequently,

e

 j j;j j  e

jej + 

e

e ; jej ; e

as desired. For k(
;
) 2 C 0 ([a; b][a; b]) and xed s; t 2 [a; b], de ne ks;t (u) := k(s; u)k(u; t); ks;t (u) := k(u; s)k(u; t); u 2 [a; b]: Let Qe denote the quadrature rule used in de ning a Nystrom approximation Te of T , that is,

Qe(x) =

r X j =1

wj x(tj ); x 2 C 0 ([a; b]):

Then the errors d1 (s; t) and d2 (s; t) de ned in Lemma 4.20 can be written as follows:

d1 (s; t) = Qe(ks;t ) ;

Zb

a

ks;t (u)du; d2 (s; t) = Qe(ks;t ) ;

Zb

a

ks;t (u)du:

This observation allows us to obtain an upper bound for jd1 (s; t)j and an upper bound for jd2 (s; t)j in respect of several well-known quadrature rules, provided the kernel k(
;
) is suciently smooth. (See Example 4.9 and Exercise 4.10.) For a positive integer p, assume that all partial derivatives of k(
;
) up to the p th order exist and are continuous on [a; b][a; b]. For i; j = 0; : : : ; p and i + j  p, let o n @ i+j k i;j := sup @t j @si (s; t) : s; t 2 [a; b] ; as in Lemma 4.11. Then for all s; t 2 [a; b], it follows that ks;t , ks;t 2 C p ([a; b]) and

(p)

ks;t

1

p   X p j =0

j p;j;0 0;j ;

(p)

ks;t

1

p   X p j =0

j p;j;0 j;0 :

237

4.3. A POSTERIORI ERROR ESTIMATES

Qe

p

Compound Rectangular Rule 1



1 1 2r

1 1 2r

Compound Mid-Point Rule

2

1 2 24 r2

1 2 24 r2

Compound Trapezoidal Rule

2

1 2 12 (r ; 1)2

1 2 12 (r ; 1)2

Compound Simpson Rule

4

1 4 180 (r ; 1)4

1 4 180 (r ; 1)4

Compound Gauss Two-Point Rule Gauss-Legendre Rule

1 4 1 4 270 r4 270 r4 3 (r!)3 2r 2r 2r (2(rr!)!)4 (2r + 1) (2r!)4 (2r + 1) 4

Table 4.1

Let 

(p)

 (p)

: s; t 2 [ a; b ] : : s; t 2 [ a; b ] ;

:= sup k p := sup ks;t p s;t 1 1

If a = 0, b = 1 and Qe denotes a quadrature rule with r nodes, r  2, then the constants and can be chosen as in Table 4.1. Note rthat in each P of the cases mentioned above, wj  0 for each j and  = wj = 1. j =1

238

4. FINITE RANK APPROXIMATIONS

Example 4.22

Let X = C 0 ([0; 1]) and for x 2 X , consider

Tx(s) :=

Z

1 0

est x(t)dt; s 2 [0; 1]:

Since k(s; t) = k(t; s) = est 2 IR for all s; t 2 [0; 1], the kernel k(
;
) satis es the requirements of Proposition 4.21. For xed s; t 2 [0; 1], we have ks;t (u) = e(s+t)u = ks;t (u); u 2 [0; 1]; and hence for any positive integer p, (p) (p) ks;t (u) = (s + t)p e(s+t)u = ks;t (u); u 2 [0; 1]: Thus

kks;tp k1 = kks;tp k1  2pe : ( )

( )

2

Let Te be a Nystrom approximation of T based on one of the quadrature rules Qe with r nodes, r  2, mentioned before. The constant = can be chosen as inr Table 4.2. P Since  = wj = b ; a = 1, Proposition 4.21 shows that if e is a j =1

p

nonzero eigenvalue of Te such that jej2 > (1 + 5) =2, then there is a nonzero eigenvalue  of T such that

je ; j  q : jej ; 2

In particular, let Qe be the Gauss Two-Point rule with 100 nodes. N are computed as follows: Then the four largest eigenvalues of Te := T100

e(1) := (0:13530301645)101 e(3) := (0:3560748)10;2

e(2) := 0:105983223 e(4) := (0:763795)10;4: 4 2 e 100;4  (See Example 5.22.) In this case, we can let = := 2270 p (0:44)10;8. For j 2 f1; 2; 3g, we see that je(j )j > (1 + 5) =2. Hence there is a nonzero eigenvalue (j ) of T such that 8 < (0:4)10;8 if j = 1; e q j(j ) ; (j )j   : (0:5)10;7 if j = 2; 2 e (0:2)10;5 if j = 3: j(j )j ;

239

4.3. A POSTERIORI ERROR ESTIMATES

Qe

=

Compound Rectangular Rule Compound Mid-Point Rule Compound Trapezoidal Rule

e2 r e2 6r 2 e2

3(r ; 1)2 4e2

Compound Simpson Rule

45(r ; 1)4

Compound Gauss Two-Point Rule Gauss-Legendre Rule

8e2 135r4 (r!)3 4r e2 (2r!)4 (2r + 1)

Table 4.2 This shows that for j 2 f1; 2; 3g, the integral part of (j ) is the same as the integral part of e(j ), and the rst 8 decimal digits of (1) are the same as those of e(1), the rst 7 decimal digits of (2) are the same as those of e(2), and the rst 5 decimal digits of (3) are the same as those of e(3). Thus we obtain

(1) = 1:35303016 : : : (2) = 0:1059832 : : : (3) = 0:00356 : : :

p

On the other hand, since je(4)j  (1 + 5) =2, we are not in a position N of T , whether T has other to say, on the basis of the approximation T100 nonzero eigenvalues. (See Example 4.17.) e (j )j Let j 2 f1; 2; 3g. Consider the relative error j(jj); (j )j and de ne

240

4. FINITE RANK APPROXIMATIONS

, so that je(j )j2 ;

j := q

j

Thus

je(j )j + j

 j(jj);(j )j(j )j  e

j

je(j )j ; j

:

e (1)j ;8 (0:240)10;8  j(1)j; (1)j  (0:241)10 ; e (2)j  (0:393)10;6; (0:392)10;6  j(2)j; (2)j e (3)j ;3 (0:348)10;3  j(3)j; (3)j  (0:349)10 ; and we obtain a fairly good idea about the relative errors in approximating three eigenvalues of the operator T by the three largest eigenvalues N. of the operator T100

4.4 Exercises

Unless otherwise stated, X denotes a Banach space over Cj , X 6= f0g and T 2 BL(X ). Prove the following assertions. 4.1 Let (n ) be a sequence of projections in BL(X ) such that n y ! y p n for every y 2 R(T ). Then TnP ! T . If T is compact, then TnP ! T;  T, TG !  T , kT G ; T S k = and if in addition (kn k) is bounded, TnS ! n n n k(TnG ; T )n k = k(TnP ; T )n k ! 0. 4.2 Let (n ) be a sequence of projections in BL(X ). Let Y be a subspace of X such that R(T )  Y and F be a linear map from Y to X such that F T 2 BL(X ). If for some constant c and a sequence n ! 0, we have kn y ; yk  ckF (y)kn for all y 2 Y , then kTnP ; T k, k(TnS ; T )T k, and k(TnG ; T )T k are all O(n ); and if in addition (n ) is bounded, then k(TnS ; T )TnS k and k(TnG ; T )TnGk are also O(n ). 4.3 Let X be a Hilbert space over Cj and (np) be a sequence of orthogonal projections de ned on X nsuch that n ! I . If T is a compact n n operator on X , then TnP ! T , TnS ! T and TnG ! T.

241

4.4. EXERCISES

4.4 Let X := L1([a; b]), a = tn; < tn; <


< tn;n = b, and hn := maxftn;j ; tn;j; : j = 1; : : : ; ng ! 0. For x 2 X , let 0

1

1

1

Zt

n;j

(n x)(t) := t ; t n;j n;j ;1 t

n;j

;1

x(s) ds; t 2 [tn;j;1 ; tn;j [; j = 1; : : : ; n:

Then n2 = n , kn k = 1 and if x 2 C 0 ([a; b]), then kn x ; xk1 ! 0. p Let T be such that Tx 2 C 0 ([a; b]) for every x 2 X . Then TnP ! T , and n   if T is compact, TnP ! T , TnS ! T and TnG ! T . (Hint: Exercise 4.1.)

4.5 Let X be the linear space of all complex-valued bounded functions on [a; b] with the sup norm k
k1 . Let a = tn;0 < tn;1 <


< tn;n = b, hn := maxftn;j ; tn;j;1 : j = 1; : : : ; ng ! 0 and sn;j 2 [tn;j;1 ; tn;j [, j = 1; : : : ; n. For x 2 X , let (n x)(t) := x(sn;j ); t 2 [tn;j;1 ; tn;j [; j = 1; : : : ; n; (n x)(b) := x(b): Then n2 = n , kn k = 1, and if x 2 C 0 ([a; b], kn x ; xk1 ! 0. Let X := L1 ([a; b]) and T be such that Tx 2 C 0 ([a; b]) for every x 2 X. p n (a) We have TnP ! T , and if T is compact, TnP ! T. 0 (b) Let X0 := C ([a; b]) and T0 = TjX ;X . Then T0 , T0S 2 BL(X0 ), p  T . (Hint: Exercise 4.1.) T0S ! T0 and if T is compact, T0S ! 0 0

0

4.6 Let k(
;
) be a complex-valued Lebesgue measurable function on [a;b][a;b] such that nZ b o (i) sup jk(s; t)jdt : s 2 [a; b] < 1 and Z ba k (s + ; t) ; k (s; t) dt = 0 for every s 2 [a; b]. (ii) lim !0 a ;

(For example, let k(s; t) = ln js ; tj or k(s; t) = js ; tj for s; t 2 [;1; 1], where ln denotes the natural logarithm and 0 <  < 1.) Let X = L1 ([a; b]) and for x 2 X , (Tx)(s) =

Zb

a

k(s; t)x(t)dt; s 2 [a; b]:

Then Tx 2 C 0 ([a; b]) for every x 2 X and T : X ! X is a compact linear operator. As such, the results of Exercises 4.4 and 4.5 apply in this case.

242

4. FINITE RANK APPROXIMATIONS

4.7 (Petrov-Galerkin) Let (n ), (en ) be sequences of projections in BL(X ) such that n ! I , en ! I . Let TnPG := en Tn. Then TnPG ! T .  T ; and if in addition   ! I  , If T is a compact operator, then TnPG ! n en ! I  , then TnPG ! T . p

p

p

p

p

n

4.8 Let X = C ([a; b]); a = tn; <


< tn;n = b and hn = maxftn;j ; tn;j; : j = 1; : : : ; ng. If n denotes the piecewise linear interpolatory 0

1

1

projection given in part (i) of Subsection 4.1.2, then kn x ; xk1  81 kx00 k1 (hn )2 for all x 2 C 2 ([a; b]): (Hint: If t 2 [tn;j;1 ; tn;j ], then n x(t) = x(tn;j;1 ) + [x(tn;j ) ; x(tn;j;1 )] ;1 = x(t)+ x00 (s) (t ; t  t t ;;tn;j n;j ;1 )(tn;j ; t), where s 2 [tn;j ;1 ; tn;j ], 2 n;j tn;j ;1 by applying Rolle's Theorem twice.)

4.9 Let X := C ([a; b]), n and r(n) be positive integers, a = tn; < tn; <


< tn;r n = b and etn;j 2 ]tn;j; ; tn;j [ for j = 1; : : : ; r(n). De ne the piecewise quadratic interpolatory projection n : X ! X as follows. For x 2 X , let (n x)j t ; ;t be the unique quadratic polynomial on 0

1

0

1

( )

[

n;j

n;j ]

1

[tn;j;1 ; tn;j ] which agrees with x at tn;j;1 ; etn;j and tn;j , j = 1; : : : ; r(n). If hn := maxftn;j ;tn;j;1 : j = 1; : : : ; r(n)g ! 0 and there are constants e

and  satisfying 0 <   such that   etn;j ; tn;j  for all n

tn;j ; tn;j;1

p and j = 1; : : : ; r(n), then n ! I . For integers j , k and ` satisfying n;j 0  j; k; `  r(n), let pk;` denote the quadratic polynomial which takes n;j denote the value 1 at tn;j and the value 0 at tn;k and etn;` ; and let pek;` the quadratic polynomial which takes the value 1 at etn;j and the value 0 at tn;k and tn;`. De ne en;0 ; : : : ; en;r(n) and een;1; : : : ; een;r(n) in C 0 ([a; b]) as follows: For j = 0; : : : ; r(n), let

8 n;j < pj ;1;j (t) if t 2 [tn;j ;1 ; tn;j ], en;j (t) := : pn;j j +1;j +1 (t) if t 2 [tn;j ; tn;j +1 ],

0

otherwise,

where tn;;1 := a ; 2, etn;;1 := a ; 1, etn;r(n)+1 := b + 1 and tn;r(n)+1 := b + 2, and for j = 1; : : : ; r(n), let

een;j (t) :=

 n;j pej;1;j (t) if t 2 [tn;j;1 ; tn;j ],

0

otherwise.

243

4.4. EXERCISES

1

6 a

1

6

1

t~n;1

6

-

tn;1

b

the function en;0

a

1

6

t~n;r(n)

tn;r(n);1 the function en;r(n)

b

t~n;j

t~n;j+1 t~n;j a tn;j;1 tn;j tn;j+1 b a tn;j;1 tn;j tn;j+1 b the function en;j ; 0 < j < r(n) the function e~n;j ; 1  j  r(n) Then for x 2 C 0 ([a; b]),

n x =

rX (n) j =0

x(tn;j )en;j +

rX (n) j =1

x(etn;j )een;j :

Also, we have

kn x ; xk1  61 kx k1 (hn ) (3)

3

for all x 2 C 3 ([a; b]):

4.10 Let a := 0, b := 1, n an odd integer  3, r(n) := (n ; 1)=2, tn;j := 2j=(n ; 1) for j = 0; : : : ; (n ; 1)=2, and etn;j := (2j ; 1)=(n ; 1) for j = 1; : : : ; (n ; 1)=2 in Exercise 4.9. Then the sequence (n ) of piecewise quadratic interpolatory projections induces a sequence (Qn ) of convergent quadrature formul known as the Compound Simpson Rule. In fact, for x 2 C 0 ([a; b]), we have 1

"

Qn (x) = 3(n ; 1) x(0)+4 We also have

nX ;1)=2  2j ; 1 

(

j =1

nX ;3)=2  2j 

(

#

x n ; 1 +2 x n ; 1 + x(1) : j =1

Z Q (x) ; b x(t)dt  kx(4) k1 for n = 1; 2; : : : and x 2 C 4 ([a; b]): n 180(n ; 1)4 a

244

4. FINITE RANK APPROXIMATIONS

4.11 For a positive integer n and nodes a = tn;0  tn;1 <


< tn;n  tn;n+1 = b, let Xn denote the subspace of X := C 0 ([a; b]) consisting of all x 2 C 2 ([a; b]) such that x is a polynomial of degree  3 on each of the subintervals [tn;1 ; tn;2 ]; : : : ; [tn;n;1; tn;n ]. (a) Let a = tn;1 and b = tn;n . Then for each j = 1; : : : ; n, there is a unique en;j 2 Xn 00such that en;j (tn;k ) = j;k for k = 1; : : : ; n 00 and en;j (tn;1 ) = 0 = en;j (tn;n ). The projection n given by n x = n P x(tn;j )en;j , x 2 X is known as a natural cubic spline interpolatory j =1 projection. (Hint: dim Xn = n + 2.) (b) Let a < tn;1 and b > tn;n . Then for each j = 0; 1; : : : ; n +1, there is a unique en;j 2 Xn such that en;j (tn;k ) = n;k for k = 0; 1; : : :; n +1, and en;j is a polynomial of degree  1 on each of the two subintervals [a; tn;1] and [tn;n ; b]. We have e00n;j (tn;1 ) = 0 = e00n;j (tn;n ) for each j = 0; : : : ; n+1. (Hint: If Xen := fx 2 Xn : x is a polynomial of degree  1 on each of the two subintervals [a; tn;1 ] and [tn;n ; b]g, then dim Xen = 4(n ; 1) + 2 + 2 ; 3n = n.) 4.12 Let X = C ([a; b]) and p be a positive integer. Suppose that R(T )  C p ([a; b]) and k(Tx) p k1  p kxk1 for some constant p and all x 2 X . Let (n ) be a sequence of interpolatory projections de ned on X and hn ! 0, where hn is the mesh of the partition a  tn; <


< tn;n  b. If kn y ; yk1  cp ky p k1 (hn )p for some constant cp and all y 2 C p ([a; b]), then kTnP ; T k, k(TnS ; T )T k and k(TnG ; T )T k are all O((hn )p ). If n ! I , then k(TnS ; T )TnS k and k(TnG ; T )TnGk are also O((hn )p ). In particular, if T is a Fredholm integral operator on X with a continuous kernel k(
;
) whose partial derivatives up to the pth order with respect to the rst variable exist and are continuous on [a; b][a; b], then the preceding results hold for p = 1 if n is a 0

( )

( )

1

p

piecewise linear interpolatory projection; for p = 2 if n is a piecewise linear interpolatory projection with tn;1 = a, tn;n = b; for p = 3 if n is a piecewise quadratic interpolatory projection as given in Exercise 4.9; and for p = 4 if n is a cubic spline interpolatory projection as given in part (ii) of Subsection 4.1.2. (Hint: Exercise 4.2 with Y = C p ([a; b]) and Fx = x(p) , parts (i) and (ii) of Subsection 4.1.2, Exercise 4.8 and Exercise 4.9.)

4.13 Let tn;1; : : : ; tn;n 2 [;1; 1] be the roots of the nth Legendre polynomial and let n denote the Lagrange interpolatory projection with nodes at tn;1 ; : : : ; tn;n.

245

4.4. EXERCISES

(a) (Vainikko) If T : L2 ([;1; 1]) ! L2 ([;1; 1]) is a compact operator, n then TnP ! T. (b) (Sloan-Burn) If T : C 0 ([;1; 1]) ! C 0 ([;1; 1]) is a Fredholm integral operator with a continuous kernel, then TnS cc ! T and hence  T . (Hint: Result of Erdos and Turan stated in the text.) TnS !

4.14 Let a = tn; < tn; <


< tn;n; < tn;n = b and d be a positive 0

1

1

(d) integer. For j = 1; : : : ; n, let n;j denote the orthogonal projection 2 de ned on L ([tn;j;1 ; tn;j [) whose range equals the set of all polynomials on [tn;j;1 ; tn;j [ of degree  d. De ne n(d) : L2 ([a; b]) ! L2 ([a; b]) as follows. For x 2 L2 ([a; b]),



(d) (n(d) x)(t) := n;j xj[t

n;j

;1 ;t



n;j [

(t); t 2 [tn;j;1 ; tn;j [;

(n(d) x)(b) := (n(d) x)(tn;n;1 ): Then n(d) is an orthogonal projection de ned on L2 ([a; b]) whose range is contained in the set of all piecewise polynomials of degree  d and p I . (Hint: If n denotes the orthogonal projection on piecewise n(d) ! constant functions as given in Subsection 4.1.3, then kn(d) x ; xk  kn x ; xk for every x 2 L2([a; b]).) 2

2

4.15 Let X := C 0 ([a; b]) and T be a Fredholm integral operator with a continuous kernel k(
;
). For each positive integer n, let a  tn;1 <


< tn;r(n)  b and en;1 ; : : : ; en;r(n) 2 X such that en;j (tn;k ) = j;k , j; k = 1; : : : ; r(n). Let n denote the corresponding interpolatory projection. For a xed s 2 [a; b], de ne ks (t) := k(s; t), t 2 [a; b], and let kn (s; t) := (n ks )(t) =

rX (n) j =1

k(s; tn;j )en;j (t); s; t 2 [a; b]:

For a xed t 2 [a; b], de ne kt (s) := k(s; t), s 2 [a; b], and let rX (n) t ln (s; t) := (n k )(s) = k(tn;j ; t)en;j (s); j =1

s; t 2 [a; b]:

The degenerate kernels kn (
;
) and ln (
;
) are obtained by interpolating the kernel k(
;
) in the second variable and in the rst variable, respectively.

246

4. FINITE RANK APPROXIMATIONS

(a) If kn (ks ) ; ks k1 ! 0 uniformly in s 2 [a; b], then kkn (
;
) ; n k(
;
)k1 ! 0, so that TnD ! T . (cf. (i) of Subsection 4.2.1.) p n (b) For the kernel ln (
;
), TnD = TnP , so that TnD ! T if n ! I.

4.16 Let X := L ([a; b]) and en ; e ; : : : form an orthonormal basis for P X . For n = 1; 2; : : : let n x = hx ; ej iej . If T is a Fredholm integral 2

1

2

j =1

operator on X with a square-integrable kernel, then TnP and TnS are degenerate kernel approximations of T . (cf. (iv) of Subsection 4.2.1.)

4.17 Let T denote the Fredholm integral operator on C ([0; 1]) with kernel k(s; t) = exp(st), 0  s; t  1, and T N denote the Nystrom ap0

120

proximation of T based on the Gauss Two-Point Rule with 120 nodes. N is (4) = (:763796)10;4. (See The fourth largest eigenvalue of T120 4 2 e (120);4 . Then   (0:22)10;8 and Example 5.22.) Let  := 2270 p  2: Hence there is a nonzero eigenvalue  of T such j (4)j2 > (1 + 5) =   0:58. If (1) = 1:35303016 : : : is the that j (4) ; j  q j (4)j2 ;  eigenvalue of T obtained in Example 4.22, then  6= (1).

Chapter 5 Matrix Formulations

In Chapter 4 we saw how to approximate a bounded operator T on a complex Banach space X by a sequence (Tn ) of bounded nite rank operators on X as well as how to nd an approximate solution of the eigenvalue problem for T by solving the eigenvalue problem for Tn . In the present section we show that the eigenvalue problem for a bounded nite rank operator Te can be solved by reducing it to a matrix eigenvalue problem in a canonical way. For a bounded nite rank operator e ; x = y (where y 2 X is given Te, solutions of the operator equation Tx e = ' (where and x 2 X is to be found) or of the eigenvalue problem T' 0 6= ' 2 X and 0 6=  2 Cj are to be found) have long been obtained with the help of matrix computations. This is usually done in a variety of ways, depending on the speci c nature of the nite rank operator Te. A uni ed treatment for solutions of operator equations involving nite rank operators was given in [73]. It was extended to eigenvalue problems for nite rank operators in [34] and [53]. Our treatment here is along those lines. Although the operator TnK which appears in the singularity subtraction technique discussed in Subsection 4.2.3 is not of nite rank, the eigenvalue problem for it can still be reduced to matrix computations. We discuss a related question about nding a basis for a nite dimensional spectral subspace for Tn in such a way that, as n tends to in nity, each element of the basis is bounded and is bounded away from the span of the other elements. We call such bases uniformly well-conditioned. We also give matrix formulations for the iterative re nement schemes and for the acceleration procedure discussed in Chapter 3 when the approximate operator is of nite rank. We illustrate the implementation of these matrix formulations by giving some numerical examples. 247

248

5. MATRIX FORMULATIONS

5.1 Finite Rank Operators

A bounded nite rank operator is typically presented to us in the following form.

Proposition 5.1 A map Te : X ! X is a bounded nite rank operator if and only if there are xe ; : : : ; xen in X and fe ; : : : ; fen in X  such that 1

e = Tx

1

n X j =1

hx ; fej ixej = xe x; fe ; x 2 X;

where xe := [xe1 ; : : : ; xen ] 2 X 1n and fe := [fe1 ; : : : ; fen ] 2 (X )1n .

Proof

It is clear that if Te is de ned as above, then it is a bounded operator since fe1 ; : : : ; fen are continuous conjugate-linear functionals on X , and the rank of Te is nite since R(Te)  spanfxe1 ; : : : ; xen g. Conversely, let the rank of Te 2 BL(X ) be nite. Then there is a nite set fxe1 ; : : : ; xen g in X such that R(Te)  spanfxe1 ; : : : ; xen g. Renumbering xe1 ; : : : ; xen , if necessary, we may assume that the set fxe1 ; : : : ; xem g is linearly independent and R(Te)  spanfxe1 ; : : : ; xem g for some m  n. Then there are unique complex numbers c1 (x); : : : ; cm (x) such that e = c1 (x)xe1 +


+ cm (x)xem : Tx For j = 1; : : : ; m, de ne fej : X ! Cj by fej (x) = cj (x), x 2 X . It is easy to see that fe1 ; : : : ; fem are conjugate-linear functionals on X and e = Tx

m X j =1

hx ; fej ixej ; x 2 X:

To see that each fej is continuous, let j := dist(xej ; spanfxei : i = 1; : : : ; m; i 6= j g); j = 1; : : : ; m: Since xej does not belong to the closed subset spanfxei : i = 1; : : : ; m; i 6= j g, we have j > 0. Now for all x 2 X , we have e k  kTek kxk: jhx ; fej ijj  khx ; fe1 ixe1 +


+ hx ; fem ixem k = kTx

5.1. FINITE RANK OPERATORS

249

Thus jfej (x)j  (kTek=j )kxk for all x 2 X , so that fej is continuous for j = 1; : : : ; m. Let fej := 0 if m < j  n. Then fe1 ; : : : ; fen 2 X  and e = Tx

n X j =1

hx ; fej ixej ; x 2 X;

as desired. It may be noted that neither the elements xe1 ; : : : ; xen of X nor the elements fe1 ; : : : ; fen of X  in the representation of a bounded nite rank operator Te are required to be linearly independent and that the choice of these elements is not unique. One can easily see how each of the bounded nite rank operators TnP , TnS , TnG, TnD , TnN and TnF considered in Chapter 4 can be represented in this manner. Let Te be a bounded nite rank operator on X as given in the statement e is determined by xe = of Proposition 5.1. Then for each x 2 X , Tx 1n e e e [xe1 ; : : : ; xen ] 2 X and f = [f1 ; : : : ; fn ] 2 (X )1n . We show that Te can be written as a composition of two operators, one of them being determined by xe and the other by fe. j j = 1; : : : ; ng Recall that Cj n1 := fu := [u(1); : : : ; u(n)]> : u(j ) 2 C; denotes the linear space of all n1 matrices with complex entries. De ne Ke : X ! Cj n1 by e := [hx ; fe1 i; : : : ; hx ; fen i]> = x; fe ; x 2 X; Kx

and Le : Cj n1 ! X by

Leu :=

n X j =1

u(j )xej = xe u; u := [u(1); : : : ; u(n)]> 2 Cj n1 :

Clearly, Ke and Le are linear maps and e Te = LeK: We de ne an operator Ae : Cj n1 ! Cj n1 by e Ae := Ke L: For all u 2 Cj n1 , we have Aeu = Ke (Le u) = Leu; fe = xe u; fe = xe ; fe u:

250

5. MATRIX FORMULATIONS

Hence the nn Gram matrix Ae := xe ; fe represents the operator Ae with respect to the standard basis for Cj n1 . Also, if the set of elements in xe are linearly independent in X and span a subspace Xn , then the matrix Ae represents the operator TejX ;X with respect to the ordered basis xe of Xn . We shall now prove a crucial result which will allow us to nd bases for eigenspaces and for spectral subspaces of Te. n

n

Lemma 5.2

Let p be a positive integer, Z 2 Cj pp such that 0 2= sp(Z), y 2 X 1p , and v := Ke y 2 Cj np . Then for x 2 X 1p and u 2 Cj np , the following holds: Te x = x Z + y and Ke x = u if and only if Ae u = u Z + v and Le u = x Z + y . In particular, let y := 0 (so that v = 0 also); and let x , u satisfy the above conditions. Then the set of p elements in x is linearly independent in X if and only if the set of p vectors in u is linearly independent in Cj n1 .

Proof

Assume that Te x = x Z + y and Ke x = u . Then Le u = Le Ke x = Te x = x Z + y . Also, Ae u = Ae Ke x = Ke Te x = Ke ( x Z + y ) = uZ+ v. Conversely, assume that Ae u = u Z + v and Le u = x Z + y . Then Ke x Z = Ke ( Le u ; y ) = Ae u ; Ke y = Ae u ; v = u Z. Since Z is nonsingular, it follows that Ke x = u . Also, Te x = Le Ke x = Le u = xZ + y. Now consider y := 0 , so that v = Ke y = 0 , and Ke x = u , e L u = x Z. Let the set of p vectors in u be linearly independent in Cj n1 , and c 2 p Cj 1 be such that x c = 0. Then u c = ( Ke x )c = Ke ( x c) = Ke (0) = 0. The linear independence of the set of p vectors in u implies that c = 0, as desired. Let the set of p elements in x be linearly independent in X , and c 2 Cj p1 be such that u c = 0. Then x (Zc) = ( x Z)c = ( Le u )c = Le( u c) = Le(0) = 0. The linear independence of the set of p elements in x implies that Zc = 0 and since Z is nonsingular, c = 0, as desired.

251

5.1. FINITE RANK OPERATORS

Proposition 5.3 sp(Te) n f0g = sp(Ae) n f0g: e ) e In particular, sp(Te) is a nite set. Let e  sp(Te) n f0g, Pe := P (T; e e e and P := P (A; ). Then e Ke Pe = ePK: Also, Ke maps R(Pe) into R(eP) in a one-to-one manner, and hence e is a spectral set of nite type for Te.

Proof

We show that re(Te) n f0g = re(Ae) n f0g. e , u := R(A; e z )v Let 0 6= z 2 re(Ae). Consider y 2 X and de ne v := Ky and x := (Le u ; y)=z . Then Aeu = z u + v and Leu = zx + y. Letting p := 1 e ; zx = y . Thus R(Te ; zI ) = X . Next, let in Lemma 5.2, we see that Tx e e . Letting p := 1 x 2 X be such that (T ; zI )x = 0 and de ne u := Kx and y := 0 in Lemma 5.2, we see that Aeu = z u and Leu = zx. Since z 2 re(Ae), we have u = 0 and so zx = Le(0) = 0. As z 6= 0, we obtain x = 0. Hence N (Te ; zI ) = f0g. Thus z 2 re(Te). Conversely, let 0 6= z 2 re(Te). To show that z 2 re(Ae), it is enough to show that N (Ae ; zI ) = f0g. Let u 2 Cj n1 be such that (Ae ; zI )u = 0 and de ne x := Leu=z . Letting p := 1 and y := 0 in Lemma 5.2, we see e = zx and Kx e = u. Since z 2 re(Te), we have x = 0 and so that Tx e u = K (0) = 0. Hence N (Ae ; zI ) = f0g, as desired. Since sp(Ae) has at most n elements, sp(Te) has at most n +1 elements. Let e  sp(Te) n f0g. Then e is a spectral set for Te as well as for Ae. e ) e such that 0 2 ext(C). Then C 2 C (A; e ) e as well. Consider C 2 C (T; Let z 2 re(Te) n f0g. Since Ke Te = Ke LeKe = AeKe , we have e z ) = R(A; e z ) K: e Ke R(T;

Integrating the identity given above over C, we obtain Z Z   e z ) dz = ; 1 e z ) dz Ke Pe = Ke ; 21 i R(T; Ke R(T; 2  i C C Z Z   e z) K e dz = ; 1 e z ) dz K e R ( A; = ; 21 i R(A; 2i e = ePK:

C

C

252

5. MATRIX FORMULATIONS

The equation Ke Pe = ePKe shows that Ke maps R(Pe) into R(eP). Now let e = 0. Then Tx e =L e Kx e =L e (0) = 0. But since x 2 R(Pe) such that Kx e e e e sp(TjR(Pe);R(Pe) ) = , and 0 2= , the operator TjR(Pe);R(Pe) is invertible. e = 0, we obtain x = 0. Hence the map K e As x 2 R(Pe) and Tx jR(Pe) is one-to-one, and rank Pe  rank eP  n. Thus e is a spectral set of nite type for Te.

Theorem 5.4 (a) Let e 2 Cj n f0g. Then e is an eigenvalue of the operator Te if and only if e is an eigenvalue of the matrix Ae . In this case, let eu 2 Cj ng form an ordered basis for the eigenspace of Ae corresponding to e. Then 'e := Le eu =e forms an ordered basis for the eigenspace of Te corresponding to e and satis es Ke 'e = eu . In particular, the geometric multiplicity of e as an eigenvalue of Te is the same as its geometric multiplicity of e as an eigenvalue of Ae .

(b) Let e  sp(Ae ) n f0g and consider an ordered basis eu 2 Cj nm for

e . Then there is a nonsingular matrix the spectral subspace M (Ae ; ) m  m e 2 Cj such that Ae eu = eu e . Further, 'e := ( Le eu )e ;1 forms e ) e and satis es an ordered basis for the spectral subspace M (T; Ke 'e = eu . In particular, if e 2 sp(Ae ) n f0g, then the algebraic multiplicity of e as an eigenvalue of Te is the same as the algebraic multiplicity of e as an eigenvalue of Ae .

Proof

Recall that Ae represents the operator Ae with respect to the standard basis for Cj n1 . (a) Letting p := 1, Z := [e] and y := 0 in Lemma 5.2, we see that e is an eigenvalue of Te if and only if e is an eigenvalue of Ae. In this case, letting p := g and Z := eIg in Lemma 5.2, we note that if 'e := Le eu =e, then Te 'e = e 'e , Ke 'e = eu and the set f'e1 ; : : : ; 'eg g of elements in 'e is a linearly independent subset of the eigenspace of Te corresponding to e. That f'e1 ; : : : ; 'eg g also spans this eigenspace can be seen as follows.

5.1. FINITE RANK OPERATORS

253

Let 'eg+1 2= spanf'e1 ; : : : ; 'eg g be such that Te'eg+1 = e'eg+1 . If e := ['e1 ; : : : ; 'eg+1 ] 2 X 1(g+1) , then Te e = e e ; and again by Lemma 5.2, fKe 'e1 ; : : : ; Ke 'eg+1 g would be a linearly independent subset of the eigenspace of Ae corresponding to e, contrary to our assumption that this eigenspace is of dimension g. e Ae eu = eu  e (b) Since eu 2 Cj nm forms an ordered basis for M (A;e ), m  m e = . e As 0 2= , e  e is invertible. for some e 2 Cj such that sp() e Letting p := m, Z :=  and y := 0 in Lemma 5.2, we see that Te 'e = e K e 'e = eu and the set of m elements in 'e is linearly independent 'e , in X . Consider the closed subspace Ye of X spanned by the elements in 'e . Since the matrix e represents the operator TejYe;Ye with respect to 'e , e = . e Hence Ye  M (T; e ) e by Proposition we have sp(TejYe;Ye ) = sp() e ). e But as we have seen in Proposition 1.28, so that m  dim M (T; e e e e ) e in a one-to-one manner, so that 5.3, K maps M (T; ) into M (A; e e e e e ). e dim M (T; )  dim M (A; )  m. Thus Ye = M (T; In particular, the case e := feg shows that if e 2 sp(Ae) n f0g, then the algebraic multiplicity of e as an eigenvalue of Te is the same as the algebraic multiplicity of e as an eigenvalue of Ae. We conclude that the spectral subspace problem for a nite rank operator Te 2 BL(X ) can be solved in the following manner. Obtain a representation of Te: e := xe x; fe ; x 2 X; Tx

where xe 2 X 1n and fe 2 (X  )1n . Form the nn Gram matrix Ae given by Ae (i; j ) := hxej ; fei i; 1  i; j  n: To nd a spectral set e for Te such that 0 2= e and to obtain a basis for e ), e we may look for a spectral set the associated spectral subspace M (T; e for the matrix Ae such that 0 2= e and nd a basis eu for the spectral e eu = eu , e ). e Then A e where the nonsingular matrix  e is subspace M (A; given by e = ( eu  eu );1 eu  Ae eu : Let 'e := ( Le eu )e ;1 = xe eu e ;1:

254

5. MATRIX FORMULATIONS

e ). e Moreover, the Gram prodThen 'e forms an ordered basis for M (T; e e e uct 'e ; f of 'e with f is equal to K 'e = eu . In the special case e = feg, where e is a nonzero eigenvalue of Te of geometric multiplicity g and we are interested only in nding a basis 'e for the corresponding eigenspace, we may nd a basis eu 2 Cj ng of the eigenspace of Ae corresponding to e, so that e = eIg ; and then let e 'e := Leu = xeeu :   Often Te is a member of a sequence (Tn ) of bounded nite rank operators on X given by

Tn x :=

rX (n) j =1

hx ; fn;j ixn;j = xn x; fn ; x 2 X;

where xn := [xn;1 ; : : : ; xn;r(n) ] 2 X 1r(n) and fn := [fn;1; : : : ; fn;r(n)] 2 (X )1r(n). Let Tn be one of the nite rank approximations of a bounded operator T given in Chapter 4. We give the expressions for xn;j and fn;i , i; j = 1; : : : ; r(n), appearing in the representation of Tn and also the entries hxn;j ; fn;i i, i; j = 1; : : : ; r(n) of the corresponding matrix An . If n is a bounded projection of rank r(n), then it is easy to see that

n x =

rX (n) j =1

hx ; en;j ien;j = en x; en  ; x 2 X;

where en := [en;1 ; : : : ; en;r(n)] 2 X 1r(n) and en  := [en;1; : : : ; en;r(n)] 2 (X )1r(n) satisfy en ; en  = Ir(n) . Further, for X := L2 ([a; b]) or X := C 0 ([a; b]) and x 2 X , let

TnD x := where

kn (s; t) :=

Zb

a

kn (s; t)x(t) dt; s 2 [a; b];

rX (n) j =1

xn;j (s)yn;j (t); s; t 2 [a; b];

for some xn;j and yn;j 2 X for j = 1; : : : ; r(n).

255

5.1. FINITE RANK OPERATORS

hx ; fei i hTx ; eei i hx ; eei i

xej TnP

eej

TnS

T eej

TnG

hT eej ; eei i hT eej ; eei i

r~ X

eej

`=1

r~ X `=1

hx ; ee` ihT ee` ; eei i

hT eej ; ee` iee` hx ; eei i Zb

TnD

xej

TnN

wej k(
; etj )

TnF

Ae (i; j )

a

x(t)yei (t) dt

x(eti )

hT eej ; eei i Zb

a

xej (t)yei (t) dt

wej k(eti ; etj )

r~ X

eej wej

r~ X `=1

`=1

we` k(eti ; et` )x(et` )

hT eej ; eei i

k(et` ; etj )ee` x(eti ) Table 5.1

Also, if T is a Fredholm integral operator on C 0 ([a; b]) with a continuous kernel k(
;
), we consider the approximations TnN and TnF based on a quadrature formula

Qn (x) :=

rX (n) j =1

wn;j x(tn;j ); x 2 C 0 ([a; b]):

In Table 5.1 we denote xn;j , fn;i , en;j , en;i , r(n), yn;i , wn;j , tn;j and An (i; j ) simply by xej , fei , eej , eei , r~, yei , wej , etj and Ae (i; j ), respectively.

256

5. MATRIX FORMULATIONS

Remark 5.5 Finite element approximation:

We make a few comments about the nite element approximation described in Subsection 4.1.4. In this case the operator T := A;1 B may not be explicitly known and hence it may not be possible to represent the projection approximation TnP := n T in the usual manner. However, as we have seen earlier, if Xn := R(n ), then nding 0 6= 'n 2 Xn and 0 6= n 2 Cj such that TnP 'n = 1 'n n

is equivalent to nding 0 6= 'n 2 Xn and 0 6= n 2 Cj such that

a('n ; y) = n b('n ; y) for all y 2 Xn: Let [xn;1 ; : : : ; xn;r(n) ] form an ordered basis for Xn . Then it is enough to consider y := xn;i , i = 1; : : : ; r(n), in the preceding equation. Also, since

'n :=

rX (n) j =1

cn;j xn;j

for some cn;j 2 Cj , j = 1; : : : ; r(n), our problem reduces to nding cn;j 2 Cj , not all zero, and 0 6= n 2 Cj such that (n)  rX

a

j =1



cn;j xn;j ; xn;i = n b

(n)  rX

j =1



cn;j xn;j ; xn;i ; i = 1; : : : ; r(n);

that is, to the generalized eigenvalue problem j An un = n Bn un ; 0 6= un 2 Cj r(n)1 ; 0 6= n 2 C;

where An (i; j ) := a(xn;j ; xn;i ); Bn (i; j ) := b(xn;j ; xn;i ) and un (i) := cn;i :

It can be easily proved that Bn is a Hermitian positive de nite matrix. Also, if the sesquilinear form a(
;
) is Hermitian, then An is a Hermitian matrix. Such a generalized eigenvalue problem can be reduced to an ordinary eigenvalue problem as explained after the proof of Theorem 6.3.

257

5.1. FINITE RANK OPERATORS

5.1.1 Singularity Subtraction

Consider the Kantorovich-Krylov approximation TnK of a compact integral operator T on X := C 0 ([0; 1]) with a weakly singular kernel, discussed in Subsection 4.2.3. As we have noted just before stating Theorem 4.19, TnK = Tn + Un ; n = 1; 2; : : : where n X Tn x := x(tn;j )xn;j ; x 2 X; j =1

for some xed xn;j 2 X , tn;j 2 [0; 1], and

Un x := xn; x; x 2 X; 0

cc for a xed xn; 2 X . We have proved in Proposition 4.18 that Tn ! T, n  T . Note that T is a nite and in Theorem 4.19 that Un ! O and TnK ! n rank operator, but TnK is not a nite rank operator. We shall now give a matrix formulation of the spectral subspace problem for the operator TnK along the lines of [8]. Let  be a spectral set for T such that 0 2= . Since the operator T is compact,  is of nite type. (See Remark 1.34.) Let C 2 C (T; ) such that 0 2 ext(C). If n := sp(TnK ) \ int(C), then by Theorem 2.12, n is a spectral set of nite type for TnK for all large n. Also, n  int(C) and (C) := minfjz j : z 2 Cg > 0. We are therefore interested in nding a spectral set of nite type for TnK which is bounded away from zero. In particular, since kxn; k1 = kUn k ! 0, we are not interested in spectral values of TnK which belong to xn; ([0; 1]), that is, the range of the function xn; . As we have seen in Example 1.17, 0

0

0

0

xn; ([0; 1]) = sp(Un ): 0

In view of these considerations, let e TeK := Te + U;

where Te is a bounded nite rank operator on X := C 0 ([0; 1]) given by e := Tx

n X j =1

x(etj )xej ; x 2 X;

258

5. MATRIX FORMULATIONS

for some xed xej 2 X , etj 2 [0; 1], and Ue is a bounded operator on X given by e := xe0 x; x 2 X; Ux for a xed xe0 2 X . We wish to nd spectral sets of nite type for TeK which do not intersect Ee := sp(Ue ) = fxe0 (t) : t 2 [0; 1]g: Let e := x; fe = [x(et1 ); : : : ; x(etn )]> 2 Cj n1 ; x 2 X; Kx Leu := xe u = u(1)xe1 +


+ u(n)xen 2 X; u 2 Cj n1 ; and e Ae := Ke L; as before. Then the nn matrix Ae given by Ae (i; j ) := xej (eti ); 1  i; j  n; represents the operator Ae : Cj n1 ! Cj n1 with respect to the standard basis of Cj n1 . De ne De : Cj n1 ! Cj n1 by De u := [xe0 (et1 )u(1); : : : ; xe0 (etn )u(n)]> ; u 2 Cj n1 ; and AeK : Cj n1 ! Cj n1 by

e AeK := Ae + D: It is easy to see that Ke Ue = De Ke . Note that e sp(De ) = fxe0 (etj ) : j = 1; : : : ; ng  E:

The matrix De := diag [xe0 (et1 ); : : : ; xe0 (etn )] 2 Cj nn represents the operator De , and the matrix Ae K := Ae + De represents the operator AeK with respect to the standard basis for Cj n1 . In order to relate the spectral subspace problem for the operator TeK to the spectral subspace problem for the matrix Ae K , we shall prove a generalization of Lemma 5.2.

Lemma 5.6

Let p be a positive integer, Z 2 Cj pp such that sp(Z) \ sp(De ) = ;,

5.1. FINITE RANK OPERATORS

259

y 2 X 1p and v := Ke y 2 Cj np . Then for x 2 X 1p and u 2 Cj np , the following holds: TeK x = x Z + y and Ke x = u if and only if AeK u = u Z + v and Le u = x Z ; Ue x + y . In particular, let y := 0 (so that v = 0 also), and let x , u satisfy the above conditions. Assume that sp(Z) \ Ee = ;. Then the set of p elements in x is linearly independent in X if and only if the set of p vectors in u is linearly independent in Cj n1 .

Proof

Assume that TeK x = x Z + y and Ke x = u . Then Le u = Le Ke x = Te x = TeK x ; Ue x = x Z ; Ue x + y . Also, AeK u = AeK Ke x = Ke TeK x = Ke ( x Z + y ) = u Z + v . Conversely, assume that AeK u = u Z + v and Le u = x Z ; Ue x + y . Then Ke x Z ; De ( Ke xe ) = Ke ( xe Z ; Ue xe ) = Ke ( Le u ; y ) = Ae u ; v = AeK u ; De u ; v = u Z ; De u. Since sp(Z) \ sp(De ) = ;, it follows from Proposition 1.50 that Ke x = u . Also, TeK x = Le Ke x + Ue x = Le u + Ue x = x Z + y . Now let y := 0 , so that v = Ke y = 0 , Le u = x Z ; Ue x . Let the set of p vectors in u be linearly independent in Cj n1 and c 2 e x )c = K e ( x c) = K e (0) = 0. jC p1 be such that x c = 0. Then u c = ( K The linear independence of the set of p vectors in u implies that c = 0, as desired. Conversely, assume that the set of p elements in x is linearly independent in X , and c 2 Cj p1 is such that u c = 0. By mathematical induction, we show that

xej0 x c = x Zj c for j = 0; 1; : : : Clearly, this equality holds for j = 0. Assume that it holds for some nonnegative integer j . Since Ke Ue = De Ke , we have u Zj c = ( Ke x )Zj c = Ke ( x Zj c) = Ke (xej0 x c) = De j Ke ( x c) = ( De j ( Ke x ))c = ( De j u )c = De j ( u c) = De j (0) = 0:

As x Z = TeK x = Le Ke x + Ue x = Le u + xe0 x , we see that

x Zj+1 c = ( Le u )Zj c + xe0 x Zj c = Le( u Zj c) + xej0+1 x c = Le(0) + xej0+1 x c = xej0+1 x c:

260

5. MATRIX FORMULATIONS

Thus the induction argument is complete. It now follows that p(xe0 ) x c = x p(Z)c for every polynomial p in one variable. Let p be the characteristic polynomial of Z. Then p(Z) = O by the Cayley-Hamilton Theorem (Remark 1.33), but p(xe0 )(t) 6= 0 for each t 2 [0; 1] since sp(Z) consists of the roots of p and sp(Z) \ xe0 ([0; 1]) = ;. Thus p(xe0 ) x c = 0. Dividing by p(xe0 ), we obtain x c = 0. The linear independence of the set of p elements in x implies that c = 0, as desired.

Proposition 5.7 e sp(TeK ) n Ee = sp(AeK ) n E: In particular, sp(TeK ) n Ee is a nite set. Let e  sp(TeK ) n Ee, C 2 e such that Ee  ext(C) (so that C 2 C (AeK ; ) e as well); Pe := C (TeK ; ) K K e , and eP := P (Ae ; ) e . Then P (Te ; )  1 Z  e e e e e e K P = PK and P = 2i (Ue ; zI );1 Le R(AeK ; z ) dz K: C

Also, Ke maps R(Pe) into R(eP) in a one-to-one manner, and hence e is a spectral set of nite type for TeK .

Proof

We show that re(TeK ) n Ee = re(AeK ) n Ee. e , u := Let z 2 re(AeK ) n Ee. Consider y 2 X and de ne v := Ky K K R(Ae ; z )v and x := (Le u ; y)=(z ; xe0 ). Then Ae u = z u + v and Leu = e + y . Letting p := 1 in Lemma 5.6, we see that TeK x ; zx = y . zx ; Ux Thus R(TeK ; zI ) = X . Next, let x 2 X be such that (TeK ; zI )x = 0 e . Letting p := 1 and y := 0 in Lemma 5.6, we see and de ne u := Kx K e e . Since z 2 re(AeK ), we have u = 0 that A u = z u and Leu = zx ; Ux e e and so (z ; xe0 )x = zx ; Ux = L(0) = 0. As z 2= Ee , we obtain x = 0. Hence N (TeK ; zI ) = f0g. Thus z 2 re(TeK ). Conversely, let z 2 re(TeK ). To show that z 2 re(AeK ), it is enough to show that N (AeK ;zI ) = f0g. Let u 2 Cj n1 be such that (AeK ;zI )u = 0 and de ne x := Leu=(z ; xe0 ). Letting p := 1 and y := 0 in Lemma 5.6, e = u. Since z 2 re(TeK ), we have x = 0 we see that TeK x = zx and Kx e and so u = K (0) = 0. Hence N (AeK ; zI ) = f0g, as desired.

261

5.1. FINITE RANK OPERATORS

Since sp(AeK ) has at most n elements, so does sp(TeK ) n Ee. Let e  sp(TeK ) n Ee . Then e is a spectral set for TeK as well as for AeK . Letting e such E := Ee in Proposition 1.29, it follows that there is C 2 C (TeK ; ) K e that Ee  ext(C) and then C 2 C (Ae ; ). K K Let z 2 re(Te ) n Ee. Since Ke Te = Ke (Te + Ue ) = (Ae + De )Ke = AeK Ke , we have e Ke R(TeK ; z ) = R(AeK ; z ) K: Also, the proof of R(TeK ; zI ) = X given before shows that for every y 2 X, e eK e R(TeK ; z )y = L R(Az ;; zx)eKy ; y = (Ue ; zI );1[I ; Le R(AeK ; z ) Ke ]y: 0

Hence

R(TeK ; z ) = (Ue ; zI );1 [I ; Le R(AeK ; z ) Ke ]:

Integrating the two identities given above over C, we obtain Z Z   1 1 K e e e e K P = K ; 2i R(T ; z ) dz = ; 2i Ke R(TeK ; z ) dz C Z ZC   1 1 K = ; 2i R(Ae ; z ) Ke dz = ; 2i R(AeK ; z ) dz Ke e = ePK:

C

C

Since sp(Ue ) = Ee  ext(C), we have Z

Z

Z

C

(Ue ; zI );1 dz = O, so that

Pe = ; 21 i R(TeK ; z ) dz = ; 21 i (Ue ; zI );1 [I ; Le R(AeK ; z ) Ke ] dz C C   1 Z ; 1e K e e e (U ; zI ) L R(A ; z ) dz K: =

2i

C

The equation Ke Pe = ePKe shows that Ke maps R(Pe) into R(eP). Now let e = 0. Then x 2 R(Pe) such that Kx Z   e = 1 e ; zI );1 L e R(AeK ; z ) dz Kx e = 0: x = Px ( U 2i C Hence the map Ke jR(Pe) is one-to-one, and rank Pe  rank eP  n. Thus e is a spectral set of nite type for TeK .

262

5. MATRIX FORMULATIONS

Theorem 5.8 (a) Let e 2 Cj n Ee. Then e is an eigenvalue of the operator TeK if and

only if e is an eigenvalue of the matrix Ae K . In this case, let eu 2 Cj ng form an ordered basis for the eigenspace of Ae K corresponding to e. Then 'e := Le eu =(e ; xe0 ) forms an ordered basis for the eigenspace of TeK corresponding to e and satis es Ke 'e = eu . In particular, the geometric multiplicity of e as an eigenvalue of TeK is the same as its geometric multiplicity of e as an eigenvalue of Ae K . (b) Let e  sp(AeK ) n Ee and consider an ordered basis eu 2 Cj nm e . Then there is a matrix  e 2 of the spectral subspace M (Ae K ; ) e K eu = eu  e and sp() e \ Ee = ;. Further, 'e := jC mm such that A ( Le eu )(e ; xe0Im );1 2 X 1m forms an ordered basis for the spectral e and satis es K e 'e = eu . subspace M (TeK ; ) In particular, if e 2 sp(Ae K ) n Ee , then the algebraic multiplicity of e as an eigenvalue of TeK is the same as the algebraic multiplicity of e as an eigenvalue of Ae K .

Proof

Recall that Ae K represents the operator AeK with respect to the standard basis for Cj n1 . (a) Letting p := 1, Z := [e] and y := 0 in Lemma 5.6, we see that e is an eigenvalue of TeK if and only if e is an eigenvalue of AeK . In this case, letting p := g and Z := eIg in Lemma 5.6, we note that if 'e := Le eu =(e ; xe0 ), then TeK 'e = e 'e , Ke 'e = eu ; and the set f'e1 ; : : : ; 'eg g of elements in 'e is a linearly independent subset of the eigenspace of TeK corresponding to e. That f'e1 ; : : : ; 'eg g also spans this eigenspace can be seen as follows. Let 'eg+1 2= spanf'e1 ; : : : ; 'eg g be such that TeK 'eg+1 = e'eg+1 . If e := ['e1 ; : : : ; 'eg+1 ] 2 X 1(g+1) , then TeK e = e e ; and again by Lemma 5.6, fKe 'e1 ; : : : ; Ke 'eg+1 g would be a linearly independent subset of the eigenspace of AeK corresponding to e, contrary to our assumption that this eigenspace is of dimension g. (b) Since eu 2 Cj nm forms an ordered basis for the spectral subspace e there exists a matrix  e 2 Cj mm such that AeK eu = eu  e M (AeK ; ),

5.1. FINITE RANK OPERATORS

263

e = . e As sp() e \ sp(Ue ) =  e \ Ee = ;, Proposition 1.50 shows and sp() 1m such that 'e e ; xe0 'e = Le eu . In fact, that there is a unique 'e 2 X ; 1 'e = ( Le eu )(e ; xe0 Im ) . Letting p := m, Z := e and y := 0 in Lemma e K e 'e = eu and that the set of m elements 5.6, we see that TeK 'e = 'e , in 'e is linearly independent in X . Consider the closed subspace Ye of X spanned by the elements in 'e . Since the matrix e represents the opere = . e Hence ator TejKYe;Ye with respect to 'e , we have sp(TejKYe;Ye ) = sp() e by Proposition 1.28, so that m  dim M (TeK ; ). e But Ye  M (TeK ; ) K e e e e as we have seen in Proposition 5.7, K maps M (T ; ) into M (AeK ; ) K K e e e e in a one-to-one manner, so that dim M (T ; )  dim M (A ; )  m. e Thus Ye = M (TeK ; ). In particular, the case e := feg shows that if e 2 sp(AeK ) n Ee, then the algebraic multiplicity of e as an eigenvalue of TeK is the same as the algebraic multiplicity of e as an eigenvalue of AeK .

We conclude that to nd a spectral set e for TeK such that e \ xe0 ([0; 1]) = ; and to obtain a basis for the associated spectral subeK e we may look for a spectral set  e for the matrix A space M (TeK ; ), n  m such that e \ xe0 ([0; 1]) = ; and nd a basis eu 2 Cj for the specK K e e e e tral subspace M (A ; ). Then A eu = eu , where the mm matrix e e \ xe0 ([0; 1]) = ; and is given by satis es sp() e = ( eu  eu );1 eu  Ae K eu : Let

'e := ( Le eu )(e ; xe0 Im );1 = xe eu (e ; xe0 Im );1 :

e Moreover, the Gram Then 'e forms an ordered basis for M (TeK ; ). product 'e ; fe of 'e with fe is equal to Ke 'e = eu . In the special case e = feg, where e is a nonzero eigenvalue of TeK of geometric multiplicity g and we are interested only in nding a basis 'e for the corresponding eigenspace, we may nd a basis eu 2 Cj ng of the eigenspace of Ae K corresponding to e, so that e = eIg ; and then let

'e := e L u = e xe u :  ; xe0  ; xe0 e

264

5. MATRIX FORMULATIONS

5.1.2 Uniformly Well-Conditioned Bases

Let T 2 BL(X ) and  be a spectral set for T such that 0 62  and the corresponding spectral subspace M := M (T; ) is of dimension m < 1. Let (Tn ) be a sequence of bounded nite rank operators such that Tn ! T . In Proposition 2.9 and Theorem 2.18, we have seen that for each large n, the operator Tn has a spectral set n such that 0 62 n, the corresponding spectral subspace Mn := M (Tn; n ) is also of dimension m, n `approximates' , and Mn `approximates' M . For simplicity we assume that rank T  n, that is, we let r(n) = n. Let n X Tn x := hx ; fn;j ixn;j = xn x; fn ; x 2 X; j =1

where xn := [xn;1 ; : : : ; xn;n ] 2 X 1n and fn := [fn;1 ; : : : ; fn;n ] 2 (X )1n , and An := xn ; fn , that is, An (i; j ) := hxn;j ; fn;i i; 1  i; j  n: Then n is a spectral set for An , and the associated spectral subspace M (An ; n) is of dimension m. Let un := [un;1 ; : : : ; un;m ] form an ordered basis for M (An ; n ). Then An un = un n , where n 2 Cj mm and sp(n ) = n . As 0 2= sp(n ), the matrix n is nonsingular. Consider Ln : Cj n1 ! X de ned by

Ln u :=

n X j =1

u(j )xn;j = xn u; u := [u(1); : : : ; u(n)]> 2 Cj n1 :

Then ( Ln un );n 1 forms an ordered basis for the spectral subspace Mn := M (Tn; n ). In general, the rank of the operator Tn will increase as n increases. From the point of view of computations, the orthonormal bases in a Hilbert space are the best among well-conditioned bases. It is desirable to obtain bases with similar properties in the general framework of a Banach space. Speci cally, one would like to construct an ordered basis 'n := ['n;1 ; : : : ; 'n;m ] for Mn such that for some constants and  satisfying 0 <   , and for all large n and j = 1; : : : ; m, k'n;j k  ; dist('n;j ; spanf'n;i : i = 1; : : : ; m; i 6= j g)  : This requirement was also stated in Proposition 2.21 for the convergence to 0 of the sequences (k( Tn ; T ) 'n k1 ) and (k 'n ; P 'n k1 ).

265

5.1. FINITE RANK OPERATORS

Further, we have come across the same requirement in Theorems 3.7 and 3.8 for the convergence of the iterative re nement schemes for a cluster of eigenvalues. Also, see Theorem 3.15(a) for a similar requirement in the acceleration technique. We shall show that this can be accomplished if we choose a norm k
k n on Cj n1 such that the norms of the operator Ln : Cj n1 ! X given above and of the operator Kn : X ! Cj n1 given by [ ]

Kn x := [hx ; fn;1 i; : : : ; hx ; fn;ni]> = x; fn ; x 2 X; are bounded; and if we can nd an ordered basis [un;1 ; : : : ; un;m ] of M (An ; n) such that for some constants c and d satisfying 0 < d  c, and for all large n and j = 1; : : : ; m,

kun;j k n  c; dist(un;j ; spanfun;i : i = 1; : : : ; m; i 6= j g)  d: [ ]

Most of the discussion in this section has appeared in [13].

Theorem 5.9 Suppose that there is a norm k
k n on Cj n such that kKnk   and kLnk  for some constants , and all positive integers n. 1

[ ]

Let un := [un;1 ; : : : ; un;m] form an ordered basis for the spectral subspace M (An ; n ) such that for all large n and j = 1; : : : ; m,

kun;j k n  c; dist(un;j ; spanfun;i : i = 1; : : : ; m; i 6= j g)  d for some constants c and d such that 0 < d  c. De ne [ ]

'n := Ln un ;n 1 = ['n;1 ; : : : ; 'n;m ]; where An un = un n . Then k;n 1 k   for some constant  and all large n, and 'n forms an ordered basis for the spectral subspace M (Tn ; n ) such that for all large n and j = 1; : : : ; m, 1

k'n;j k  ; dist('n;j ; spanf'n;i : i = 1; : : : ; m; i 6= j g)  ; where := c  and  := d=.

Proof

It follows from Theorem 5.4(b) that for all large n, 'n forms an ordered basis for M (Tn ; n).

266

5. MATRIX FORMULATIONS

Let An := Kn Ln and note that kAn k = kKnLnk  kKnk kLnk   . The matrix An represents the operator An with respect to the standard basis for Cj n1 . First we nd an upper bound for knk . Let n;i;j denote the (i; j )th entry of the mm matrix n . For each j = 1; : : : ; m, we have An un;j = n;1;j un;1 +


+ n;m;j un;m; so that for all i = 1; : : : ; m, we obtain djn;i;j j  kAn un;j k  kAn k kun;j k n   c: Hence m X knk = j=1max jn;i;j j  m c ;:::;m d : 1

[ ]

1

i=1

Recall that we are given a spectral set  for T such that 0 62  and the dimension of the corresponding spectral subspace M (T; ) of T is m < 1. By Theorem 1.32  consists of a nite number of nonzero isolated spectral values of T . Let C be a Cauchy contour which separates  from sp(T ) n  as well as from 0. In particular, 0 2 ext(C). Since Tn ! T , Proposition 2.9 shows that C  re(Tn ) and n = sp(Tn ) \ int(C) for all large n. As sp(n ) = n , it follows that if  := dist(0; C), then n := minfjn j : n 2 sp(n )g   > 0: Now by Lemma 2.20(b), m=2

m=2  m c m;1

k;n k  mm kn km;   := mm n 1

1

1

d

1

:

Hence for all large n and j = 1; : : : ; m, k'n;j k  k 'n k1  k Ln un k1 k;n 1k  kLnk j=1max ku k k;1k ;:::;m n;j n n  := c: Since Kn 'n = un by Theorem 5.4(b), Kn'n;j = un;j for each j = 1; : : : ; m. Thus for all large n, all complex numbers c1 ; : : : ; cm and each j = 1; : : : ; m, 1

[ ]

1

m m

  X X

d  un;j ; ci un;i [n] = Kn 'n;j ; ci 'n;i

[n] i=1; i6=j i=1; i6=j m m



X X  kKn k

'n;j ; ci 'n;i

 

'n;j ; ci 'n;i

; i=1; i6=j i=1; i6=j

267

5.1. FINITE RANK OPERATORS

so that dist('n;j ; spanf'n;i : i = 1; : : : ; m; i 6= j g)   := d=. Letting m := 1 in the preceding theorem, we obtain the following result.

Corollary 5.10 Suppose that k
k n is a norm on Cj n such that kKnk   and kLnk  for some constants , and all positive integers n. Let  be a nonzero  T and  be the simple eigenvalue of T simple eigenvalue of T , Tn ! n n such that n !  and jn j   > 0. Let un be an eigenvector of An corresponding to n such that kun k n = 1 and let 'n := Lnun =n . Then 1

[ ]

[ ]

'n is an eigenvector of Tn such that

1  k' k  : n   The utility of the above-mentioned considerations depends on the possibility of 1. nding for each large n a norm k
k n on Cj n1 such that the sequences of operator norms (kKnk) and (kLnk) are bounded, and [ ]

2. nding an ordered basis un of the spectral subspace M (An ; n ) satisfying the conditions stated in the theorem. We now discuss these two points. First we give several examples where a suitable choice of a norm on Cj n1 makes the sequences (kKn k) and (kLn k) bounded.

Example 5.11 2-norm on Cj n : 1

Let X := L2 ([a; b]) with

kxk :=

Z b

2

a

jx(t)j dt 2

1=2

; x 2 X;

and T 2 BL(X ). Consider an orthonormal basis (ej ) for X and let n denote the orthogonal projection

n x :=

n X j =1

hx ; ej iej ; x 2 X:

268

5. MATRIX FORMULATIONS

Upper bound for kKn k

Upper bound for kLnk

TnP

kT k

1

TnS

1

TnG

1

kT k kT k Table 5.2

Choose the norm k
k n on Cj n1 given by

kuk n := [ ];2

n X j =1

[ ];2

ju(j )j

2

1=2

; u := [u(1); : : : ; u(n)]> 2 Cj n1 :

Then we have Table 5.2. Note that these upper bounds depend only on T .

Example 5.12 1-norm on Cj n : Let X := C ([a; b]) with the sup norm and T be a Fredholm integral operator with a continuous kernel k(
;
). Consider a  tn; <


< tn;n  b 1

0

1

and en;1; : : : ; en;n in X such that en;j (tn;k ) = j;k , j; k = 1; : : : ; n. Let n denote the interpolatory projection and Qn the quadrature formula given by

n x :=

n X j =1

x(tn;j )en;j ;

Choose the norm k
k n

[ ];

and Qn (x) := 1

n X j =1

wn;j x(tn;j ); x 2 X:

on Cj n1 given by

kuk n 1 := supfju(j )j : j = 1; : : : ; ng; u := [u(1); : : : ; u(n)]> 2 Cj n : [ ];

1

Then we have Table 5.3. p Note that if n ! I , then the sequence (kn k) is bounded; and if the sequence (Qn ) of quadrature formul is convergent, then the sequence n X  jwn;j j is bounded. j =1

269

5.1. FINITE RANK OPERATORS

Upper bound for kKnk

Upper bound for kLn k

TnP

kT k

TnS

1

TnG

1

kn k kn k(b ; a)kk(
;
)k1 kn k(b ; a)kk(
;
)k1

TnN

1

kk(
;
)k1

TnF

1

kn k kk(
;
)k1

n X j =1

jwn;j j

n X j =1

jwn;j j

Table 5.3 The preceding two examples may explain why the norm k
k n on

Cj n1 is usually employed while discretizing the in nite dimensional space L2 ([a; b]) and why the norm k
k n 1 on Cj n1 is usually employed while discretizing the in nite dimensional space C 0 ([a; b]). However, the [ ];2

[ ];

following example suggests that this should not be followed as a rule.

Example 5.13 1-norm on Cj n : Let X := C ([a; b]), a = tn;  tn; <


< tn;n  tn;n = b, and en; ; : : : ; en;n be the corresponding hat functions. For an integral operator T with a continuous kernel k(
;
), consider the degenerate kernel 1

0

0

1

+1

1

kn (s; t) :=

n X j =1

k(s; tn;j )en;j (t); s; t 2 [a; b];

obtained by interpolating k(
;
) in the second variable. Then for the degenerate kernel approximation TnD of T , we have

Kn x =

hZ b

a

x(t)en;1 (t) dt; : : : ;

Zb

a

i>

x(t)en;n (t) dt ; x 2 X;

270

5. MATRIX FORMULATIONS

Ln u = u(1)k(
; tn;1 ) +


+ u(n)k(
; tn;n ); u 2 Cj n1 : Let us choose the norm k
k n 1 on Cj n1 . Then [ ];

nZ b

o en;j (t) dt : j = 1; : : : ; n  3h2n ; a where hn := maxftn;j ; tn;j;1 : j = 1; : : : ; n + 1g. If hn ! 0, then kKnk ! 0 and, in particular, the sequence (kKn k) is bounded. However,

kKnk  max

since we only have

kLnk  k jk(
; tn; )j +


+ jk(
; tn;m )j k1 ; the sequence (kLnk) may not be bounded. For example, if a := 0, b := 1, k(s; t) := est for 0  s; t  1 and tn;j := (j ; 1)=n, j = 1; : : : ; n, then for u := [1; : : : ; 1]> 2 Cj n , we have n X kLnuk1 = e j; =n = e e=n;;1 1 j 1

1

(

=1

1)

1

and hence kLnk ! 1. On the other hand, if we choose the norm k
k n on Cj n1 given by [ ];1

kuk n := ju(1)j +


+ ju(n)j; u := [u(1); : : : ; u(n)]> 2 Cj n ; 1

[ ];1

then it is easy to see that

kKn k  b ; a and kLnk  kk(
;
)k1 : Thus in this case, to obtain uniformly well-conditioned bases for the approximate spectral subspaces, we choose the norm k
k n on Cj n1 rather than the norm k
k n 1 on Cj n1 while discretizing the in nite dimensional space C 0 ([a; b]). [ ];1

[ ];

Remark 5.14

Given a norm k
k n on Cj n1 , we discuss the question of nding an ordered basis un := [un;1 ; : : : ; un;m ] for the spectral subspace M (An ; n ) such that for all large n and j = 1; : : : ; m, [ ]

kun;j k n  c; dist(un;j ; spanfun;i : i = 1; : : : ; m; i 6= j g  d; where c and d are constants satisfying 0 < d  c. [ ]

271

5.1. FINITE RANK OPERATORS

First we consider the most simple case: Let k
k n be the 2-norm k
k n on Cj n1 . Given any ordered basis for M (An ; n), the GramSchmidt Process described in Proposition 1.39 yields an ordered basis un := [un;1 ; : : : ; un;m ] of M (An ; n) such that, for j = 1; : : : ; m, [ ]

[ ];2

kun;j k n = 1 = dist(un;j ; spanfun;i : i = 1; : : : ; m; i 6= j g: [ ];2

Thus in this case, we can take c = d = 1. In the general case, the following result can be used.

Proposition 5.15

Let n and m be positive integers, m  n, k
k n be a norm on Cj n1 and let v := [v1 ; : : : ; vm ] form an ordered basis for a subspace S of Cj n1 . Then an ordered basis u := [u1 : : : ; um ] for S may be constructed such that for each j = 1; : : : ; m, spanfu1 ; : : : ; uj g = spanfv1 ; : : : ; vj g, kuj k n = 1 and dist(uj ; spanfui : i = 1; : : : ; m; i 6= j g)  2m1;1 . [ ]

[ ]

Proof

De ne u1 := v1 =kv1 k n . If j = 2; : : : ; m and we have found u1 ; : : : ; uj;1 such that spanfu1 ; : : : ; uj;1 g = spanfv1 ; : : : ; vj;1 g and ku1 k n =


= kuj;1 k n = 1, consider the matrix [ ]

[ ]

[ ]

Cj := [u1 ; : : : ; uj;1 ] 2 Cj n(j;1)

and nd wj 2 Cj (j;1)1 such that n

kvj ; Cj wj k n = min kvj ; Cj wk n : w 2 Cj j; [ ]

n

(



1) 1

[ ]

o

Since Cj w : w 2 Cj (j;1)1 = spanfu1 ; : : : ; uj;1 g, we have

kvj ; Cj wj k n = dist(vj ; spanfu ; : : : ; uj; g): [ ]

Let Then it follows that

uj :=

1

vj ; Cj wj kvj ; Cj wj k n : [ ]

1

o

:

272

5. MATRIX FORMULATIONS

spanfu1; : : : ; uj g = spanfv1 ; : : : ; vj g,

kuj k n = 1 and dist(uj ; spanfu : : : ; uj; g) = 1. 1

[ ]

1

We prove that for each j = m; m ; 1; : : : ; 1, dist(uj ; spanfui : i = 1; : : : ; m; i 6= j g)  2m1;1 : By assumption, we have dist(um ; spanfui : i = 1; : : : ; m ; 1g)  1: Hence the result holds for j = m. Next, x j such that m ; 1  j  1. For given complex numbers ci , i = 1; : : : ; m, i 6= j , let

0 :=

uj ;

X

i6=j



ci ui

[n] and 1 :=

uj ;

X

i6=j;m

ci ui

[n] :

If jcm j  1 =2, then

0  1 ; jcm j kumk n = 1 ; jcm j  21 ; [ ]

and if jcm j > 1 =2, then

0 = jcm j

um +

ci u ; 1 u

 jc j > 1 : i c j [n] m 2 m i6=j;m cm X

Thus in any case 0  1 =2. Similarly, if we let

2 :=

uj ;

X

i6=j;m;m;1

ci ui

[n] ;

we obtain 1  2 =2 and hence 0  2 =22. Repeating this argument m ; j times, we see that 0  2m1;j kuj ; c1 u1 ;


; cj;1 uj;1 k n  2m1;j  2m1;1 : Hence for each xed j = m; m ; 1; : : : ; 1, we have dist(uj ; spanfui : i = 1; : : : ; m; i 6= j g)  2m1;1 ; [ ]

273

5.1. FINITE RANK OPERATORS

as desired.

Remark 5.16

The proof of Proposition 5.15 involves a construction of wj 2 Cj (j;1)1 such that n

kvj ; Cj wj k n = min kvj ; Cj wk n : w 2 Cj j; [ ]

(

[ ]



1) 1

o

;

where vj 2 Cj n1 and Cj := [u1 ; : : : ; uj;1 ] 2 Cj n(j;1) are given, j = 2; : : : ; m. This construction is in fact an implementation of the wellknown Riesz Lemma (Lemma 5.3 of [55] or Lemma 2.5-4 of [48]) for the nite dimensional normed space Cj n1 with the norm k
k n . It is equivalent to the computation of a minimum norm solution of the over-determined system Cj w = vj ; of n equations in j ; 1 unknowns, where j = 2; : : : ; m and m  n. The reader is referred to [72] for various algorithms which yield such a solution if the norm k
k n is the p-norm k
k n p and the scalars are real numbers. (See Chapter 1 if p = 1, Chapter 4 if 1 < p < 1, and Chapter 6 if p = 1.) If the norm k
k n is the p-norm k
k n p and the scalars are complex numbers (as in our case), we may utilize the minimum norm solution for the real scalars to obtain a result similar to Proposition 5.15. We proceed as follows. Suppose we are given a matrix C 2 Cj nk and v 2 Cj n1 . We write C := C1 +i C2 and v := v1 +i v2 , where C1 ; C2 2 IRnk and v1 , v2 2 IRn1 . For any w 2 Cj k1 , write w := w1 + i w2 with w1 ; w2 2 IRk1 . Then [ ]

[ ]

[ ];

[ ]

[ ];

v ; Cw = (v1 + i v2 ) ; (C1 + i C2 )(w1 + i w2 ) = (v1 ; C1 w1 + C2 w2 ) + i (v2 ; C2 w1 ; C1 w2):

Let 1  p  1. By one of the algorithms cited above, nd we 1 ; we 2 2 IRk1 such that

     

v1 C we 1 1 ;C2

v2 ; C2 C1 w e 2 [2n]; p

     

v1 C 1 ;C2

= min 1 v ; C C ww1

: 2 2 1 2 w1 ;w2 2IR [2n]; p k

274

5. MATRIX FORMULATIONS

Note that for z 2 Cj , we have the following inequalities: If 1  p  2, then (j
kv ; Cwe k n p  2jp; j= p min kv ; Cwk n p j  [ ];

2

2

w2 C k

[ ];

1

for any p; 1  p  1. This procedure allows us to nd u := [u1 ; : : : ; um ] such that spanfu1; : : : ; uj g = spanfv1 ; : : : ; vj g and kuj k n = 1 for j = 1; : : : ; m, dist(uj ; spanfu1 ; : : : ; uj;1 g)  jp;12j=2p for j = 2; : : : ; m. 2 The method given in the proof of Proposition 5.15 then yields a positive constant d (independent of n but possibly depending on m) such that [ ]

dist(uj ; spanfui : i = 1; : : : ; m; i 6= j g)  d m

for all j = 1; : : : ; m. In fact, we may take d = (1 + dd0 )m+1 , where 0 d0 = 2jp;12j=2p . (See Exercise 5.8.) To conclude, we observe that starting with any ordered basis vn of the spectral subspace M (An ; n ), we can construct an ordered basis un := [un;1 ; : : : ; un;m ] for it such that for each j = 1; : : : ; m,

kun;j k n p = 1 and dist(uj ; spanfui : i = 1; : : : ; m; i 6= j g)  d; [ ];

for some constant d independent of n, where k
k n p denotes the pnorm on Cj n1 , 1  p  1. These uniformly well-conditioned bases for M (An ; n ) allow us to nd uniformly well-conditioned bases 'n for M (Tn ; n ) as shown in Theorem 5.9. [ ];

275

5.2. ITERATIVE REFINEMENT

5.2 Iterative Re nement

In this section we consider the implementation of the iterative re nement schemes discussed in Section 3.1 when the approximation Tn of T is of nite rank. For convenience of notation, assume that rank Tn  n, where n is a xed relatively small positive integer. Let Tn be given by

Tn x :=

n X j =1

hx ; fn;j ixn;j = xn x; fn ; x 2 X;

where xn := [xn;1 ; : : : ; xn;n ] 2 X 1n and fn := [fn;1 ; : : : ; fn;n] 2 (X )1n ; as in Proposition 5.1. De ne Kn : X ! Cj n1 by Kn x := [hx ; fn;1 i; : : : ; hx ; fn;ni]> = x; fn ; x 2 X; and Ln : Cj n1 ! X by

Lnu :=

n X j =1

u(j )xn;j = xn u; u = [u(1); : : : ; u(n)]> 2 Cj n1 :

Then, as we have seen in Section 5.1, Tn = Ln Kn ; and if we let An := Kn Ln ; then the nn matrix An de ned by An (i; j ) := hxn;j ; fn;i i represents the operator An with respect to the standard basis for Cj n1 . Since for all x 2 X and f 2 X , hTn x ; f i = h xn x; fn ; f i = xn ; f x; fn = hx ; fn xn ; f i;

we conclude that Tn : X  ! X  is given by

n X Tnf := fn xn ; f  = hxn;j ; f ifn;j :

Similarly, Ln : X  ! Cj n1 is given by

j =1

Ln f = xn ; f ; f 2 X ;

and Kn : Cj n1 ! X  is given by Knv = fn v; v 2 Cj n1 :

276

5. MATRIX FORMULATIONS

Hence Tn = KnLn , An = Ln Kn, and the matrix An represents the operator An with respect to the standard basis for Cj n1 : Let us rst take up the case of a nonzero simple eigenvalue  of T and see how to implement the Elementary Iteration considered in Subsection 3.1.2: 8 0 ' := 'n ; and for k = 1; 2; : : : > > > n ( )

> <

(nk) := hT'(nk;1) ; 'n i; > > > > : '(k) := '(k;1) + S ((k) '(k;1) ; T'(k;1) ): n n n n n n

(E )

As we have mentioned while deriving the relation (E )3 of Subsection 3.1.2,

'nk = 'n + Sn [(nk ; n )'nk;1 + (Tn ; T )'nk;1 ]: ( )

( )

(

)

(

)

This way of writing 'nk avoids the computation of the residual nk 'nk;1 ;T'nk;1 . Firstly, we need to nd an eigenvector 'n of Tn corresponding to a nonzero simple eigenvalue n . For this purpose, we may nd an eigenvector un of the matrix An corresponding a nonzero simple eigenvalue n and de ne 'n := 1 Ln un = 1 xn un : ( )

(

( )

(

)

)

n n Then Tn'n = n 'n and Kn 'n = un as proved in Theorem 5.4(a) with g := 1. In particular, 'n 6= 0 as un 6= 0. Secondly, we need to nd the eigenvector 'n of Tn corresponding to the simple eigenvalue n such that h'n ; 'n i = 1. Now since n is a simple eigenvalue of An , there is a unique eigenvector vn of An corresponding to n such that vn un = hun ; vn i = 1. (See Theorem

1.52.) If we de ne

'n := Knvn = vn (1)fn;1 +


+ vn (n)fn;n ;

then Tn'n = KnLn Kn vn = KnAn vn = n Knvn = n 'n , and

n h'n ; 'n i = hLn un ; Knvn i = hKn Lnun ; vn i = hAn un ; vn i = hn un ; vn i = n vn un = n ;

so that h'n ; 'n i = 1, as desired.

277

5.2. ITERATIVE REFINEMENT

Thirdly, to nd 'nk for k = 1; 2; : : : we observe that ( )

hnk 'nk; ; T'nk; ; 'n i = nk h'nk; ; 'n i ; hT'nk; ; 'n i = nk ; nk = 0; that is, Pn (nk 'nk; ; T'nk; ) = 0. Hence we can nd 'nk if we are able to nd Sn y whenever y 2 X and Pn y = 0. Let y 2 X be such that Pn y := hy ; 'n i'n = 0. Then vn Kn y = hKn y ; vn i = hy ; Kn vn i = hy ; 'n i = 0; ( )

(

1)

(

1)

( )

(

( )

( )

(

1)

(

1)

(

1)

( )

1)

( )

that is, Kny belongs to the null space of the spectral projection associated with An and n . Since the operator An ; n I maps the above null space onto itself in a one-to-one manner, there is a unique wn 2 Cj n1 such that An wn ; n wn = Kn y = y; fn ; vn wn = 0:

If we de ne then

x := 1 (Ln wn ; y) = 1 ( xn wn ; y); n

n

Kn x = 1 (Kn Ln wn ; Kny) = 1 (An wn ; y; fn ) = wn ; n

n

so that

Tn x ; n x = LnKn x ; (Ln wn ; y) = Ln wn ; Ln wn + y = y: Further, since

hx ; 'n i = hx ; Knvn i = hKn x ; vn i = hwn ; vn i = vn wn = 0; we see that x 2 N (Pn ), and hence x = Sn y, as desired. (See Remark 1.25.) We summarize this discussion by noting that we need to perform the following matrix computations to implement the Elementary Iteration (E ) for the case of a nonzero simple eigenvalue  of T :

(i) Solve the eigenvalue problem An un = n un . (See Section 6.3.) Ensure that kun k n = 1, where k
k n is a norm on Cj n1 such that the sequences (kKn k) and (kLnk) are bounded. (See Corollary 5.10 and Examples 5.11, 5.12, 5.13.) [ ]

[ ]

278

5. MATRIX FORMULATIONS

(ii) Compute the unique vector vn such that

An vn = n vn ; un vn = 1:

To nd the vector vn , we may nd an eigenvector evn of the matrix An corresponding to n in the same manner as we found an eigenvector un of the matrix An corresponding to n , and let vn := evn =unevn . Alternatively, one may solve the system (An ; n In )v = 0; un v = 1

of n + 1 linear equations in n unknowns to nd its unique solution v = vn . (iii) Assuming that 'nk;1 is already computed, nd 'nk as follows: Compute (

)

( )

nk := hT'nk;1 ; 'n i = vn T'nk;1 ; fn ; y := (nk ; n )'nk;1 + (Tn ; T )'nk;1 ; bk := y; fn : ( )

(

( )

Solve the system

)

(

(

)

)

(

)

[An (In ; un vn ) ; n In ]w = bk

of n linear equations in n unknowns to nd its unique solution w = wn or equivalently, solve the system

(An ; n In )w = bk ; vn w = 0 of n + 1 linear equations in n unknowns to nd its unique solution w = wn . Then let 'nk := 'n + 1 ( xn wn ; y): ( )

n

We remark that the coecient matrix An (In ; unvn ) of the nn linear system given above is nonsingular since n 6= 0 and sp(An (In ; un vn )) = sp((In ; un vn )An (In ; un vn )) = sp(An ) [f0gnfn g: Moreover, this matrix does not change with the iterate number k.

279

5.2. ITERATIVE REFINEMENT

A similar procedure for implementing the Elementary Iteration (E ) was outlined in Section 4 of [34]. It is easily seen that for implementing the Double Iteration (D) for a nonzero simple eigenvalue  of T , the only extra computations consist of evaluations of Tx for x 2 X . In fact, each step of the Double Iteration consists of (i) an application of T and (ii) the Elementary Iteration described above. Let us now turn to the general case of a spectral set  of nite type for T such that 0 62 . Let us assume that the spectral projection P associated with T and  is of rank m, and n  m. The considerations for the implementation of the Elementary Iteration (E )

8 (0) 'n := 'n ; and for k = 1; 2; : : : > > > > < (nk) := T 'n(k;1) ; 'n > > > > : (k) 'n := 'n(k;1) + Sn ( 'n(k;1) (nk) ;

T 'n k;1 ) are similar to those for the case of a simple eigenvalue of  of T . Firstly, we need to nd a basis 'n = ['n;1 ; : : : ; 'n;m ] of the spectral subspace associated with Tn and a spectral set n such that 0 2= n . For this purpose, we may nd a basis un = [un;1 ; : : : ; un;m ] of the spectral (

)

subspace associated with An and a spectral set n such that 0 62 n. Since the subspace of Cj n1 generated by un is invariant under An , we see that An un = un n for some mm matrix n . Further, since sp(n ) = n and 0 62 n , we see that n is nonsingular. One may nd the matrix n as follows. Since the subset fun;1 : : : ; un;m g of Cj n1 is linearly independent, the mm matrix un un is nonsingular. As un An un = un un n, we obtain n = ( un un );1 un An un :

De ne

'n := Ln un ;n 1 = xn un ;n 1 : Then Tn 'n = 'n n , and Kn 'n = un as proved in Theorem 5.4(b). In particular, since un forms a basis for an m-dimensional subspace of Cj n1 , 'n also forms a basis for an m-dimensional subspace of X . Secondly, we need to nd a basis 'n of R(Pn ) such that 'n ; 'n = Im . Now since un forms a basis for the m-dimensional spectral subspace

280

5. MATRIX FORMULATIONS

associated with An and the spectral set n , there is a unique adjoint basis vn of the spectral subspace associated with An and n . If we de ne

'n := Kn vn = fn vn ;

then Tn 'n = Kn Ln Kn vn = Kn An vn = Kn vn n = 'n n and 'n ; 'n = Ln un ;n 1; Kn vn = Kn Ln un ;n 1 ; vn = An un ;n 1 ; vn = un ; vn = vn un = Im ; as desired. Thirdly, to nd 'n k for k = 1; 2; : : : we observe that ( )

'n k;1 nk ; T 'n k;1 ; 'n = 'n k;1 ; 'n nk ; T 'n k;1 ; 'n =nk ; nk = O; that is, Pn ( 'n k;1 nk ; T 'n k;1 ) = 0 : (See Theorem 1.52.) Hence we can nd 'n k if we are able to nd Sn y whenever y 2 X 1m and Pn y = 0 . Let y 2 X 1m with Pn y = 'n y ; 'n = 0 : Then (

)

( )

(

)

(

( )

(

)

( )

(

)

( )

(

)

( )

)

( )

vn Kn y = Kn y ; vn = y ; Kn vn = y ; 'n = O;

that is, Kn y belongs to the null space of the operator w 7! An w ; w n from Cj nm to Cj nm . Proposition 1.56 shows that this operator maps the null space onto itself in a one-to-one manner. Hence there is a unique wn 2 Cj nm such that An wn ; wn n = y ; fn ;

vn wn = O:

If we de ne

x := ( Ln wn ; y );n 1 = ( xn wn ; y );n 1 ;

then

Kn x = ( Kn Ln wn ; Kn y );n 1 = ( An wn ; Kn y );n 1 = wn ;

so that

Tn x ; x n = Ln Kn x ; ( Ln wn ; y ) = y :

Further, since

x ; 'n = x ; Kn vn = Kn x ; vn = wn ; vn = O;

281

5.2. ITERATIVE REFINEMENT

we see that x 2 N ( Pn ), and hence by Proposition 1.56, x = Sn y , as desired. (See Remark 1.58.) Thus to implement the Elementary Iteration ( E ), we need to perform the following matrix computations: (i) Solve the spectral subspace problem An un = un n . (See Section 6.3.) Let un := [un;1 ; : : : ; un;m ]. Ensure that kun;j k n  c and dist(un;j ; spanfun;i : i = 1; : : : ; m; i 6= j g)  d for some constants c and d independent of n and j , in a norm k
k n on Cj n1 such that the sequences (kKn k) and (kLnk) are bounded. (See Theorem 5.9 and Examples 5.11, 5.12, 5.13.) (ii) Compute the unique vectors in vn such that [ ]

[ ]

An vn = vn n ; un vn = Im : To nd the m vectors in vn , we may nd a basis evn for the spectral subspace M (An ; n ) in the same manner as we found a basis un for M (An ; n), and let vn := evn ( un evn );1 . Alternatively, we may solve the coupled Sylvester equation An v ; v n = 0 ; un v = Im ; to nd its unique solution v = vn . (iii) Assuming that 'n k;1 is already computed, nd 'n k as follows: Compute nk := T 'n k;1 ; 'n = vn T 'n k;1 ; fn ; y := 'n k;1 (nk ; n ) + ( Tn ; T ) 'n k;1 ; b k := y ; fn : Solve the single Sylvester equation An (In ; un vn ) w ; w n = b k to nd its unique solution w = wn or equivalently, solve the coupled Sylvester equation An w ; w n = b k ; vn w = O; to nd its unique solution w = wn . Then let 'n k := 'n + ( xn wn ; y );n 1 : (

)

( )

(

( )

(

( )

)

)

( )

(

)

(

)

282

5. MATRIX FORMULATIONS

A similar procedure for the implementation of the Elementary Iteration ( E ) was outlined in Section 4 of [53]. As before, the Double Iteration ( D ) poses no new problems.

5.3 Acceleration Let T 2 BL(X ) and Te 2 BL(X ) be a nite rank operator given by e := Tx

n X j =1

hx ; fej ixej = xe x; fe

for x 2 X:

Fix an integer q  2 and let 2 3 x1 o X := X q1 = x := 64 ... 75 : xi 2 X for 1  i  q : xq n

The operators T : X ! X and Te : X ! X given by 2 3 2 3 x1 Tx 1 6 .. 7 6 x 7 T 666 ... 777 := 664 ..1 775 . 4 . 5 x q ;1 xq

and

3 2 3 2 qP ;1 x1 k e e (T ; T ) Txk+1 7 6 .. 77 66 k=0 7 6 7 x1 Te 66 ... 77 := 666 7 7 . 4 . 5 4 .. 5 xq xq;1

for x 2 X were considered while discussing the accelerated spectral approximation in Section 3.2. If X is in nite dimensional, then Te need not be of nite rank. In the present section we shall show that spectral computations for Te can still be reduced to spectral computations for an nqnq matrix. This was rst shown in [11]. Our treatment is substantially dierent.

283

5.3. ACCELERATION

Recall the operators Ke : X ! Cj n1 and Le : Cj n1 ! X given by 2 hx ; fe1 i 3 7 e := 6 Kx 4 ... 5 = hx ; fen i

x; fe

for x 2 X;

and

2 3 u(1) for u := 64 ... 75 2 Cj n1 :

Leu := u(1)xe1 +


+ u(n)xen = xe u

u(n)

e As in Section 5.1, we let Ae := K e Le : Cj n1 ! Then Te = LeK: e e e e e e e e e e e e e e Then K T = K LK = AK and LA = LK L = T L: We identify ( Cj n1 )q1 with Cj nq1 and de ne Ae : Cj nq1 ! by 2 q ;1 e (T 2 3 6PK u1 k=0 6 Ae 64 ... 75 := 666 4 uq

Cj n1 . Cj nq1

3

; Te)k Leuk+1 7

u1 .. . uq;1

2 3 u1 7 7 6 for 4 ... 75 2 Cj nq1 : 7 7 5 uq

The operator Ae is represented by the following nqnq matrix with respect to the standard basis for Cj nq1 : 2 eh i eh i 3 A A





Ae hq; i 6 6 6 6 6 6 6 e A := 66 6 6 6 6 6 4

0

1

1

7

7 O 7 7

O









O

In

... ...

.. .

.. .

... ...

7 .. 77 . 7

O

O




In

O

In

7

7 .. 77 . 7; 7 5

where for k = 0; : : : ; q ; 1, Ae hki is the nn matrix given by Ae hki (i; j ) := h(T ; Te)k xej ; fei i; 1  i; j  n; and In is the nn identity matrix.

284

5. MATRIX FORMULATIONS

e : X ! Cj nq1 and L e : Cj nq1 ! X be de ned by Let K 2 3 2e 3 2 3 2e 3 x1 Kx1 u1 Lu1 6 7 6 7 6 7 6 . . . e e K 4 .. 5 := 4 .. 5 and L 4 .. 5 := 4 ... 75 e q xq uq Kx Leuq 2 3 2 3 x1 u1 6 7 6 . e and . for 4 . 5 2 X and 4 ... 75 2 Cj nq1 : It is easy to see that Te 6= Le K xq uq e e e A 6= KL. However, we have Ke Te = Ae Ke

since LeKe = Te. On the other hand, LeAe 6= Te Le since

qP ;1 (T k=0

; Te)k Te in general.

qP ;1 Te(T k=0

; Te)k 6=

Let p be a positive integer and consider the natural extensions T ; Te : ! X 1p , Ae : Cj np ! Cj np , Ke : X 1p ! Cj np and Le : jC np ! X 1p of the operators T; Te : X ! X , Ae : Cj n1 ! Cj n1 , Ke : X ! Cj n1 and Le : Cj n1 ! X . Let Z 2 Cj pp be a nonsingular matrix. We de ne an operator Le(Z) : Cj np ! X 1p by

X 1p

Le(Z) u :=

q;1 h X k=0

i

( T ; Te )k Le u Z;k

for u 2 Cj np :

Next, we let X := X 1p , and consider the natural extensions T ; Te : X ! X , Ae : Cj nqp nq!1Cj nqpnq, Ke1 : X ! Cj nqp nqofthe operators T ; Te : X ! X , Ae : Cj !2Cj 3 and Ke : X ! Cj 1 . For

x

x1

:= [x1 ; : : : ; xp ] = 64 ... 75 xq

2 3 u1 6 .. 7 2 Cj nqp , we have 4 . 5 uq

Tx

2 X and u := [u1 ; : : : ; up ] =

2 3 T x1 6 x1 7 := [Tx1 ; : : : ; Txp ] = 664 .. 775 ; . x q;1

285

5.3. ACCELERATION

Te x

2 q;1 P (T 6 k=0 6 e 1 ; : : : ; Tx e p] = 6 := [Tx 6 6 4

3

; Te )k Te x k+1 7

7 7 ; 7 7 5

x1 .. .

x q;1 3 2 q ;1 P e e )k Le u k+1 K ( T ; T 7 6 k=0 7 6 7 6 u1 ; Ae u := [Ae u1; : : : ;Ae up ] = 66 7 7 . . 5 4 . u q;1 2 e 3 K x1 Ke x := [Ke x1; : : : ; Ke xp] = 64 ... 75 : Ke x q Finally, consider the operator Le (Z) : Cj nqp ! X de ned by 2e 2 3 L(Z) u 1 3 u1 7 6 6 . e L(Z) u := 4 .. 5 for u = 4 ... 75 2 Cj nqp : uq Le(Z) u q

In order to obtain an analog of Lemma 5.2 for the operators Te and

Ae , we rst prove some preliminary results. We de ne the operator F1 : X ! X 1p by F1 y :=

q;1 X k=1

( T ; Te )k Te  k

for

k P

y

2 3 y1 6 := 4 ... 75 2 yq

X;

where  k := y `+1 Z;(k;`+1) for k = 1; : : : ; q ; 1. `=1 For i = 1; : : : ; q ; 1, let Fi+1 : X ! X 1p be de ned by h

Fi+1 y := y i + Fi y ; and F :

X!X

q;1 X

k=0

i

( T ; Te )k Te y i+1 Z;k Z;1 for

be de ned by

Fy

2 F1 y 3 := 64 ... 75 Fq y

for

y 2 X:

y 2 X;

286

5. MATRIX FORMULATIONS

Lemma 5.17

Let p be a positive integer, Z 2 Cj pp 2be such 3 that 0 2= sp(Z), and F x1 be de ned as above. Then for x := 64 ... 75 2 X , we have Te x =

xq

xZ + y

if and only if x i = x i+1 Z + y i+1 for i = 1; : : : ; q ; 1 and qP ;1 ( T ; Te )k Te x 1 Z;k = x 1 Z + y 1 + F1 y . In this case, we also

k=0

qP ;1 ( T ; Te )k Te x i Z;k = x i Z + y i + Fi y for i = 2; : : : ; q ; 1, k=0 Ae Ke x = Ke x Z + Ke y and Le (Z) Ke x = x Z + y + F y .

have

Proof

The de nition of the operator Te shows that Te x = x Z + y if and qP ;1 only if ( T ; Te )k Te x k+1 = x 1 Z + y 1 and x i = x i+1 Z + y i+1 for k=0 i = 1; : : : ; q ; 1. In this case, we obtain for k = 1; : : : ; q ; 1, x k+1 = ( x k ; y k+1 )Z;1 = [( x k;1 ; y k )Z;1 ; y k+1 ]Z;1 =


= x 1 Z;k ; kP +1 y ` Z;(k+2;`) . Hence `=2

q;1

k

h

X X x 1 Z + y 1 = Te x 1 + ( T ; Te )k Te x 1 Z;k ; y `+1 Z;(k;`+1)

= =

q;1 X k=0 q;1 X k=0

k=1

( T ; Te )k Te x 1 Z;k ;

q;1 X k=1

i

`=1

( T ; Te ) Te  k

( T ; Te )k Te x 1 Z;k ; F1 y ;

as desired. The converse follows by working backwards. Assume now that we have proved q;1 X k=0

( T ; Te )k Te x i Z;k = x i Z + y i + Fi y

for some i such that 1  i  q ; 1. Putting x i = x i+1 Z + y i+1 in this

287

5.3. ACCELERATION

equation, we obtain q;1 X k=0

( T ; Te )k Te x i+1 Z;k+1 = x i+1 Z2 + y i+1 Z + y i + Fi y

;

q ;1 X

( T ; Te )k Te y i+1 Z;k ;

k=0

so that qP ;1 (T k=0

; Te )k Te x i+1 Z;k h

i

qP ;1 = x i+1 Z + y i+1 + y i + Fi y ; ( T ; Te )k Te y i+1 Z;k Z;1 k=0 = x i+1 Z + y i+1 + Fi+1 y ;

as desired. Next,

Ae Ke x = Ke Te x = Ke x Z + Ke y and

Le (Z) Ke x

2 qP 3 i ;1 h e )k Le K e x 1 Z;k 2 e 3 6 ( T ; T 7 K x1 k=0 7 6 7 6 7 6 . . .. = Le (Z) 4 .. 5 = 66 7 7 i h q ; 1 4P 5 Ke x q k ; k e e e (T ; T ) L K xq Z k=0 2 3 x 1 Z + y 1 + F1 y 7 = xZ +F y .. = 64 5 . x q Z + y q + Fq y

since Le Ke = Te . We now carry out a similar analysis in Cj nqp . De ne an operator G1 : Cj nqp ! Cj np by

G1 v :=

q ;1 X k=1

Ke ( T ; Te )k Le w k for v

2 3 v1 6 := 4 ... 75 2 Cj nqp ; vq

288

5. MATRIX FORMULATIONS k P

where w k := v `+1 Z;(k;`+1) for k = 1; : : : ; q ; 1. `=1 For i = 1; : : : ; q ; 1, let Gi+1 : Cj nqp ! Cj np be de ned by h

Gi+1 v := v i + Gi v ; and G : Cj nqp !

q;1 X

i Ke ( T ; Te )k Le v i+1 Z;k Z;1 for v 2 Cj nqp ;

k=0 Cj nqp be de ned by

Gv

2 G1 v 3 := 64 ... 75 Gq v

for v 2 Cj nqp :

e y , and It can be easily seen that for every y 2 X , Ke F1 y = G1 K e y for i = 1; : : : ; q ; 1, so that recursively, Ke Fi+1 y = Gi+1 K

Ke F = G Ke : Lemma 5.18

Let p be a positive integer, Z 2 Cj p2p be 3such that 0 2= sp(Z), and G u1 6 be de ned as above. Then for u := 4 ... 75 2 Cj nqp , we have Ae u = uq u Z + v if and only if u i = u i+1 Z + v i+1 for i = 1; : : : ; q ; 1 and qP ;1 Ke ( T ; Te )k Le u 1 Z;k = u 1 Z + v 1 + G1 v . In this case, we also k=0

have

qP ;1 Ke ( T k=0

; Te )k Le u i Z;k = u i Z + v i + Gi v for i = 2; : : : ; q ; 1,

Ke Le (Z) u = u Z + v + G v and Te Le (Z) u where 

Proof

qP ;1 h := Le(Z) Ke ( T k=1

2 3  7 6 0 = Le (Z)( u Z + v ) + 664 .. 775, . i

0

; Te )k Le ; ( T ; Te )k Te Le(Z) w k .

The de nition of the operator Ae shows that Ae u = u Z + v if and only

289

5.3. ACCELERATION qP ;1 Ke ( T k=0

; Te )k Le u k+1 = u 1 Z + v 1 and u i = u i+1 Z + v i+1 for i = 1; : : : ; q ; 1. The proof of the remaining part of the equivalence of

if

the stated conditions is very similar to the proof of Lemma 5.17 if we note that for k = 1; : : : ; q ; 1, u k+1 = u 1 Z;1 ;

k X `=1

v `+1 Z;(k;`+1) = u 1 Z;1 ; w k :

Next,

2 qP ;1 h 2 e e 3 6 Ke ( T K L(Z) u 1 k=0 6 Ke Le (Z) u = 64 ... 75 = 666 ;1 h 4 qP Ke Le(Z) u q Ke ( T k =0 2 3 u 1 Z + v 1 + G1 v 7 = uZ + .. = 64 5 . u q Z + v q + Gq v

; ;

3 i k ; k e e T ) L u1 Z 7 7 7 .. 7 . 7 i 5 k ; k e e T ) L uq Z

v +G v:

Lastly, we have

2 qP ;1 3 6 (T 2e L(Z) u 1 6 k=0 6 e e e T L(Z) u = T 4 ... 75 = 666 Le(Z) u q 4

3

; Te )k Te Le(Z) u k+1 7 7 7 Le(Z) u 1 7: .. . Le(Z) u q;1

7 5

Now, since Te = LeKe , qP ;1 ( T ; Te )k Te Le(Z) u k+1 k=0 qP ;1 qP ;1 = Te Le(Z) u 1 + ( T ; Te )k Te Le(Z) u 1 Z;k ; ( T ; Te )k Te Le(Z) w k k=1 k=1 qP ;1 = Le(Z) Ke Le(Z) u 1 ; ( T ; Te )k Te Le(Z) w k k=1 h i qP qP ;1 ;1 e = L(Z) u 1 Z + v 1 + Ke ( T ; Te )k Le w k ; ( T ; Te )k Te Le(Z) w k k=1 k=1 = Le(Z)[ u 1 Z + v 1 ] +  :

290

5. MATRIX FORMULATIONS

Thus

2 2 3 3 2 3   u 1Z + v 1 7 7 6 7 6 6 u 0 0 Te Le (Z) u = Le(Z) 664 .. 1 775 + 664 .. 775 = Le (Z)( u Z + v ) + 664 .. 775 ; . . . u q;1 0 0

as desired. We are now in a position to prove an analog of Lemma 5.2 for the operators Te and Ae .

Lemma 5.19

Let p be a positive integer, Z 2 Cj pp such that 0 2= sp(Z), y 2 X , and e y 2 Cj nqp . Then for x 2 X and u 2 Cj nqp , the following v := K e x = u if and only if A e u = uZ + v holds: Te x = x Z + y and K and Le (Z) u = x Z + y + F y . In particular, let y := 0 (so that v = 0 also), and let x , u satisfy the above conditions. Then the set of p elements in x is linearly independent in X if and only if the set of p vectors in u is linearly independent in Cj nq1 .

Proof

e x = u . Then by Lemma 5.17, Assume that Te x = x Z + y and K Le (Z) u = Le (Z) Ke x = x Z + y + F y and Ae u = Ae Ke x = Ke Te x = Ke ( x Z + y ) = u Z + v . Conversely, assume that Ae u = u Z + v and Le (Z) u = x Z + y +F y . e (Z) u ; y ;F y ] = u Z + v +G v ; e xZ = K e [L Then by Lemma 5.18, K Ke y ; Ke F y = u Z, because Ke y = v and Ke F y = G Ke y = G v as we have mentioned just before Lemma 5.18. Since Z is nonsingular, e x = u . Again, by Lemma 5.18, it follows that K

Te x Z = Te [Le (Z) u ; y ; F y ]

2 3  6 7 0 = Le (Z)( u Z + v ) + 66 .. 77 ; Te ( y + F y ) 4.5

0

291

5.3. ACCELERATION

2 3  6 07 = ( x Z + y )Z + (F y )Z + Le (Z) v + 664 .. 775 ; Te ( y + F y ); .

0

i qP ;1 h Le(Z) Ke ( T ; Te )k Le ; ( T ; Te )k Te Le(Z) w k , and k=1 k P w k := v `+1 Z;(k;`+1) for k = 1; : : : ; q ; 1. `=1 To show that Te x = x Z + y it is enough to prove that 2 3  6 07 Le (Z) v + 664 .. 775 = Te ( y + F y ) ; F y Z: .

where  :=

0

Now

2 qP 3 2 qP 3 i i ;1 h ;1 h e )k Le v 1 Z;k e )k Te y 1 Z;k ; T ( T ; T ( T 7 6 7 6 k=0 k=0 7 6 7 6 7 6 7 6 . . .. .. Le (Z) v = 66 = 7 6 7 7 6 7 h i h i q ; 1 q ; 1 5 4P 5 4P k ; k k ; k e e e e (T ; T ) L vq Z (T ; T ) T yq Z k=0 k=0 since Le v i = Le Ke y i = Te y i for i = 1; : : : ; q ; 1. Also,

=

q;1 X q;1 X k=1 j =0 q;1 X

; =

( T ; Te )j Le Ke ( T ; Te )k Le w k Z;j

( T ; Te )k Te

k=1 qX ;1 X q;1 h k=1 j =0

q;1 X j =0

( T ; Te )j Le w k Z;j

( T ; Te )j Te ( T ; Te )k i

;( T ; Te )k Te ( T ; Te )j Te  k Z;j k k P P since Le w k = Le v `+1 Z;(k;`+1) = Te y `+1 Z;(k;`+1) = Te  k , `=1 `=1 k P ; ( k ; ` +1) for k = 1; : : : ; q ; 1. where  k = y `+1 Z `=1

292

5. MATRIX FORMULATIONS

On the other hand, e 3 2 2T q(;y1 + F y ) ; F y Z =3 2 q;1 3 P P k k e e e e ( T ; T ) T y k+1 7 6 ( T ; T ) T Fk+1 y 7 F1 y Z 6 k=0 k=0 7 6 6 7 6 F2 y Z 77 7 6 6 7 6 y1 F1 y ; + 7 6 6 7 6 7 4 ... 7 6 7 6 5 .. .. 5 4 5 4 . . Fq y Z F y y

q;1 q;1 2 q ;1 3 P e )k Te ( y k+1 ; Fk+1 y ) ; F1 y Z ( T ; T 6 7 k=0 6 7 6 7 y 1 + F1 y ; F2 y Z = 66 : 7 7 . . 4 5 . y q;1 + Fq;1 y ; Fq y Z Let 2  i  q. The ith component of Te ( y + F y ) ; F y Z is h y i;1 + Fi;1 y ; Fi y Z = y i;1 + Fi;1 y ; y i;1 + Fi;1 y q;1 i X ; ( T ; Te )k Te y i Z;k Z;1 Z k=0 qX ;1 = ( T ; Te )k Te y i Z;k ; k=0 2 3  6 07 which equals the ith component of Le (Z) v + 66 .. 77. As for the rst 4.5

components, we note that

F1 y =

0

q ;1 X j =1

( T ; Te )j Te  j

and the recursive de nition of Fk+1 y gives

Fk+1 y =

k X

y ` Z;(k;`+1) +

`=1 q;1 X

;

j =1

q;1 X j =1

( T ; Te )j Te  j Z;k

( T ; Te )j Te  k Z;j ; Te  k

293

5.3. ACCELERATION

for k = 1; : : : ; q ; 1. Employing this formula, it can be (painstakingly) e checked that for each i = 1; : : : ; q, the rst component 2 of3 T ( y + F y ) ;



607 F y Z as well as the rst component of Le (Z) v + 664 .. 775 have the same .

0 terms involving Te y i for i = 1; : : : ; q. Thus Te x = x Z + y . e y = 0 , and K e x = u, A eu= Finally, let y := 0 , so that v = K u Z, Le (Z) u = x Z. Let the p vectors in u be linearly independent in Cj nq1 , and c 2 Cj p1 e x )c = K e ( x c) = K e (0) = 0. The be such that x c = 0. Then u c = ( K linear independence of the p vectors in u implies that c = 0, as desired. Let the p elements in x be linearly independent in X , and c 2 Cj p1 be such that u c = 0. By mathematical induction, u Zj c = 0 for j = 0; 1; : : : since u Zj+1 c = u ZZj c = Ae u Zj c = Ae (0) = 0. It follows that u p(Z)c = 0 for every polynomial p in one variable. As the matrix Z is nonsingular, there is a polynomial p such that Z;1 = p(Z) by the Cayley-Hamilton Theorem (Remark 1.33). Hence u Z;1 c = 0, and in turn u Z;k c = 0 for all k = 1; 2; : : : Now 2 e 3 2 3 (L(Z) u 1 )c 0 6 7 6 7 . .. ( x Z)c = (Le (Z) u )c = 4 5 = 4 ... 5 0 (Le(Z) u q )c since for i = 1; : : : ; q, (Le(Z) u i )c =

q;1 X

k=0

(T ; Te)k ( Le ( u i Z;k ))c =

q ;1 X

k=0

(T ; Te)k Le(0) = 0:

The linear independence of the set of p elements in Zc = 0, and since Z is nonsingular, c = 0, as desired.

x

implies that

Proposition 5.20 sp(Te ) n f0g = sp(Ae ) n f0g: In particular, sp(Te ) is a nite set. Let e q  sp(Te ) n f0g, Pe := P (Te ; e q ) and Pe := P (Ae ; e q ). Then [ ]

[ ]

[ ]

Ke Pe = PeKe :

294

5. MATRIX FORMULATIONS

e maps R(Pe ) into R(Pe) in a one-to-one manner, and hence  eq Also, K is a spectral set of nite type for Te .

[ ]

Proof

We show that re(Te ) n f0g = re(Ae ) n f0g. e y , u := Let 0 6= z 2 re(Ae ). Consider y 2 X and de ne v := K e e e R(A; z )v and x := (L ([z ])u ; y)=z . Then Au = z u + v and Le ([z ])u = e ; z x = y. z x + y. Letting p := 1 in Lemma 5.19, we see that Tx e Thus R(T ; z I) = X . Next, let x 2 X be such that (Te ; z I)x = 0 e x. Letting p := 1 and y := 0 in Lemma 5.19, we and de ne u := K e see that Au = z u and Le ([z ])u = z x. Since z 2 re(Ae ), we have u = 0 and so z x = Le ([z ])(0) = 0. As z 6= 0, we obtain x = 0. Hence N (Te ; z I) = f0g. Thus z 2 re(Te ). Conversely, let 0 6= z 2 re(Te ). To show that z 2 re(Ae ), it is enough to show that N (Ae ; z I) = f0g. Let u 2 Cj nq1 be such that (Ae ; z I)u = 0 and de ne x := Le ([z ])u=z . Letting p := 1 and y := 0 in Lemma 5.19, e = z x and K e x = u. Since z 2 re(Te ), we have x = 0 and we see that Tx e so u = K(0) = 0. Hence N (Ae ; z I) = f0g, as desired. Since sp(Ae ) has at most nq elements, sp(Te ) has at most nq + 1 elements. Let e q  sp(Te ) n f0g. Then e q is a spectral set for Te as well as for Ae . Consider C 2 C (Te ; e q ) such that 0 2 ext(C). Then C 2 C (Ae ; e q ) as well. e Te = A eK e , we have Let z 2 re(Te ) n f0g. Since K [ ]

[ ]

[ ]

[ ]

Ke R(Te ; z) = R(Ae ; z) Ke : Integrating the identity given above over C, we obtain Z Z   Ke Pe = Ke ; 21 i R(Te ; z) dz = ; 21 i Ke R(Te ; z) dz C Z Z C   1 1 e e e = ; 2i R(A; z ) K dz = ; 2i R(Ae ; z ) dz K e: = PeK

C

C

ee e ee e e The equation 2 3KP = PK shows that K maps R(P ) into R(P). Now

x1

let x := 64 ... 75 xq

f := R(Pe ) such that K e x = 0. We rst claim that 2M

295

5.3. ACCELERATION

e k+1 = LeKx e k+1 = Le (0) = 0 for each k = 0; : : : ; q ;1, (Te )q x = 0. Since Tx 2 q ;1 3 2 3 P k e e 0 (T ; T ) Txk+1 7 6 6 k=0 7 6 x1 77 7 6 x1 e =6 Tx = : 6 7 6 6 7 4 ... 7 5 .. 4 5 . xq;1 xq;1

Again, we have

2 q;1 3 2 3 P e)k Tx e k 0 ( T ; T 6 7 k=1 6 7 6 0 77 6 7 6 0 6 7 6 2 x (Te ) x = 66 = 66 1 777 : 7 x1 7 6 7 4 ... 5 .. 4 5 . xq;2 xq;2

e q Proceeding  in this q fashion, we obtain (T ) x = 0, as claimed. f e f Now Te jM f;M f x = 0, since x 2 M and T maps M into itself. But eq eq e since sp(Te jM f) =  and 0 2=  , the operator T jM f;M f is invertible.  f ;M q f e q Hence Te jM f ;M f is invertible. As x 2 M and (T ) x = 0, we obtain e e x = 0. Thus the operator Ke jM f is one-to-one, and rank P  rank P  nq. Thus e q is a spectral set of nite type for Te . [ ]

[ ]

[ ]

Theorem 5.21 (a) Let e q 2 Cj n f0g. Then e q is an eigenvalue of the operator Te if [ ]

[ ]

and only if e q is an eigenvalue of the matrix Ae . In this case, let ue 2 Cj nqg form an ordered basis for the eigenspace of Ae corresponding to e[q] . Then 'e := Le ([e[q] ]) ue =e[q] forms an ordered basis for the eigenspace of Te corresponding to e[q] and sate 'e = ue . is es K In particular, the geometric multiplicity of e[q] as an eigenvalue of Te is the same as its geometric multiplicity of e[q] as an eigenvalue of Ae . [ ]

296

5. MATRIX FORMULATIONS

(b) Let e q  sp(Ae ) n f0g and consider an ordered basis ue 2 Cj nqm [ ]

for the spectral subspace M (Ae ; e q ). Then there is a nonsingular matrix e q 2 Cj mm such that Ae ue = eu e q . Further, 'e := (Le (e q ) ue )(e q );1 forms an ordered basis for the spectral subspace M (Te ; e q ) and satis es Ke 'e = ue . [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

In particular, if e q 2 sp(Ae ) nf0g, then the algebraic multiplicity of e q as an eigenvalue of Te is the same as the algebraic multiplicity of e q as an eigenvalue of Ae . [ ]

[ ]

[ ]

Proof

Recall that Ae represents the operator Ae with respect to the standard basis for Cj nq1 . (a) Letting p := 1, Z := [e q ] and y := 0 in Lemma 5.19, we see that e q is an eigenvalue of Te if and only if e q is an eigenvalue of Ae . In this case, letting p := g and Z := e q Ig in Lemma 5.19, we note e' e = eu , and the set that if 'e := Le (Z) eu =e q , then Te 'e = e q 'e , K f'e 1 ; : : : ; 'e g g of elements in 'e is a linearly independent subset of the eigenspace of Te corresponding to e q . That f'e 1 ; : : : ; 'e g g also spans this eigenspace can be seen as follows. Let 'e g+1 2= spanf'e 1 ; : : : ; 'e g g be such that Te 'e g+1 = e q 'eg+1 . If e := ['e 1 ; : : : ; 'e g+1 ] 2 X q(g+1) , then Te e = e q e , and again by Lemma 5.19, fKe 'e 1; : : : ; Ke 'e g+1g would be a linearly independent subset of the eigenspace of Ae corresponding to e q , contrary to our assumption that this eigenspace is of dimension g. (b) Since ue 2 Cj nqm forms an ordered basis for M (Ae ; e q ), Ae eu = e q for some  e q 2 Cj mm such that sp( eq)= e q . As 0 2=  eq, eq eu  is invertible. Letting p := m, Z := e q and y := 0 in Lemma 5.19, we e' e = eu , and the set of m elements in 'e is linsee that Te 'e = 'e e q , K early independent in X . Consider the closed subspace Ye of X spanned by the elements in 'e . Since the matrix e q represents the operator Te jYe ;Ye with respect to 'e , we have sp(Te jYe ;Ye ) = sp(e q ) = e q . Hence Ye  M (Te ; e q ) by Proposition 1.28, so that m  dim M (Te ; e q ). But e maps M (Te ;  e q ) into M (A e;  eq) as we have seen in Proposition 5.20, K q q e e e e in a one-to-one manner, so that dim M (T ;  )  dim M (A;  )  m. Thus Ye = M (Te ; e q ). In particular, the case e q := fe q g shows that if e q 2 sp(Ae ) n f0g, [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

297

5.3. ACCELERATION

then the algebraic multiplicity of e q as an eigenvalue of Te is the same as the algebraic multiplicity of e q as an eigenvalue of Ae . [ ]

[ ]

The result in part (b) of Theorem 5.21 is stated without proof in [16]. Let T 2 BL(X ) and  be a spectral set of nite type for T such that 0 2=  and m := dim M (T; ). Consider a sequence (Tn ) of nite  T . As we have seen in Theorem rank operators on X such that Tn ! 3.12, for each large n, there is a spectral set nq of nite type for Tnq which does not contain 0 and approximates . Fix n and write Te := Tn, Te := Tnq and e q := nq . Theorem 5.21 says that e q can be located as a spectral set for the matrix Ae . Further, Theorem 3.15 shows that if 'e forms an ordered basis for the associated spectral subspace M (Te ; e q ), and Te 'e = 'e e q , where e q is a nonsingular mm matrix, then [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

2 3 'e[q] e [q] );1 7 6 'e[q] ( 7 'e = 664 7 2 X; .. 5 . [q ] e [q ] ;q +1 'e ( ) and its rst component 'e[q] provides an approximation of a basis for the spectral subspace M (T; ). Theorem 5.21 says that 'e[q] can be found in the following manner. Find a basis eu for the spectral subspace M (Ae ; e [q] ). Then Ae eu = eu e [q] , where the nonsingular matrix e [q] is

given by Let

and

e q = ( eu  eu );1 eu  Ae eu : [ ]

2 3 eu 1 6 ue = [eu1 ; : : : ; eum ] = 4 ... 75 2 Cj nqm ; Le := Le (e [q] ); eu q 2e 3 L eu 1 6 [q ] ;1 e e 'e := (L eu )( ) = 4 ... 75 (e [q]);1 : Le eu q

Hence the accelerated approximation of a basis for M (T; ) is given by 'e q := (Le u 1 )(e q );1 [ ]

[ ]

= ( Le u 1 )(e [q] );1 +

q ;1 X k=1

( T ; Te )k Le u 1 (e q );k;1 [ ]

298

5. MATRIX FORMULATIONS

= xe u 1 (e q );1 + [ ]

q;1 X k=1

( T ; Te )k ( xe u 1 )(e q );k;1 : [ ]

e 'e = eu , the Gram product 'e q ; fe is equal to Moreover, since K Ke 'e q = u 1 . In the special case e q = fe q g, where e q is a nonzero eigenvalue of Te of geometric multiplicity g and we are interested only in nding the rst components 'e q of a basis 2 3for the corresponding eigenspace of eu 1 6 Te , we may nd a basis eu = 4 ... 75 2 Cj nqg of the eigenspace of Ae eu q q q e e corresponding to  , so that  = e q Ig ; and then let [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

[ ]

q;1 e u1 X e ke eu 1 L L q 'e := e[q] = e[q] + ( T ; eT[q]) kL u 1   ( ) k=1 qX ; 1 e k = xee[uq]1 + ( T ; Te[q)] (k xe u 1 ) : ( )  k=1 [ ]

Let T 2 BL(X ) and (Tn ) be a sequence of bounded nite rank operators considered in Chapter 4. In Table 5.4, we give the entries Ae hki (i; j ) := h(T ; Tn)k xn;j ; fn;i i; 1  i; j  n; of the nn matrices Ae hki , k = 0; : : : ; q ; 1, which are involved in the nqnq matrix Ae . Note that Ae h0i = An := xn ; fn . We use the same notation as in Table 5.1. Further, for a kernel k(
;
) in C 0 ([a; b]  [a; b]) and a xed etj 2 [a; b], we let ehj (s) := k (s; etj )

for s 2 [a; b]:

It is possible to obtain uniformly well-conditioned bases for the approximate spectral subspaces M (T n ; nq ) by choosing uniformly wellconditioned bases for the approximate spectral subspaces M (An ; nq ). See Theorem 5.9, Exercise 5.15 and Theorem 4.1 of [16]. Such uniformly well-conditioned bases are needed for deriving the error estimate k 'nq ; P 'nq k1 = O(k(T ; Tn )q Tnk) stated in Theorem 3.15(a). [ ]

[ ]

[ ]

[ ]

299

5.3. ACCELERATION

Ae hki (i; j )

TnP TnS

hT (T ; TnP )k ej ; ei i h(T ; TnS )k T eej ; ei i r~ X

TnG

TnD TnN

h(T ; TnG)k ej ; e` ihT ee` ; ei i

`=1 r~ X `=1 Zb a

hT eej ; e` ih(T ; TnG)k ej ; ei i

[(T ; TnD )k xej ](t)yei (t) dt

wej [(T ; TnN )k ehj ](eti ) r~ X

TnF

`=1

wej

we` k(eti ; etj )[(T ; TnF )k eej ](et` )

r~ X `=1

k(et` ; etj )[(T ; TnF )k e`](eti )

Table 5.4

300

5. MATRIX FORMULATIONS

5.4 Numerical Examples

The matrix formulations discussed in this chapter can be implemented on a computer by making use of standard routines. In this section we present some illustrative numerical examples. We have made use of MATLAB for this purpose. The underlying complex Banach space is X := C 0 ([0; 1]) with the norm k
k1 . For x 2 X , let (Tx)(s) =

Z1 0

k(s; t)x(t) dt; s 2 [0; 1];

where the kernel k(
;
) is de ned on [0; 1][0; 1]. Except in Example 5.26, we replace Tx by TN x in our computations, where TN is a member of a speci ed approximation (Tn ) of T , and N is large. Also, we replace kxk1 by the 1-norm of the vector [x(tN;1 ); : : : ; x(tN;N )]> in Cj N 1 , where tN;1; : : : ; tN;N are the speci ed nodes in [0; 1]. By (j ) we denote the j th eigenvalue of T , and by n (j ) the j th eigenvalue of Tn in the descending order of modulus.

Example 5.22 Kernel

k(s; t) := est ; s; t 2 [0; 1]

Approximation

Nystrom (Subsection 4.2.2)

Quadrature formula

Gauss Two Point Rule

Numerical parameters n = 10 i; i 2 [ 1; 12 ] Eigenvalues

n (j ); j 2 [ 1; 4 ]

The kernel is in nitely dierentiable with respect to each of the two variables s and t.

5.4. NUMERICAL EXAMPLES

301

n n (1) n (2) n (3) n (4) 10 (:135302849429)101 (:1059756286)100 (:35524057)10;2 (:744126)10;4 20 (:135303006009)101 (:1059827474)100 (:35602204)10;2 (:762500)10;4 30 (:135303014406)101 (:1059831298)100 (:35606444)10;2 (:763538)10;4 40 (:135303015820)101 (:1059831942)100 (:35607159)10;2 (:763715)10;4 50 (:135303016206)101 (:1059832118)100 (:35607355)10;2 (:763763)10;4 60 (:135303016345)101 (:1059832181)100 (:35607425)10;2 (:763780)10;4 70 (:135303016404)101 (:1059832209)100 (:35607455)10;2 (:763788)10;4 80 (:135303016433)101 (:1059832222)100 (:35607470)10;2 (:763791)10;4 90 (:135303016449)101 (:1059832229)100 (:35607478)10;2 (:763793)10;4 100 (:135303016457)101 (:1059832233)100 (:35607482)10;2 (:763795)10;4 110 (:135303016463)101 (:1059832235)100 (:35607485)10;2 (:763795)10;4 120 (:135303016466)101 (:1059832237)100 (:35607487)10;2 (:763796)10;4 A posteriori error bounds and the computations of 100 (1), 100 (2), 100 (3) are used in Example 4.22 to conclude that (1) = 1:35303016 : : : (2) = 0:1059832 : : : and (3) = 0:00356 : : : The computation of 120 (4) is referred to in Exercise 4.17.

Example 5.23 Kernel

k(s; t) := ln js ; tj; 0  s 6= t  1

Approximation

Kantorovich-Krylov (Subsection 4.2.3)

Quadrature formula

Compound Trapezoidal Rule

grids with n nodes for some Numerical parameters Uniform n 2 [ 11; N ] ; where N = 501 Eigenvalue

n (3); N (3)

302

5. MATRIX FORMULATIONS

The kernel is weakly singular. As in 5.1.1, we obtain an eigenvector 'n of TnK by using an eigenvector of the matrix AKn corresponding  j ; 1  to its N  1 eigenvalue n (3). De ne vn 2 Cj by vn (j ) := 'n N ; 1 for j in [ 1; N ] , and n := vn AKN vn =vnvn . The relative residual in the 1-norm is shown in the following table.

n n (3)

kTNK 'n ; n 'n k1 k'n k1

n

(:28)10;01 (:17)10;02 (:50)10;03 (:47)10;04 (:67)10;05 (:70)10;06

11 ;:428340 ;:43018041 51 ;:429776 ;:42996942 101 ;:429908 ;:42996587 301 ;:429959 ;:42996464 451 ;:429963 ;:42996453 496 ;:429964 ;:42996452 1

0.5

0

n=11 n=501

−0.5

−1 0

0.2

0.4

0.6

0.8

1

The gure above shows an eigenvector ' of T K and an eigenvector ' of T K normalized such that k' k1 = k' k1 = 1. 11

501

501

11

11

501

303

5.4. NUMERICAL EXAMPLES

Example 5.24 

Kernel

5 if 0  s  t  1 k(s; t) := 3t s= ; 2s=5 if 0  t  s  1

Approximation

Fredholm (Subsection 4.2.2)

Quadrature formula

Compound Trapezoidal Rule

Numerical parameters Uniform grids: n = 11 and N = 1001 nodes Eigenvalues

n (5); N (5)

The kernel is the Green Function for the Sturm-Liouville Problem: ;'00 =  ' on [0; 1], '(0) = 0, 2'0 (0) + 3'0 (1) = 0. It is a continuous function on [0; 1][0; 1], but it is not smooth. All the eigenvalues of the corresponding Fredholm integral operator T are positive. Each one of them is a simple eigenvalue. In particular, the simple eigenvalue (5) of T is approximated by a simple eigenvalue n (5) of Tn . The results of iterative re nement with the Elementary Iteration (E ) and with the Double Iteration (D) are shown in the following table. Computations are stopped when the relative residual in the 1-norm is less than (:5)10;9. In this table k denotes the iterate number. We observe that the desired accuracy is achieved by the fourth iterate of the Double Iteration, while only the ninth iterate of the Elementary Iteration achieves this accuracy.  T , the iterates of the Elementary Iteration are expected Since TnF ! to converge in a semigeometric manner, while the iterates of the Double Iteration are expected to converge in a geometric manner.

304

5. MATRIX FORMULATIONS

(E )

k

(D)

kTN 'nk ; nk 'nk k1 kTN nk ; nk k'nk k1 k nk k1 ( )

0 1 2 3 4 5 6 7 8 9

( )

( )

( )

( )

( )

( )

(:82)10;3 (:24)10;3 (:45)10;4 (:74)10;5 (:12)10;5 (:21)10;6 (:35)10;7 (:61)10;8 (:11)10;8 (:19)10;9

(:82)10;3 (:17)10;4 (:27)10;6 (:44)10;8 (:75)10;10

(k) n k1

100 initial last 50

0

−50

−100 0

0.2

0.4

0.6

0.8

1

The gure above shows the initial approximation of an eigenvector corresponding to N (5) and the last iterate in the Elementary Iteration.

305

5.4. NUMERICAL EXAMPLES

The initial approximation is a piecewise linear function on [0; 1] having ten linear pieces.

Example 5.25 

Kernel

2 if 0  s  t  1 k(s; t) := s= t ; s=2 if 0  t  s  1

Approximation

Fredholm (Subsection 4.2.2)

Quadrature formula

Compound Trapezoidal Rule

Numerical parameters Uniform grids: n = 21 and N = 2001 nodes Eigenvalues

Clusters fn (5); n (6)g; fN (5); N (6)g

The kernel is the Green Function for the Sturm-Liouville Problem: ;'00 =  ' on [0; 1], '(0) = 0, '0 (0) + '0 (1) = 0. It is a continuous function on [0; 1][0; 1], but it is not smooth. The eigenvalues of the corresponding Fredholm integral operator T are 1=(2j ; 1)2 2 ; j = 1; 2; : : : Each one of them is a double defective eigenvalue. In particular, the double eigenvalue (3) of T is approximated by a cluster fn (5); n (6)g of two simple eigenvalues of Tn. The results of iterative re nement with the Elementary Iteration ( E ) and with the Double Iteration ( D ) are shown in the following table. Computations are stopped when the relative residual in the 1-norm is less than (:5)10;9. In this table k denotes the iterate number. We observe that the desired accuracy is achieved by the third iterate of the Double Iteration, while only the seventh iterate of the Elementary Iteration achieves this accuracy.  Since TnF ! T , the iterates of the Elementary Iteration are expected to converge in a semigeometric manner, while the iterates of the Double Iteration are expected to converge in a geometric manner.

306

5. MATRIX FORMULATIONS

(E )

k

(D)

k TN 'n k ; 'n k nk k1 k TN n k ; n k nk k1 k 'n k k1 k n k k1 ( )

0 1 2 3 4 5 6 7

( )

( )

( )

( )

( )

( )

(:35)10;5 (:29)10;6 (:78)10;7 (:31)10;7 (:49)10;8 (:29)10;8 (:60)10;9 (:24)10;9

(:35)10;5 (:82)10;7 (:19)10;8 (:29)10;9

( )

80 60

initial last

40 20 0 −20 −40 −60 −80 0

0.2

0.4

0.6

0.8

1

307

5.4. NUMERICAL EXAMPLES 150 100

initial last

50 0 −50 −100 −150 0

0.2

0.4

0.6

0.8

1

The spectral subspace associated with the operator TN and the cluster fN (5); N (6)g of its eigenvalues is two dimensional. Each of the preceding two gures show the initial approximation of an element of a basis for this subspace and the corresponding last iterate of the Double Iteration. The initial approximations are piecewise linear functions on [0; 1] having twenty linear pieces. Further, since the 22 matrix n was chosen to be upper triangular, the initial approximation in the rst of these gures is an eigenvector of the operator Tn corresponding to its eigenvalue n (5). The initial approximation in the second gure is a linear combination of two eigenvectors of the operator Tn, one corresponding to its eigenvalue n (5) and the other corresponding to its eigenvalue n (6).

308

5. MATRIX FORMULATIONS

Example 5.26 

Kernel

(1 ; t) if 0  s  t  1 k(s; t) := st(1 ; s) if 0  t  s  1

Approximation

Galerkin (Section 4.1)

Projection

Piecewise linear interpolatory

grids: n = 10; 20; 30; 60 nodes Numerical parameters Uniform Acceleration order: q = 1; 2; 3

n (2); (2)

Eigenvalues

The kernel is the Green Function for the Sturm-Liouville Problem: ;'00 =  ' on [0; 1], '(0) = 0 = '(1). The kernel is a continuous function on [0; 1][0; 1], but it is not smooth. The eigenvalues of the corresponding Fredholm integral operator T are 1=j 22 , j = 1; 2; : : : Each one of them is a simple eigenvalue. In this example the computation of Tx for x 2 X is done in exact arithmetic. Note that for the accelerated approximation of order q with n nodes, we solve a matrix eigenvalue problem of size nq. The relative residuals in the 1-norm for the usual approximation (that is, when q = 1) and for the accelerated approximation (that is, when q > 1) are shown in the following table.

q

q q q n kT'n ; qn 'n k1 k'n k1

1 2 3

60 30 20

[ ]

[ ]

[ ]

[ ]

(:36)10;4 (:99)10;4 (:54)10;5

309

5.5. EXERCISES

Note that if q = 1, then 'nq = 'n . Also, as stated in Exercise 5.12, if Tn = TnG and q = 2, then 'nq = T'n . This may explain why 'nq , when q = 2 and n = 30, does not compare well with 'nq , when q = 1 and n = 60, in this special case. [ ]

[ ]

[ ]

[ ]

0.5 q=1 q=3

0

−0.5 0

0.2

0.4

0.6

0.8

1

The gure above shows two approximate eigenvectors corresponding to (2); one is computed with n = 10; q = 1 and the other with n = 10; q = 3. The rst is a piecewise linear function on [0; 1] having ten linear pieces.

5.5 Exercises

Unless otherwise stated, X denotes a Banach space over Cj , X 6= f0g and T 2 BL(X ). Prove the following assertions.

5.1 Let p be a positive integer, Z 2 Cj pp such that 0 2= sp(Z), v 2 Cj np and y := Le v 2 X 1p . Then for x 2 X 1p and u 2 Cj np ,

310

5. MATRIX FORMULATIONS

the following holds: Te x = x Z + y and Ke x = u Z + v if and only if Ae u = u Z + v and Le u = x . In particular, let v := 0 (so that y = 0 also), and let x , u satisfy the above conditions. Then the set of p elements in x is linearly independent in X if and only if the set of p vectors in u is linearly independent in Cj n1 . Also, if z 2 re(Te) n f0g = re(Ae) n f0g, then e z ) = L R(A; z )K ; I R(T; z e

e

e

e z ) = K R(T; z )L ; I : and R(A; z e

e

e

(Compare Lemma 5.2.) r(n)

5.2 (a) Let n be a projection in BL(X ) given by nx = P hx ; en;j ien;j , j =1 x 2 X . If hTen;j ; en;i i = hen;i ; T en;j i, i; j = 1; : : : ; r(n), then every nonzero eigenvalue of each of the operators TnP , TnS and TnG is real, and its algebraic multiplicity equals its geometric multiplicity. In particular, this is the case if X is a Hilbert space, T is a self-adjoint operator and n is an orthogonal projection. (Hint: Theorem 5.4.) (b) Let X = C ([a; b]) and T be a Fredholm integral operator with a continuous kernel k(
;
) such that k(s; t) = k(t; s) for all s; t 2 [a; b]. rP (n) Let Qn be a quadrature rule given by Qn x = wn;j x(tn;j ) such that j =1 wn;j > 0 for all j = 1; : : : ; r(n). Then every nonzero eigenvalue of each of the operators TnN and TnF is real, and its algebraic multiplicity equals its geometric multiplicity. (Hint: The matrix [wn;j k(tn;i ; tn;j )] is similar to the matrix [pwn;i k(tn;i ; tn;j )pwn;j ]. Also, Remark 1.1 and Exercise 1.19.)

5.3 Let n be an interpolatory projection on C 0 ([a; b]) and Qn be the

quadrature formula induced by n . If TnN is the Nystrom approximation of T based on Qn, then TnN = TnN n . Let TnF := n TnN , n be a nonzero simple eigenvalue of TnF and 'n be a corresponding eigenvector. De ne 'Nn := TnN 'n =n and 'nN = 'n . Then TnN 'Nn = n 'Nn , (TnN ) 'nN = n 'nN and h'Nn ; 'nN i = 1. Similar results hold for an ordered basis 'n of M (TnF ; n ), where n is a nite subset of sp(TnF ) such that 0 62 n .

5.4 Let Te, Ke , Le, Ae and e be as in Theorem 5.4(a), and Ae = Ke Le. For e )j ) and Vj := N ((Ae ; I e )j ). each positive integer j , let Yj := N ((Te ; I Then Ke maps Yj into Vj in a one-to-one manner and Le maps Vj into Yj

311

5.5. EXERCISES

in a one-to-one manner, so that dim Yj = dim Vj . As a result, the ascent of e as an eigenvalue of Te is equal to its ascent as an eigenvalue of Ae .

5.5 Let TeK , Ue , Ke , Le, AeK , De and e be as in Theorem 5.8(a), and e )j ) AeK = Ke Le + De . For each positive integer j , let YjK := N ((TeK ; I K K j K K e ) ). Then K e maps Yj into Vj in a one-to-one and Vj := N ((Ae ; I manner and Le j maps VjK into YjK in a one-to-one manner, where ( e (I ; Ue );1 Leh i if j = 1; Le j := (I e ; Ue );1 L e ) if j  2; e ; Le j;1 (AeK ; I

so that dim YjK = dim VjK . As a result, the ascent of e as an eigenvalue of TeK is equal to its ascent as an eigenvalue of Ae K .

5.6 Let TeK , Ue , AeK and Ee be as in Theorem 5.8. Then sp(TeK ) = sp(AeK ) [ Ee . Also, if either (i) e 2 sp(AeK ) n Ee , or if (ii) e 2 Ee and e is

an eigenvalue of Ue , then e is an eigenvalue of TeK . (Hint: Proposition 5.7 and Exercise 1.9.)

5.7 Let Kn; un and 'n be as in Theorem 5.9. If k
k n is a norm on jC n1 such that kKnk ! 0, then for each j = 1; : : : ; m, [ ]

dist('n;j ; spanf'n;i : i = 1; : : : ; m; i 6= j g) ! 0: (Hint: 'n;j = Kn(un;j ), j = 1; : : : ; m.) 5.8 Let k
k n be a norm on Cj n1 and fu1; : : : ; umg be a linearly independent subset of Cj n1 such that kuj k n = 1 for each j = 1; : : : ; m. If dist(uj ; spanfui : i = 1; : : : ; j ; 1g)  d0 > 0 for all j = 2; : : : ; m, m then dist(uj ; spanfui : i = 1; : : : ; m; i 6= j g)  (1 + dd0 )m+1 for all 0 j = 1; : : : ; m. [ ]

[ ]

5.9 Let n be a nonzero simple eigenvalue of TnG, 'n be a corresponding eigenvector of TnG, and 'n be the eigenvector of (TnG) corresponding to n such that h'n ; 'n i = 1. (a) Let 'Pn := 'n , 'nP := T 'n =n , 'Sn := T'n=n and 'nS := 'n . Then TnP 'Pn = n 'Pn , h'Pn ; 'nP i = 1, TnS 'Sn = n 'Sn , h'Sn ; 'nS i = 1. (b) For k = 1; 2; : : : let (nk) (G), '(nk) (G); (nk) (P ); '(nk) (P ) and (nk) (S ), k 'n (S ) be the kth iterates in the Elementary Iteration (E ) for TnG, TnP ( )

312

5. MATRIX FORMULATIONS

and TnS , respectively. Then n1 (G) = n , 'n1 (G) = 'Sn , n2 (G) = 1n (P ), and for k = 1; 2; : : : nk (S ) = nk (P ), 'nk (S ) = T'nk (P )=n . Also, if for k = 1; 2; : : : nk (P ), nk (P ) and nk (S ), nk (S ) denote the kth iterates in the Double Iteration (D) for TnP and TnS respectively, then nk (S ) = nk (P ) and nk (S ) = T nk (P )=n : (Hint: SnS T = TSnP .) 5.10 Let n be a nite subset of sp(TnG) such that 0 62 n, 'n be an ordered basis for the spectral subspace M (TnG; n ), TnG 'n = 'n n, and let 'n be the adjoint ordered basis for M ((TnG) ; n ). (a) Let 'nP := 'n , 'nP := T  'n (n );1 , 'nS = T 'n ;n 1 and  'nS := 'n . Then 'nP (resp., 'nS ) forms an ordered basis for the spectral subspace M (TnP ; n) (resp., M (TnS ; n )) and TnP 'nP = 'nP n, 'nP ; 'nP = I, TnS 'nS = 'nS n , 'nS ; 'nS = I. (b) For k = 1; : : : let nk (G), 'n k (G); nk (P ), 'n k (P ) and nk (S ), 'n k (S ) be the kth iterates in the Elementary Iteration ( E ) for TnG, TnP and TnS respectively. Then n1 (G) = n , 'n 1 (G) = 'nS , n2 (G) = n1 (P ), and for k = 1; 2; : : : nk (S )n = n nk (P ), 'n k (S ) = T 'n k (P );n 1 : Also, if for k = 1; 2; : : : nk (P ), n k (P ) and nk (S ), n k (S ) denote the k th iterates in the Double Iteration ( D ), then nk (S )n = n nk (P ) and n k (S ) = T n k (P );n 1 . (Hint: SnS T = T SnP .) 5.11 Let Tn 2 BL(X ), Kn, Ln, An, An , Tn, Ln and Kn be as in Section 5.2. (a) Let n be a nonzero simple eigenvalue of Tn , and let un , vn be as in Section 5.2. For y 2 X; we have  P y = hy ; Kn vn i L u and S y = 1 (L w ; y + P y); ( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

n

( )

n n n n where wn satis es An wn ; n wn = Kn (y ; Pn y) and vn wn = 0. (b) Let n be a spectral set of nite type for Tn such that 0 2= n, and let un , vn , n be as in Section 5.2. For y 2 X 1m ; we have Pn y = y ; Kn vn Ln un ;n 1 and Sn y = ( Ln wn ; y + Pn y );n 1 ; where wn satis es An wn ; wn n = Kn ( y ; Pn y ) and vn wn = O. 5.12 Let Tn := TnG, the Galerkin approximation of T , and q = 2. (a) Let n 2 Cj , n 6= 0. Then n is an eigenvalue of Tn if and only if n itself is an eigenvalue of Tn . Further, if 'n is an eigenvector of

n

n n

( )

n

313

5.5. EXERCISES 



n Tn corresponding to n , then ' n := T'T'= an eigenvector of Tn n n corresponding to n . Also, if f'n;1 ; : : : ; 'n;g g is a linearly independent set of eigenvectors of Tn corresponding to n , then f' n;1 ; : : : ; ' n;g g is a linearly independent set of eigenvectors of Tn corresponding to n . (b) Let   Cj be a spectral set of nite type for T such that 0 62 .  T . For large n, let  be the spectral set of nite Assume that TnG ! n type for Tn as given by Theorem 2.18. Then n itself is a spectral set of nite type for Tn . Further, if 'n 2 X 1m forms  an ordered  basis for T ' n M (Tn ; n ) and Tn 'n = 'n n , then 'n := T ' ;1 forms an n n ordered basis for M (Tn ; n ) and satis es Tn ' n = 'n n .

5.13 Let Te 2 BL(X ) and q  2. Then the following conditions are qP ;1 equivalent: (i) I ; Te q is invertible in BL(X q ); (ii) I ; (T ; Te)k Te is [ ]

[ ]

k=0 qP ;1 k invertible in BL(X ); (iii) I ; Ke (T ; Te) Le is invertible in BL( Cj n1 ); k=0 and (iv) I ; Ae is invertible in BL( Cj nq1 ). As a result, re(Te [q] ) n f0g = re(Ae ) n f0g: (Hint: Let w = 1=z and consider the operators w? Te [q] , w? Ae obtained by replacing T , Te, Le by wT , wTe, wLe, respectively.)

5.14 Let e q , Te , Ke , Le ([e q ]), Ae be as in Theorem 5.21(a), and Ae , F [ ]

[ ]

be as in Lemma 5.19 with p := 1 and Z := [e q ]. For each positive e integer j , let Y j := N ((Te ; e q I)j ) and V j := N ((Ae ; e q I)j ). Then K e maps Y j into V j in a one-to-one manner and L j maps V j into Y j in a one-to-one manner, where [ ]

[ ]

Le j :=

(

[ ]

(e q );1Leh ([e q ]) i if j = 1; q ; 1 q q e e e e e e ( ) L ([ ]) ; (I ; F )L j;1 (A ;  I) if j  2; [ ]

[ ]

[ ]

[ ]

[ ]

so that dim Y j = dim V j . As a result, the ascent of e q as an eigenvalue of Te is equal to its ascent as an eigenvalue of Ae . 5.15 Uniformly well-conditioned bases for accelerated approximation: Let  O, where each T is a nite rank operator on X . Let (Tn ; T )2 ! n Tn = KnLn and k
k n be a norm on Cj n1 such that (kKn k) and (kLn k) are bounded sequences, as in Subsection 5.1.2. Consider a spectral set  of nite type for T such that 0 62 : Let q  2 be an integer and [ ]

[ ]

314

5. MATRIX FORMULATIONS

u n = [un;1 ; : : : ; un;m] form an ordered basis for M (An ; nq ) such that for all large n and j = 1; : : : ; m, kun;j k1  c; dist(un;j ; spanfun;i : i = 1; : : : ; m; i 6= j g)  d for some constants c and d satisfying 0 < d  c: Let An u n = u n nq for some nonsingular nq 2 Cj mm , and de ne 'n := Ln (nq ) u n(nq );q1 = ['n;1; : : : ; 'n;m]. Then 'n forms an ordered basis for M (T n ; n ) such that for all large n and j = 1; : : : ; m; k' n;j k1  ; dist(' n;j ; spanf'n;i : i = 1; : : : ; m; i 6= j g)   for some constants and  satisfying 0 <   . (Compare 5.9 and [ ]

[ ]

[ ]

[ ]

[ ]

[ ]

Theorem 3.15.)

Chapter 6 Matrix Computations This chapter is devoted to some topics in Numerical Linear Algebra which are relevant to spectral computations and to the considerations of the accuracy of such computations. The main references for these topics are [74], [39] and [42]. The QR Method is the most widely used algorithm for computing eigenvalues and eigenvectors of large full matrices. We describe the Basic QR Method and prove its convergence under suitable hypotheses. A twofold discussion of the error analysis is given next. The forward error analysis involves nding a condition number, that is, a number which gives a bound for the relative error in the exact solution of a perturbed problem. The backward error analysis gives a measure of the stability of the computed solution of a problem by showing that the computed solution is in fact the exact solution of a perturbed problem. The condition number for a problem does not depend on the speci c method that may be used to solve the problem, while the stability considerations depend on the speci c algorithm used to solve that problem. Condition numbers for the following problems are considered: solution of a linear system, computation of a multiple eigenvalue of a matrix, and solution of a Sylvester equation. Stability of the Basic QR Method is discussed. An algorithm, which involves the QR Method and Gaussian Elimination with partial pivoting, is suggested for solving a Sylvester Equation, and its stability is discussed. It may be recalled from Section 5.2 that we need to solve Sylvester equations for implementing the Elementary Iteration ( E ) and the Double Iteration ( D ) for the case of a cluster of eigenvalues. The total relative error is less than or equal to the product of the condition number and the relative perturbation of the data. A stopping criterion for the Basic QR Method is given which ensures that the total relative error is less than a prescribed level. 315

316

6. MATRIX COMPUTATIONS

6.1 QR factorization

Let X denote an n-dimensional Hilbert space with inner product h
;
i and let e := [e1 ; : : :; en ] form an orthonormal basis for X . If ' := ['1 ; : : :; 'm ] is an ordered set of elements in X , then each 'i is a unique linear combination of the vectors in e , that is, there exists a matrix V 2 Cj nm such that

' = e V;

so that V = ' ; e :

Thus the j th column V consists of the coordinates of 'j relative to the basis e . If ' is an orthonormal set and Q 2 Cj nm is such that ' = e Q, then Im = ' ; ' = e Q; e Q = Q e ; e Q = Q In Q = QQ;

that is, Q is a unitary matrix. In this chapter, we let X := Cj n1 with the canonical inner product hx ; yi := y x for x; y 2 Cj n1 : For x := [x(1); : : :; x(n)]> 2 Cj n1 , let

kxk :=

n X

2

i=1

and for a matrix A 2 Cj mn , let

jx(i)j2

1=2

kAk := supfkAxk : x 2 Cj n1 ; kxk  1g: 2

2

2

Unless otherwise speci ed, the orthonormal basis of reference will be assumed to be the standard basis [e1 ; : : :; en ], where ei := [0; : : :; 0; 1; 0; : : :; 0]> 2 Cj n1 ; the ith entry being equal to 1 and the other entries equal to 0.

Lemma 6.1

Let n, m be integers such that 1  m  n: For every matrix A 2 Cj nm such that rank A = m, there are matrices Q 2 Cj nm and R 2 Cj mm such that

6.1. QR FACTORIZATION

317

(i) Q is a unitary matrix, (ii) R is a nonsingular upper triangular matrix, and (iii) A = QR: If the entries of A are real, those of Q and R can also be chosen real.

Proof

This result is just a matrix version of the Gram-Schmidt Process presented in Proposition 1.39 and revisited in Example 1.45. Indeed, let xj = A(1; j )e1 +


+ A(n; j )en for j 2 [ 1; m ] . The Gram-Schmidt process applied to this set will give rise to an ordered set of m orthonormal vectors [q1 ; : : : ; qm ] such that the j th vector in this set is a linear combination of the rst j vectors of the former set. Let xj := r1;j q1 +: : :+rj;j qj for j 2 [ 1; m ] , Q := [q1 ; : : : ; qm ], R(i; j ) := ri;j for i; j 2 [ 1; m ] and R(i; j ) := 0 for i 2 [ 2; m ] , j 2 [ 1; i ; 1 ] . Then A = QR, where Q is a unitary matrix since the set fq1 ; : : : ; qm g is orthonormal, and the matrix R is upper triangular. If A(i; j ) 2 IR for all i; j , then clearly Q(i; j ) 2 IR and R(i; j ) 2 IR for all i; j .

Theorem 6.2 QR Factorization: Let n, m be integers such that 1  m  n: For every matrix A 2 Cj nm such that rank A = m, there is a unique pair of matrices (Q(A); R(A)), where Q(A) 2 Cj nm and R(A) 2 Cj mm such that (i) (ii) (iii) (iv)

Q(A) is unitary, R(A) is upper triangular, A = Q(A)R(A), and all the diagonal entries of R(A) are positive.

Proof

Let R and Q be matrices given by Lemma 6.1. Since R is nonsingular, R(i; i) 6= 0 for i 2 [ 1; m ] and di := R(i; i)=jR(i; i)j is well de ned. Set D := diag [d1 ; : : :; dm ], which is unitary. Clearly Q(A) := QD is unitary, R(A) := D R is upper triangular with positive diagonal entries, and Q(A)R(A) = A. Suppose now we have two pairs (Q; R) and (Qe ; Re ) satisfying the same properties as (Q(A); R(A)). Since R and Re are nonsingular, Qe Q =

318

6. MATRIX COMPUTATIONS

e ;1 , which is a unitary upper triangular matrix with positive diagonal RR entries. Hence Qe  Q = I. It follows that Qe = Q and R = Re .

Let U := fU 2 Cj nm : rank U = mg. Clearly, U 2 U if and only if det(U U) 6= 0. Since U 2 Cj nm 7! det(U U) 2 IR is a continuous function, it follows that U is an open subset of Cj nm . The maps Q : U ! Cj nm ; A 7! Q(A) and R : U ! Cj mm ; A 7! R(A) are known as the unitary factor map and the upper triangular factor map, respectively. Any two matrices Q and R such that Q is unitary, R is upper triangular, and QR = A will be called QR factors of A.

Theorem 6.3 Cholesky Factorization: Let B 2 Cj nn be a Hermitian positive de nite matrix. Then there exists a unique upper triangular matrix R 2 Cj nn with positive diagonal entries such that B = RR. Moreover, R depends continuously on B. Proof

Let B1=2 be the unique Hermitian positive de nite square root of B (see Remark 1.3), and Q := Q(B1=2 ), R := R(B1=2 ). Then B = RR, where R is an upper triangular matrix with positive diagonal entries. To prove the uniqueness of R, suppose that Re is an upper triangular matrix with positive diagonal entries satisfying B = Re Re . Then e ;1 ) = RRe ;1 , the matrix on the left being lower triangular and (RR the one on the right upper triangular. This implies that D := RRe ;1 is a diagonal matrix. Moreover D has positive diagonal entries and e ;1 ) = (RRe ;1 );1 = D;1 . Thus D = I. This shows that D = D = (RR eR = R. Finally, we prove that R depends continuously on B. Let Be be a Hermitian positive matrix in Cj nn and Be = Re Re , where Re is upper triangular with positive diagonal entries. De ne H := Re ; R and K := Be ; B. Then H is an upper triangular matrix with real diagonal entries, K is a Hermitian matrix and H H + RH + HR = K: To show that H ! O if K ! O, we proceed as follows. Suppose that n = 1, H := [], R := [r] and K := [ ], where  2 IR, r > 0, r +  > 0 and 2 IR. Then 2 + 2r ; = 0;

319

6.1. QR FACTORIZATION p

and since  + r > 0,  = ;r + r2 + for all 2 IR such that r2 + > 0. This shows that  ! 0 if ! 0. Suppose now that n  2. For , and r in IR, a, b and c in Cj (n;1)1 , H0 , R0 and K0 in Cj (n;1)(n;1) , de ne H, R and K as follows. 











  c : H = 0 Ha ; R = 0r Rb ; K = c W 0 0 0

Then

2 + 2r ; = 0 and a + ra + b = c; p where  +r > 0. Hence for 2 IR such that r2 + > 0,  = ;r+ r2 + , as in the case n = 1. This shows that  ! 0 if ! 0. Also, a = c;+rb . Hence a ! 0 if ! 0 and c ! 0. Finally, H0 H0 + R0 H0 + H0 R0 = K0 := W0 ; aa ; ba ; ab ;

which allows us to complete the mathematical induction, since K0 is Hermitian and K0 ! O if K ! O. The matrix R in the preceding theorem is called the Cholesky factor of B. The Cholesky Factorization B = RR of B may be used to reduce a generalized eigenvalue problem Au = Bu, u 6= 0, where B is a Hermitian positive de nite matrix, to an ordinary eigenvalue problem as follows. As R is a nonsingular matrix, the generalized eigenvalue problem can be rewritten as the ordinary eigenvalue problem (R );1 AR;1 v = v. Note that v := Ru 6= 0 if and only if u 6= 0. If A is a Hermitian matrix, then (R );1 AR;1 continues to be a Hermitian matrix. Dierential properties of the QR factors and the Cholesky factor are studied in [51].

6.1.1 Householder symmetries

A Householder symmetry is a matrix of the form H := I ; vv ;

where

( 2 := v v if v 6= 0

0

if v = 0:

320

6. MATRIX COMPUTATIONS

A simple calculation shows that H is nonsingular and H;1 = H = H;

that is, H is Hermitian and unitary. The following result shows the utility of such a symmetry for calculating the QR factors of a full rank matrix.

Lemma 6.4 Let x := [x(1); : : :; x(n)]> 2 Cj n1 and k 2 [ 1; n ; 1 ] be such that there exists i 2 [ k + 1; n ] satisfying x(i) 6= 0. De ne v u n uX  := t jx(j )j2 j =k

and

8 < x( k ) s := :  jx(k)j if x(k) 6= 0;  if x(k) = 0:

Let v := [v(1); : : :; v(n)]> 2 Cj n1 be de ned by

8 for j 2 [ 1; k ; 1 ] ; <0 v(j ) := : x(k) + s for j = k; x(j ) for j 2 [ k + 1; n ] ;

and

:= s v1(k) :

Then H := I ; vv is a Householder symmetry and Hx =

kX ;1 j =1

x(j )ej ; sek :

Proof

From the de nition of v and s, we get n n X X  v v = v(j )v(j ) = (x(k) + s)(x(k) + s) + jx(j )j2 j =1

j =k+1

= (x(k) + s)(x(k) + s) + ss ; x(k)x(k) = 2s(s + x(k)) 6= 0: Hence

1 2 = = : v v s(s + x(k))

321

6.1. QR FACTORIZATION

This shows that H is a Householder symmetry. On the other hand, v x = (x(k) + s)x(k) +

n X

j =k+1

x(j )x(j )

= (x(k) + s)x(k) + jsj2 ; jx(k)j2 = (x(k) + s)s = (x(k) + s)s;  and hence v x = 1. Thus Hx =

=

kX ;1

j =1 kX ;1 j =1

[x(j ) ; 0]ej + [x(k) ; (x(k) + s)]ek +

n X k+1

[x(j ) ; x(j )]ej

x(j )ej ; sek ;

and the proof is complete.

Example 6.5 Computing QR factorizations: Let

2 2 A := 4 2

3

0 ;4 5 : i 9;i We rst compute the QR factors using the Gram-Schmidt Process. Let 2 3 2 3 2 0 x1 := 4 2 5 and x2 := 4 ;4 5 : i 9;i We compute q kx1 k = x1 x1 ; q1 = kxx1k ; eq2 = x2 ; x2 q1 q1 and q2 = keqeq2k : 1 2 Then 22 1+i 3 p 6 3 2 5 77 6 6  62 i ;17 7 2 q1   3 ;3(1p+ i )  k x k x 6 7 1 Q := [q1 q2 ]= 66 3 p 77 ; R = 0 keq k = 0 4 5 : 2 2 57 6 2

2

2

2

6 4i

3

7 2 5

p

5

2

322

6. MATRIX COMPUTATIONS

We remark that the Gram-Schmidt process is a sequential algorithm in the sense that no orthonormalized vector can be produced before the preceding ones are computed. This procedure is not convenient for large full matrices since the last columns of Q may fail to be orthogonal to the initial ones due to roundo errors, and hence Q will not be a unitary matrix. We now compute the QR factors using complex rotations. The classical real 22 planar rotations used to annihilate the second coordinate of a given 21 column can be extended to the complex case as follows:  

Let y := uv 2 Cj 21 be such that kyk 6= 0,  := kyuk and := ;v . Then the matrix U :=   ;  satis es Uy =  kyk . 0  kyk 2

2

2

2

Let A 2 Cj nm be a given matrix. If the column index j is xed, then we can annihilate the entries A(i; j ) for i 2 [ j + 1; n ] by using these `rotations'. It is easy to prove that if this process is carried out for a column index j 0 > j , then the entries A(i; j ) for i 2 [ j + 1; n ] are not modi ed. Consider the same matrix A as before. We obtain 2 1 p 6 2 6 6 ;1 U1 := 666 p 6 2 4

3

p1 0 7 2 7 7 7 1 p 0 77 2 75

0

0

and

1

2 4 p 6 2 6 6 U1 A := 6 6 0 6 6 4

i

3 2 77 7 7 ; 4 p 77 : 2 75

;4 p

9;i

Now we set A1 := U1 A. Then 2 4 p 0 6 3 2 6 6 U2 := 666 0 1 6 4 ;i

p

2

0

3 3 77 7 0 777 7 4 5

;i

p

3 2

and

2 3 6 6 60 U2 A1 := 6 6 6 6 4

0

;3(1 + i ) 3 7 7 ; 4 p 77 2 p12 2

: 7 7 7 5

323

6.1. QR FACTORIZATION

Finally, we set A2 := U2 A1 and we have 2 1 6 6 6 U3 := 666 0 6 4

0

0 3

7 7 ; 1 3 p p 777 10 10 7 7 ;3 1 5

0 p

10

p

and

2 3 6 6 U3 A2 := 6 60 4

0

10

;3(1 + i ) 3 p 777

4 5 7: 5 0

We recognize the upper triangular factor R as the submatrix 2 3 R := 4

0

;3(1 + i ) 3 p 5 4 5

of U3 A2 and note that the unitary factor Q consists of the rst two columns of (U3 U2 U1 ) = U1 U2 U3 . We remark that in general, each `rotation' contributes with one zero entry in the construction of the upper triangular factor. Next, let us construct a QR factorization of a matrix A 2 Cj nm such that rank A = m, where n  m  2, using Householder symmetries. Let k := 1 and x := A(
; k) in Lemma 6.4. If x satis es the hypothesis of the Lemma, then de ne H1 to be the Householder symmetry H given in the statement of the Lemma, and let H1 := I otherwise. Set A1 := H1 A. Then A1 (i; 1) = 0 for each i 2 [ 2; n ] . If n = m = 2, then R := A1 and Q := H1 provide a QR factorization of A. Suppose that m  3, and for some ` 2 [ 1; m ; 2 ] and each j 2 [ 1; ` ] , a matrix A` has been constructed such that A` (i; j ) = 0 for each i 2 [ j + 1; n ] . Let k := ` + 1 and x := A` (
; k) in Lemma 6.4. If x satis es the hypothesis of the Lemma, then de ne H`+1 to be the Householder symmetry H given in the statement of the Lemma, and let H`+1 := I otherwise. Set A`+1 := H`+1 A. Then A`+1 (i; j ) = 0 for each j 2 [ 1; ` + 1 ] and for each i 2 [ j + 1; n ] . This means that  

R ; An;1 := O

where R 2 Cj mm is a nonsingular upper triangular matrix. If Q is de ned to be the matrix formed by the rst m columns of H1


Hn;1 , then A = QR.

324

6. MATRIX COMPUTATIONS

Consider the same matrix A as before. Let the computations be carried out with 7 decimal gures. We obtain 2 ;(:6666667)100 H1 := 4 ;(:6666667)100

3 (:7333333)100 (:1333333)100i 5 ;

;(:6666667)100 (:3333333)100i

;(:3333333)100i ;(:1333333)100i (:9333333)100

2 ;(:3)101 A1 := 4 0

3

(:30)101 + (:30)101i (:28)101 + (:12)101i 5 ; 0 (:84)101 ; (:40)100i 2 3 (:1)101 0 0 H2 := 4 0 ;(:3405877)100 ;(:8455971)100 5 0 ;(:8455971)100 (:3405877)100 2 3 0 0 0 0 ;(:4110541)100i 5 ; +40 0 0 (:4110541)10 i 0 2 3 1 1 ;(:3)10 (:3000000)10 + (:3000000)101i ;(:8221083)101 ; (:3523321)101i 5 ; A2 := 4 0 0 (a + bi )
10;7 p

where (:22)101  jaj2 + jbj2  (:25)101. We recognize the upper triangular factor R as the submatrix 

1 R := ;(:3)10 0

(:3000000)101 + (:3000000)101i ;(:8221083)101 ; (:3523321)101i



of A2 . The presence of the `spurious' value A2 (3; 2) = (a + bi )
10;7 which should be zero will be explained in Subsection 6.4.2. The unitary factor Q consists of the rst two columns of H1 H2 . We remark that Hj contributes with n ; j zeros in constructing the upper triangular factor, and that the diagonal entry R(2; 2) is not a real number.

6.1.2 Hessenberg Matrices

A matrix A 2 Cj nn is said to be an upper Hessenberg matrix if A(i; j ) = 0 when i > j + 1. A matrix A 2 Cj nn is said to be a lower Hessenberg matrix if A is an upper Hessenberg matrix. We say that an upper Hessenberg matrix is irreducible if A(j + 1; j ) 6= 0 for j 2 [ 1; n ; 1 ] . nP ;1 Consider a polynomial p(t) := a0 + aj tj + tn . The irreducible j =1

325

6.1. QR FACTORIZATION

upper Hessenberg matrix

2 0 6 A := 6 6I 4 n;1

;a0 ;a1 .. .

;an;1

3 7 7 7 5

is called the companion matrix of the polynomial p. It can be proved that p is the characteristic polynomial of A. (See [43].)

Proposition 6.6

Every irreducible upper Hessenberg matrix is similar to the companion matrix of its characteristic polynomial.

Proof

Let A be an irreducible upper Hessenberg matrix. Then rank(A)  n ; 1. If  is any eigenvalue of A, then dim N (A ; I) = 1 since A ; I is also an irreducible upper Hessenberg matrix. Let A : Cj n1 ! Cj n1 be the linear operator represented by A relative to the standard basis for Cj n1 . Set u1 = e1 and ui+1 = Aui for i 2 [ 1; n ; 1 ] . Since ui belongs to spanfej : j 2 [ 1; i ] g n spanfej : j 2 [ 1; i ; 1 ] g for i 2 [ 1; n ] , the ordered set [u1 ; : : :; un ] forms an ordered basis for Cj n1 . The matrix representing A relative to this basis is the companion matrix of the characteristic polynomial of A.

Proposition 6.7 For A 2 Cj nn , there are a unitary matrix Q and an upper Hessenberg matrix A0 such that

A0 = Q AQ;

so that every matrix A 2 Cj nn is unitarily similar to an upper Hessenberg matrix.

Proof nn Let A 2 Cj . We set A(2; 1) w2 := [0; A(2; 1); : : :; A(n; 1)]> ; 2 := jA(2; 1)j kw2 k ; v2 := 2 e2 2

326

6. MATRIX COMPUTATIONS

and the Householder symmetry

v v H2 := I ; 2 2 2 : v2 v2 It is easy to verify that H2 v2 = w2 and H2 e1 = e1 : Hence   ea>  H2 AH2 = w2 + A(1; 1)e1 A ; 2

where ea 2 Cj (n;1)1 and A2 2 Cj (n;1)(n;1) . Now we suppose that for some p 2 [ 1; n ; 1 ] ; there is a unitary matrix Up such that Up AUp is partitioned as follows: 2 3 B(p) Ae 6 7 0 : : : 0 p+1 6 7 6  Up AUp = 66 0 : : : 0 0 (p+1) 7 ; 7 7 A . . . . . . . . 4. .. . 5

0 ::: 0 0

where B(p) is an upper Hessenberg matrix in Cj pp and Ae and A(p+1) respectively belong to Cj p(n;p) and Cj (n;p)(n;p) : If A(p+1) is not an upper Hessenberg matrix, then we set wp>+2 := [ 0| ; :{z: :; 0} ; A(p + 2; p + 1); : : :; A(n; p + 1)]> ; p+1 times

+ 2; p + 1) kw k ; p+2 := jAA((pp + 2; p + 1)j p+2 2

vp+2 := p+2 ep+2 ;

and It is clear that

v v Hp+2 := I ; 2 p+2 p+2 : vp+2 vp+2

3 2 ee (p+1) A B 7 6 7 6 0 : : : 0 p+2 7 6 7 0 : : : 0 0 Hp+2 Up A(Hp+2 Up ) = 6 7; 6 6 .. . . .. .. A(p+2) 7 5 4. .. .

0 ::: 0 0

6.2. CONVERGENCE OF A SEQUENCE OF SUBSPACES

where

327

3

2 6 6 6 ( p +1) B := 666 6 4

B(p)

Ae (1; p + 1) 7 .. 7 . 7 e A(p; p + 1) 7 7 7 7 5

0 : : : 0 p+1 A(p+1) (1; 1) is an upper Hessenberg matrix. By nite induction, there is a unitary matrix Q such that A0 := QAQ is an upper Hessenberg matrix.

Proposition 6.8

Let A be a nonsingular irreducible upper Hessenberg complex matrix. Let A = QR be a QR factorization of A and let B = RQ. Then Q and B are nonsingular irreducible upper Hessenberg matrices.

Proof

Since A and Q are nonsingular matrices, R is a nonsingular matrix and R;1 is an upper triangular matrix with nonzero diagonal entries. Let n be the order of A. Since Q = AR;1 , for each j 2 [ 1; n ; 1 ] , the j th column of Q is a linear combination of the rst j columns of A such that the scalar multiplying the j th column of A is nonzero. Hence Q is a nonsingular irreducible upper Hessenberg matrix. Also, for each j 2 [ 1; n ; 1 ] , the j th column of B is a linear combination of the rst j + 1 columns of R such that the scalar multiplying the (j + 1)th column of R is nonzero. Hence B is an irreducible upper Hessenberg matrix. B is nonsingular because Q and R are.

6.2 Convergence of a Sequence of Subspaces 6.2.1 Basic De nitions

Note that for a sequence (Ak ) in Cj mn and A 2 Cj mn , kAk ; Ak ! 0 if and only if Ak (i; j ) ! A(i; j ) for every i 2 [ 1; m ] and every j 2 [ 1; n ] : 2

328

6. MATRIX COMPUTATIONS

Lemma 6.9

Let (Rk ) be a sequence of upper triangular matrices in Cj mm with positive diagonal entries. If kRk Rk ; Ik ! 0, then kRk ; Ik ! 0. 2

2

Proof

Let Bk := Rk Rk and B := I = I I. The conclusion follows from the continuity of the Cholesky factor established in Theorem 6.3. In applications, we frequently need to approximate spectral subspaces of operators. Dierent de nitions of the convergence of a sequence of subspaces have been proposed. (See for instance [60].) We shall adopt the following de nition. We denote by M the set of all linear subspaces of Cj n1 . Let (M1 ; M2) be a pair of elements of M, and 1 (resp. 2 ) denote the matrix representing the orthogonal projection on M1 (resp. M2 ) relative to the standard basis for Cj n1 . We de ne

d(M1 ; M2 ) := k1 ; 2 k : 2

Remark 6.10

It can be easily proved that d is a metric on M, and that d(M1 ; M2 ) = gap(M1 ; M2 ). (See Remark 2.16.) This statement is valid for closed subspaces M1 and M2 of any Hilbert space. (See page 198 of [47].) Since it is dicult to compute d(M1 ; M2 ), we give some equivalent characterizations of the convergence of a sequence of subspaces in terms of ordered bases. Let M 2 M. An ordered basis for M is represented by a matrix U whose columns contain the coordinates of the corresponding vectors relative to the standard basis for Cj n1 . If m := dim M , then U belongs to Cj nm .

Proposition 6.11

Let (Mk )k1 be a sequence which converges to M in M. Then

dim Mk = dim M

Proof

for all large enough k:

Let  (resp. k ) denote the matrix representing the orthogonal projec-

6.2. CONVERGENCE OF A SEQUENCE OF SUBSPACES

329

tion onto Mk (resp. M ) in the standard basis for Cj n1 . Since (Mk ) converges to M , d(Mk ; M ) := kk ; k < 1 for all large enough k. Hence (k ; ) < 1 for all large enough k, and by Lemma 2.11, dim Mk = dim M . 2

Theorem 6.12

The following statements are equivalent:

(i) The sequence of subspaces (Mk ) converges to a subspace M . (ii) Given any orthonormal basis for M , let Q be its matrix representation in the standard basis for Cj n1 . Then there is a sequence (Qk ) such that Qk represents an orthonormal basis for Mk in the standard basis for Cj n1 , and lim kQ ; Qk = 0: k!1 k 2

(iii) Given any orthonormal basis for M , let Q be the matrix representing it in the standard basis for Cj n1 . Then there exists a matrix Y such that [Q; Y] represents an ordered basis for Cj n1 in its standard basis, and there exist matrices Xk 2 Cj nm , Ck 2 Cj mm , Dk 2 Cj (n;m)m such that (a) Xk represents an ordered basis for Mk in the standard basis for Cj n1 for all large enough k, (b) Ck is invertible for all large enough k, (c) Xk = QCk + YDk , and (d) kDk C;k 1 k ! 0. (iv) For each sequence (Qk ) in Cj nm such that Qk represents an orthonormal basis for Mk in the standard basis for Cj n1 , there is a sequence (Q[k] ) in Cj nm such that Q[k] represents an orthonormal basis for M in the standard basis for Cj n1 , and lim kQ ; Q[k] k = 0: k!1 k 2

2

Proof

(i) implies (ii): Suppose that Mk ! M . Let Q represent an orthonormal basis for M . Then Xk := k Q ! Q = Q. Hence Xk Xk ! Q Q = I. Let

330

6. MATRIX COMPUTATIONS

Qk := Q(Xk ) and Rk := R(Xk ) be the unitary and upper triangular factors of Xk , respectively. Then Rk Rk ! I and by Lemma 6.9, Rk ! I. Hence R;k 1 ! I and kQk ; Qk = kXk R;k 1 ; Qk  kXk ; Qk kR;k 1 k + kQk kR;k 1 ; Ik ! 0: (ii) implies (iii): Suppose that for each matrix Q representing an orthonormal basis for M , there exists a matrix Qk representing an orthonormal basis for Mk such that kQk ; Qk ! 0. Let Y 2 Cj (n;m)n represent an orthonormal basis for M ? . Then [Q; Y] represents an orthonormal basis for Cj n1 , and there exist matrices Ck 2 Cj mm and Dk 2 Cj (n;m)m such that Qk = QCk + YDk . Since Q Y = O and Y Y = I, then Q Qk = Ck and Y Qk = Dk . Thus kCk ; Ik  kQk ; Qk ! 0 and kDk k = kYQk k ! 0. This shows that Ck is nonsingular for all large enough k and that kC;k 1 k ! 1. We conclude that kDk C;k 1 k ! 0. (iii) implies (iv): Let Qk represent an orthonormal basis for Mk , Q represent an orthonormal basis for M , Y be such that [Q; Y] represents a basis for Cj n1 , Ck and Dk be such that Xk := QCk + YDk represents a basis for Mk , and kDk C;k 1 k ! 0 as k ! 1. There exists a nonsingular matrix Bk such that Qk Bk = Xk C;k 1 = Q + YDk C;k 1 and hence kQk Bk ; Qk ! 0 and kBk Bk ; Ik ! 0 as k ! 1. Let Bk = Q(Bk )R(Bk ) be the QR factorization of Bk . Set Rk := R(Bk ). Then by Lemma 6.9, kRk Rk ; Ik ! 0 and kR;k 1 ; Ik ! 0 as k ! 1. From the identity Qk ; QB;k 1 = (Qk Bk ; Q)B;k 1 = (Qk Bk ; Q)R;k 1 Q(Bk ); we conclude that kQk ; QB;k 1 k ! 0 as k ! 1. Now, let R[k] := R(QB;k 1 ) and Q[k] := Q(QB;k 1 ) be the QR factors of QB;k 1 . We can prove, as before, that kR[k] ; Ik ! 0 and hence kQk ; Q[k] k ! 0 as k ! 1. (iv) implies (i): Suppose that for each matrix Qk representing an orthonormal basis for Mk , there exists a matrix Q[k] representing an orthonormal basis for M , such that kQk ; Q[k] k ! 0. The matrices of the orthogonal projections onto Mk and M can be computed as k = Qk Qk and  = Q[k] Q[k] , respectively. Hence kk ; k  k(Qk ; Q[k] )Qk k + kQ[k](Qk ; Q[k] )k  2kQk ; Q[k] k ! 0: 2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

6.2. CONVERGENCE OF A SEQUENCE OF SUBSPACES

331

This proves that Mk ! M .

Theorem 6.13

Let S be a proper subspace of M and Sk be a proper subspace of Mk . Suppose that Sk ! S and Mk ! M as k ! 1. Let Uk := [Xk ; Yk ] represent an orthonormal basis for Mk in the standard basis for Cj n1 , such that Xk represents an orthonormal basis for Sk in the standard basis for Cj n1 . Then there exists a matrix U[k] := [X[k] ; Y[k] ] representing an orthonormal basis for M in the standard basis for Cj n1 , such that X[k] represents an orthonormal basis for S in the standard basis for Cj n1 , and kUk ; U[k] k2 ! 0 as k ! 1.

Proof

Let S (resp. M ) be the matrix representing the orthogonal projection onto S (resp. M ). Then kS Xk ; Xk k2 ! 0 and kM Yk ; Yk k2 ! 0. Hence, for large enough k, Z[k] := [S Xk ; M Yk ] represents a basis for M and S Xk represents a basis for S . Let U[k] := Q(Z[k] ) and R[k] := R(Z[k] ) be the QR factors of Z[k] . Then U[k] = [X[k] ; Y[k] ] represents an orthonormal basis for M such that X[k] represents an orthonormal basis for S . Since kZ[k] ; Uk k2 ! 0, we conclude that kR[k]R[k] ; Ik2 ! 0 and by Lemma 6.9, kR[k] ; Ik2 ! 0. Hence kU[k] ; Uk k2 ! 0.

6.2.2 Krylov Sequences

Let A : Cj n1 ! Cj n1 be a linear map, A 2 Cj nn be the matrix representing A in the standard basis for Cj n1 and M0 be a linear subspace of Cj n1 . The sequence of subspaces (Mk )k1 de ned by

Mk := A(Mk;1 ) is called the Krylov sequence corresponding to the pair (A; M0 ). We shall prove that under suitable hypotheses on A and M0 , the Krylov sequence converges to an invariant subspace of A.

Theorem 6.14

Assume that the eigenvalues i , i 2 [ 1; n ] , of A satisfy:

(H1)

j1 j 


 jm j > jm+1 j 


 jn j > 0;

332

6. MATRIX COMPUTATIONS

for some integer m  n ; 1. We set

M :=

m M i=1

N (A ; i I ):

Let P be the projection onto M along

N :=

n M

i=m+1

N (A ; i I ):

Let M0 be an m-dimensional subspace of Cj n1 such that (H2) dim(P (M0 )) = m: Then the Krylov sequence corresponding to the pair (A; M0 ) converges to M .

Proof

Let X0 represent a basis for M0 , Q an orthonormal basis for M , and Y an orthonormal basis for N . Then [Q; Y] forms a basis for Cj n1 and hence there exist C 2 Cj mm and D 2 Cj (n;m)m such that X0 = QC + YD: Let P be the matrix representing P in the standard basis for Cj n1 . Hypothesis (H2) implies that C is nonsingular since PY = O and PX0 represents a basis for M . Since M and N are invariant subspaces under A, there exist  2 Cj mm and Z 2 Cj (n;m)(n;m) such that AQ = Q and AY = YZ: Recursively, we get Ak X0 = Qk C + YZk D: De ne Xk := Ak X0 , Ck := k C and Dk := Zk D. Then conditions (a), (b) and (c) of the assertion (iii) of Theorem 6.12 are satis ed. We now prove that condition (d) is also satis ed. The matrix  represents the restriction of A to the spectral subspace associated with the subset of eigenvalues f1 ; : : :; m g in the orthonormal basis for this subspace represented by Q. Also, Z represents the restriction of A to the spectral subspace associated with the subset of eigenvalues fm+1 ; : : :; n g in the orthonormal basis for this subspace represented by Y. Hence (;1 ) = 1 ; (Z) = j j and (;1 )(Z) = jm+1 j < 1:

jm j

m+1

jm j

6.2. CONVERGENCE OF A SEQUENCE OF SUBSPACES

333

On the other hand,

kDk C;k 1 k2  kZk k2 kDk2 kC;1 k2 k;k k2 : ;1 For  := 1 ; (2 )(Z) > 0, there exists an integer k0 such that for all ;1 k > k0 , k;k k12 =k kZk k12 =k <  + (;1 )(Z) = 1 + (2 )(Z) < 1, so that  k ;1 k;k k2 kZk k2 < 1 + (2 )(Z) ! 0

as k ! 1. Thus condition (d) is satis ed. Hence the Krylov sequence corresponding to the pair (A; M0 ) converges to M .

Remark 6.15

Let us regard the rate of convergence to 0 of k;k k2 kZk k2 to be the rate of convergence of the Krylov sequence. The proof of Theorem 6.14 shows that this rate can be arbitrarily close to jjm+1j j since  can be m chosen arbitrarily close to 0. This implies that if we apply the method to A ; I instead of A, then a suitable choice of  2 Cj can substantially improve the rate of convergence. Such a parameter  is called a shift. The optimal shift, if it exists, would be the solution of the following optimization problem: Find b 2 Cj such that

jm+1 ; bj = min  jm+1 ; j :  2 Cj  : jm ; bj jm ; j

However, since the eigenvalues f1 ; : : : ; n g, are not known, it is not possible to determine b.

Remark 6.16 The Power Method:

The most elementary application of the convergence of a Krylov sequence is the case m = 1 in Theorem 6.14. Let z0 2 Cj n1 be such that z0 6= 0 and  be a given positive number. The Power Method consists of the following algorithm:

334

6. MATRIX COMPUTATIONS

Begin read(,MaxIter,z0 ); z z := 0 kz0 k2 k:=1; while Az 6= 0, k<MaxIter and k(I ; zz )Azk2 >  do begin Az z:= kAzk2 ; k:=k+1 end; End. The vector (I ; zz )Az is the residual of z as an approximation to an eigenvector of A corresponding to an eigenvalue which is approximated by zAz. Thus the stopping criterion of this algorithm is based on a maximal residual tolerance:

k(I ; zz )Azk2  :

We remark that the sequence of vectors produced by the Power Method may not speci c eigenvector of A. For example, if  itself  converge toa  ; 1 0 1 A := 0 0 and z0 := 0 , then the kth iterate is given by zk = (;1)k z0 and is an eigenvector of A corresponding to the eigenvalue  = ;1. A nonzero vector y is called a left eigenvector of A corresponding to an eigenvalue  of A if y A = y , that is, y is an eigenvector of A corresponding to the eigenvalue  of A . Let us assume that the eigenvalues i of A satisfy the hypothesis (H1) of Theorem 6.14 for m = 1:

j1 j > j2 j  j3 j  : : :  jn j > 0: The hypothesis (H2) of Theorem 6.14 now reads as follows: M0 is spanned by a vector z0 such that y z0 6= 0 for any left eigenvector y corresponding to 1 . If u1 is a unit eigenvector of A corresponding to 1 , then 1 = u1 Au1 . According to Theorem 1.2, there is a unitary matrix U 2 Cj nn such that    v 1  U AU = 0 B for some v 2 Cj (n;1)1 and B 2 Cj (n;1)(n;1) .

6.2. CONVERGENCE OF A SEQUENCE OF SUBSPACES

335

A left eigenvector y of A corresponding to 1 satis es 8    ; where < and6= 0 U y = w : w is such that v + B w = 1 w: The eigenvalues of B are 2 ; : : : ; n . Since dim N (A ; 1 I) = 1, our hypothesis implies that c := u1 z0 6= 0. Hence the coordinates of z0 relative to the basis U are vc and 2

nX ;1

3

i  n;1;i v 7 n UAn U vc = 64 1 c + i=0 1 v B 5: n B v  

This procedure is known as the Inverse Iteration Algorithm when it is applied to the inverse of the shifted matrix A ; I, where  2= sp(A) is an approximate eigenvalue of A. The rate of convergence of the Power Method can be arbitrarily close to j2 j=j1 j. It may be very poor, if j2 j is close to j1 j.

Example 6.17 Actual rate of convergence of the Power Method: Let 2 3 133:21 152:07 ;165:05 A := 4 ;96:15 ;110:05 117:75 5 : 20:03 23:01 ;24:15

This matrix is constructed in such a way that its eigenvalues are  := 1:01 (which is a simple eigenvalue), and ;1 (which is a double defective eigenvalue). The Power Method will give a sequence of one-dimensional subspaces converging to the eigenspace corresponding to  and the expected rate of convergence is arbitrarily close to r(A) := 1:101 = 0:9900(9900) : : : The shifted matrix A0 := A + 1:01I has the same spectral subspaces as A. If the Power Method is applied to A0 , then the expected rate of convergence is arbitrarily close to r(A0 ) := 02::01 02 = 0:00495:

336

6. MATRIX COMPUTATIONS

When we run the algorithm on a computer, starting with z0 := e1 , we observe the following behavior: Iteration sin  sin 0 ; 2 1 (:745)10 (:518)10;2 2 (:901)10;2 (:159)10;3 3 (:733)10;2 (:121)10;5 4 (:823)10;2 0 .. .. . . 1800 (:997)10;6 where  (resp. 0 ) is the acute angle between the approximate eigenspace and the exact eigenspace when the method is applied to A (resp. A0 ). It can be shown that the sinus of this angle is equal to the gap between the subspaces. (See Remark 2.16 and [26].) Thus if we apply the Power Method to A, we need 1800 iterations to produce an approximate subspace with a gap less than 10;6, while if we apply it to A0 , we only need 4 iterations to achieve this.

6.3 QR Methods and Inverse Iteration

6.3.1 The Francis-Kublanovskaya QR Method

There are many algorithms to approximate the eigenvalues and eigenvectors of matrices. One of the most classical ones is the Basic QR Method: Begin read(A); A0 := an upper Hessenberg matrix unitarily similar to A; k := 1; while Not(Stopping Criterion) do begin Ak;1 := Qk Rk % some QR factorization of Ak;1 Ak := Rk Qk k := k + 1 end; End.

6.3. QR METHODS AND INVERSE ITERATION

337

Before presenting the stopping criterion for this method as well as some of the main variants of the algorithm, we shall give a convergence result for the basic version:

Theorem 6.18 Convergence of the Basic QR Method: Let A0 2 Cj nn be a nonsingular irreducible upper Hessenberg matrix. Suppose that its eigenvalues, denoted by j , satisfy

j1 j > j2 j > : : : > jn;1 j > jn j > 0:

Let Ak , Qk and Rk be the matrices de ned in the Basic QR Method. De ne Uk := Q1 Q2


Qk ; and Tk := Rk Rk;1


R1 : For each j 2 [ 1; n ] and each positive integer k, let Mk (j ) denote the subspace of Cj n1 spanned by the rst j columns of Ak0 and set

M (j ) :=

j M i=1

N (A0 ; i I):

Then the following results hold: (a) For each positive integer k, Qk and Ak are irreducible upper Hessenberg matrices and Ak = Uk A0 Uk ; (b) for each positive integer k, Uk Tk = Ak0 and for each j 2 [ 1; n ] , the rst j columns of Uk form an orthonormal basis for Mk (j ); (c) for each j 2 [ 1; n ] , the sequence (Mk (j ))k1 converges to M (j ); (d) for each positive integer k, there exists a unitary matrix U[k] such that for each j 2 [ 1; n ] , the rst j columns of U[k] span M (j ), and lim kU ; Uk kF = 0; k!1 [k]

(e) for each positive integer k, the matrix A[k] := U[k] A0 U[k] is upper triangular, its diagonal coecients are the eigenvalues of A0 arranged in the decreasing order of moduli and lim kA ; Ak kF = 0; k!1 [k]

338

6. MATRIX COMPUTATIONS

(f) the rate of convergence of the Basic QR Method is arbitrarily close to o n r := max jji+1j j : i 2 [ 1; n ; 1 ] : i Proof (a) By Proposition 6.8, if Ak is a nonsingular irreducible upper Hessenberg matrix, then so are Qk and Ak . Since for each positive integer k, Ak = Qk Ak;1 Qk , we obtain Ak = Uk A0 Uk . (b) For each k  1, Uk Tk = Q1


Qk;1 Qk Rk Rk;1


R1 : Replacing the product Qk Rk by Rk;1 Qk;1 , we obtain Uk Tk = Q1


Qk;2 Qk;1 Rk;1 Qk;1 Rk;1 Rk;2


R1

and now the product Qk;1 Rk;1 appears twice. Replacing it by Rk;2 Qk;2 , we obtain Uk Tk = Q1


Qk;2 Rk;2 Qk;2 Rk;2 Qk;2 Rk;2


R1

and now the product Qk;2 Rk;2 appears three times. Continuing in this way, the only product present in Uk Tk will be Q1 R1 and it will appear k times. But Q1 R1 = A0 . Thus Uk Tk = Ak0 . (c) Let A0 : Cj n1 ! Cj n1 be the linear operatornrepresented by the matrix A0 with respect to the standard basis for Cj 1 . Result (b) means that the Basic QR Method produces a set of n nested Krylov sequences corresponding to (A0 ; M0 (j )), j 2 [ 1; n ] , where

M0(j ) := span [e1 ; : : :; ej ]; that is, for each j 2 [ 1; n ] , M0 (j ) is the j dimensional subspace spanned by the rst j vectors in the standard basis for Cj n1 . The hypothesis on the eigenvalues of A0 implies that hypothesis (H1) of Theorem 6.14 is satis ed for each j 2 [ 1; n ; 1 ] . We shall prove that hypothesis (H2) of Theorem 6.14 is satis ed for each j 2 [ 1; n ] . For this purpose, we recall that for any linear space X , any subspace M of X , and any linear map P : X ! X , N (P jM ) = M \ N (P ); also, if M is nite dimensional, then the restriction P jM : M ! X is injective if and only if rank(P jM ) = dim M . These facts imply that if P (j ) denotes the

6.3. QR METHODS AND INVERSE ITERATION

339

spectral projection onto M (j ), then hypothesis (H2) of Theorem 6.14 is equivalent to

N (P (j )) \ M0 (j ) = f0g for each j 2 [ 1; n ] : Suppose this is not the case. Then for some j and some ` such that 1  `  j , one should have e` 2 N := N (P (j )) = R(I ; P (j )). Since N is invariant under A0 , the vectors A0 e` ; A20 e` ; : : :; An0 ;` e` must belong to N . But since A0 is bijective, all these vectors are linearly independent and hence dim N  n;`+1. As dim N = n;j , we obtain a contradiction `  j + 1. The convergence of the sequence (Mk (j ))k1 to the subspace M (j ) for each j 2 [ 1; n ] follows from Theorem 6.14. (d) We apply Theorem 6.13 recursively: First consider the pair of subspaces M := M (n) = Cj n1 and S := M (n ; 1) which are the limits of Mk := Mk (n) and Sk := Mk (n ; 1), respectively. Then let M := M (n ; 1) and S := M (n ; 2) which are the limits of Mk := Mk (n ; 1) and Sk := Mk (n ; 2), respectively. Finally, let M := M (2) and S := M (1) which are the limits of Mk := Mk (2) and Sk := Mk (1), respectively. (e) For each j 2 [ 1; n ] , the rst j columns of U[k] form an ordered basis for M (j ). Since M (j ) is invariant under A0 , we conclude that each of the rst j columns of A0 U[k] is a linear combination of (only) the rst j columns of U[k] , that is, there exists an upper triangular matrix A[k] such that A0 U[k] = U[k] A[k] . Since U[k] is a unitary matrix, the upper triangular matrix A[k] is unitarily similar to A0 and has the eigenvalues of A0 along its diagonal. The order in which the eigenvalues appear along the diagonal of A[k] follows from the order of the columns of U[k] : for each j 2 [ 1; n ] , the rst j columns form an ordered basis for M (j ), the spectral subspace associated with the subset of eigenvalues f1 ; : : :; j g of A0 , where j1 j >


> jj j. Now, for k  2, Ak is unitarily similar to A0 since Ak = Uk A0 Uk . We thus have A[k] ; Ak = (U[k] ; Uk )A0 U[k] + Uk A0 (U[k] ; Uk ):

p

Since kU[k] kF = kUk kF = n (because U[k] and Uk are unitary matrices), we conclude that kA[k] ; Ak kF tends to 0 as k tends to in nity. (f) Since the n Krylov sequences on which the Basic QR Method is based are nested, Remark 6.15 gives the rate of convergence of the method.

340

6. MATRIX COMPUTATIONS

Note that all that can be said about the superdiagonal entries of Ak is that they remain bounded in k, since kAk kF = kA0 kF for all k. This means that the sequence (Ak ) may not converge to a speci c matrix. For the same reasons which motivated shifting in the Power Method, we may consider shifting the Basic QR Method at each step. The Single Shift QR Method reads as follows: Begin read(A); A0 := an upper Hessenberg matrix similar to A k := 1; while Not(Stopping criterion) do begin Choice of the shift k Ae k;1 := Ak;1 ; k I Ae k;1 := Qe k Re k % some QR factorization of Ae k;1 e e Ak := Rk Qk + k I k := k + 1 end; End.

Example 6.19 Usual single shifts for the QR Method: Rayleigh's shift:

k := Ak;1 (n; n);

Wilkinson's shift:

k := ek;1 ;

where ek;1 is the eigenvalue of   A ( n ; 1 ; n ; 1) A ( n ; 1 ; n ) k ; 1 k ; 1 Bk;1 := A (n; n ; 1) Ak;1 (n; n) k ;1 which is closest to Ak;1 (n; n). These shifts lead, under certain hypotheses, to a quadratic convergence of the Single Shift QR Method. (See [74], [63], [37] and [71].) However, the Single Shift QR Method is almost never used. To complete this subsection we present an implicit Double Shift QR Method.

Lemma 6.20

Let B be an upper Hessenberg matrix and p a polynomial none of whose

341

6.3. QR METHODS AND INVERSE ITERATION

roots is an eigenvalue of B. Then there is a unitary matrix Q such that Q p(B) is an upper triangular matrix and Q BQ is an upper Hessenberg matrix.

Proof

d Q

Suppose p(t) := a (t ; i ). De ne B1 := B. Let Q1 R1 be a QR i=1 factorization of B1 ; 1 I and let B2 := Q1 B1 Q1 . We obtain p(B) = aQ1 Q1 Q1

d Y

(B1 ; i I)Q1 (B1 1 I) = aQ1

i=2

d Y

i=2

(B2 ; i I)R1 ;

and B2 = Q1 [(B1 ; 1 I) + 1 I]Q1 = R1 Q1 + 1 I. Suppose that we have constructed a unitary matrix Qj and an upper triangular matrix Rj such that Bj ; j I = Qj Rj for j 2 [ 1; ` ] . Let Bj+1 := Rj Qj and Vj := Q1


Qj . Then Bj+1 = Qj (Qj Rj + j I)Qj = Qj Bj Qj = Vj B1 Vj ; and p(B) = a

d Y

(B ; i I)

V`;k V`;k (B ; `;k I)

k=0 `Y ;1 =a (B ; i I)V` Q`;k (B`;k ; `;k I) i=`+1 k=0 d `Y ;1 Y =a (B ; i I)V` R`;k : i=`+1 k=0

Hence p(B) = a =a

i=`+1 d Y

`Y ;1

d Y

`Y ;1

(B ; i I)V` [V` (B ; `+1 I)V` ]

i=`+2 d Y

`Y ;1

i=`+2

k=0

(B ; i I)V` (B`+1 ; `+1 I)

k=0

R`;k

R`;k :

The QR factorization B`+1 ; `+1 I = Q`+1 R`+1 concludes the nite induction. Setting Q := Q1 Q2


Qd and R := RdRd;1


R1 ;

342

6. MATRIX COMPUTATIONS

we obtain a QR factorization for p(B): p(B) = QR. The QR factorization of p(B) exhibited in Lemma 6.20 is called an explicit multiple iteration of order d. It involves the QR factorization of d matrices and needs the knowledge of the roots of p. However, the following theorem shows that an implicit multiple iteration is possible, that is to say, QR factors of p(B) can be computed implicitly, which amounts to using multiple shifts implicitly.

Theorem 6.21 Implicit Multiple Shift QR Theorem: Let A be an nn matrix. Let Q and Qe be unitary matrices such that A0 := Q AQ and Ae 0 := Qe AQe are both irreducible upper Hessenberg e 1 are linearly dependent, then there is a unitary matrices. If Qe1 and Qe e . diagonal matrix D such that Q = QD

Proof

e 1 are linearly dependent, there is 1 2 IR such that Qe1 = If Qe1 and Qe e 1 . Suppose there exists an integer p; 1  p < n; and p real ei 1 Qe e j : Since A = QA0 Q = numbers j , j 2 [ 1; p ] , such that Qej = ei j Qe e p = Q eA e 0 ep and therefore Qe Ae 0 Qe  , we have AQep = QA0 ep = ei p AQe pP +1 pP +1 e j : According to our hypothesis, we A0 (j; p)Qej = ei p Ae 0 (j; p)Qe

j =1

j =1

e p+1 , and since A0 (p + 1; p) get A0 (p + 1; p)Qep+1 = ei p Ae 0 (p + 1; p)Qe and Ae 0 (p + 1; p) are nonzero, there exists p+1 2 IR such that Qep+1 = e p+1 . By nite induction, the proof is complete. ei p+1 Qe

One of the advantages of the Implicit Double Shift QR Method is that when A is real, one complex shift and its conjugate can be employed without using complex arithmetic computations.

6.3.2 Simultaneous Inverse Iteration Method

The approximate eigenvalues produced by the QR Method can be used as shifts for the Simultaneous Inverse Iteration Method which we now describe. Let  2 Cj be an approximate eigenvalue of A in the sense that there exists an eigenvalue  of A such that 0 < j ; j < minfj ; j :  2 sp(A);  6= g:

6.3. QR METHODS AND INVERSE ITERATION

343

If the algebraic multiplicity of  is m,  may be de ned as the arithmetic mean of the cluster of approximations of  produced by the QR Method. Let Z0 be a given matrix in Cj nm of rank m and  a given positive number. The Simultaneous Inverse Iteration Method consists of the following algorithm: Begin read(,MaxIter,Z0 ); Z := Z0 k:=1; while k<MaxIter and k(I ; ZZ)AZkF >  do begin Solve (A ; I)Y = Z; Z:=Orthonormalization of Y; k:=k+1 end; End. The orthonormalization step may be accomplished by a QR factorization of Y. We emphasize that the iterations are stopped with a residual test.

Remark 6.22

The preceding algorithm generates the Krylov sequence corresponding to the matrix (A ; I);1 and the starting basis Z0 . Hence it converges if the hypothesis H2 of Theorem 6.14 is satis ed. In the case of a simple eigenvalue, this happens if we start the algorithm with z0 := en . In case of convergence, the rate of convergence can be arbitrarily close to n j j ;  j r() := max jj ; ; j :  2 sp(A) ;  6=  ; j ; j < 1g: Since  is close to an eigenvalue of A, we may expect that the matrix (A ; I);1 be ill conditioned with respect to the linear system involved in the computation of Y satisfying (A ; I)Y = Z; but fortunately, at least when  is a simple eigenvalue, the error involved in computing Y 2 Cj n1 has its major component in the direction of the eigenspace associated with . (See [26].) However, when  is a multiple defective eigenvalue, this error may aect the precision of the generalized approximate eigenvectors associated with . In order to overcome this lack of precision, some variants of the Newton Method have been proposed. (See Sections 2.10 and 5.9 of [26]).

344

6. MATRIX COMPUTATIONS

6.4 Error Analysis

6.4.1 Condition Numbers or Forward Error Analysis

Roughly speaking, the forward error analysis attempts to measure the sensitivity of the solution of an equation under small perturbations of the parameters appearing in the equation. These parameters are known as the data. If the relative error of the solution of the perturbed equation is bounded by some constant times the relative perturbation of the data, then that constant is called a condition number of the data relative to the equation.

Remark 6.23 Condition number for linear systems: Let A 2 Cj nn be a nonsingular matrix and b 2 Cj n1 . Consider the problem of computing the (unique) solution x = u 2 Cj n1 of the linear system Ax = b. Let F : Cj nn  Cj n1  Cj n1 ! Cj n1 be the operator de ned by

F (E; x; y) := Ex ; y; (E; x; y) 2 Cj nn  Cj n1  Cj n1 : Then F (A; u; b) = 0. The partial Frechet derivatives of F with respect to (E; y) and with respect to x at (A; u; b) are the linear operators D(E;y) F (A; u; b) and Dx F (A; u; b) given by D(E;y) F (A; u; b)(H; k) := Hu ; k for (H; k) 2 Cj nn  Cj n1 ; and

Dx F (A; u; b)h := Ah for h 2 Cj n1 ; respectively. Since A is nonsingular, Dx F (A; u; b) is bijective. The equation F (E; x; y) = 0 implicitly de nes a Frechet dierentiable function (E; y) 7! x(E; y) for (E; y) in some open neighborhood O of (A; b), x(A; b) = u = A;1 b and Dx(A; b)(H; k) = ;Dx F (A; A;1 b; b);1 (D(E;y) F (A; A;1 b; b)(H; k)) = ;A;1 (HA;1 b ; k) = ;A;1 (Hu ; k): Let k
k2 be the matrix subordinated norm in Cj nn . Since kbk2 =

345

6.4. ERROR ANALYSIS

kAuk2  kAk2 kuk2 we have kkbukk2  kAk2 . Hence if 2

(Ae ; eb) := kAk;AkAk2 + kbk;bkbk2 2 2 e

e

is so small that (Ae ; eb) 2 O, then there exists a unique vector eu := x(Ae ; eb) such that Aeeu = eb and keu ; uk2  kAk kA;1 k (Ae ; eb) + o((Ae ; eb)): 2 2 kuk2 Thus 2 (A) := kAk2 kA;1 k2 is a condition number of A relative to the linear system Ax = b.

Lemma 6.24 Let A : Cj n1 ! Cj n1 be a bijective linear map,  2 sp(A), P := P (A; fg), M := R(P ), m := dim M , A 2 Cj nn be the matrix which represents A, and P 2 Cj nn be the matrix which represents P with respect to the standard basis for Cj n1 . Consider a matrix U 2 Cj nm representing an orthonormal basis for M and a matrix Y 2 Cj nm such that Y U = Im . Then  2= sp((In ; UY )A). Proof

Let V 2 Cj n(n;m) be such that [U; V] 2 Cj nn is a nonsingular matrix, Z 2 Cj n(n;m) be such that [Y; Z] = [U; V];1 and  := Y AU. Since Y U = Im , the matrix UY represents a projection onto M in the standard basis for Cj n1 and sp() = fg. Also, Z V = In and In ; UY = VZ . But     A = UY + (In ; UY ) A UY + (In ; UY )    AV   Y   AV   Y  Y = [U; V] O Z AV Z = [U; V] O Z AV [U; V];1 : Hence  2= sp(Z AV). To conclude that  2= sp((In ; UY)A) we proceed as follows. Let x 2 Cj n1 be such that x 6= 0 and (In ; UY )A)x = x. Then x = (In ; UY )x and hence (In ; UY )A(In ; UY )x = x. Thus x = VZ x 6= 0, and VZ AVZ x = x. Multiplying by Z on the left, we get Z AV(Z x) = (Z x). This implies that Zx = 0 because  2= sp(Z AV). It follows that x = 0 and hence  2= sp((In ; UY )A).

346

6. MATRIX COMPUTATIONS

Consider now the equations (In ; XY )AX = O, Y X = Im . Let F : Cj nn  Cj nm ! Cj nn  Cj nm be the operator de ned by F (E; X) := (In ; XY )EX for (E; X) 2 Cj nn  Cj nm : Then F (A; U) = (O; O) and the partial Frechet derivatives of F at (A; U) are the linear operators DE F (A; U) and DX F (A; U) given by DE F (A; U)K := (In ; UY )KU for K 2 Cj nn ; and

DX F (A; U)H := (In ; UY )AH ; H for H 2 Cj nm ; respectively. By Lemma 6.24,  2= sp((In ; UY )A), so that DX F (A; U) is bijective. The equation F (E; X) = O implicitly de nes a Frechet dierentiable function E 7! X(E) for E in some open neighborhood OA of A, X(A) = U and DX(A)K = ;DX F (A; U);1 (DE F (A; U)K) = ;U;Y (KU); by the Implicit Function Theorem. Let k
k2 be the matrix subordinated norm in Cj nn and Cj nm . It follows that if e (Ae ) := kAk;AkAk2 2

is so small that Ae 2 OA , then there exists a unique Ue := X(Ae ) such that

kUe ; Uk2  kAk2 kU;Y k (Ae ) + o((Ae ));

where k
k is the operator norm in BL( Cj nm ) subordinated to the 2norm in Cj nm . This suggests that condU (A) := kAk2 kU;Y k is a condition number of A relative to the equations (In ; XY )AX = O, Y X = Im . It can be shown that the norm of U;Y is minimized by the choice Y := U, that is, when the projection on M is orthogonal. (See [2].) We consider the operator G : Cj nn ! Cj mm de ned by G(E) := Y EX(E) for E 2 Cj nn : Clearly, G(A) = . The Frechet derivative of G at A is given by DG(A)K := Y K + Y ADX(A)K for K 2 Cj nn :

6.4. ERROR ANALYSIS

347

Choose Y to be the matrix representing the unique ordered basis for  represents the R(P  ) such that Y U = Im . Then the matrix P := UY n  1 operator P with respect to the standard basis for Cj . We have UDG(A)K = PKU ; PAU;Y (KU) = PKU;

since P and A commute and PU;Y = O. As a consequence, DG(A)K = UPKU

and

kDG(A)k  kPk2 : Hence if (Ae ) is so small that Ae 2 OA , then letting e := G(Ae ), we have ke ; k2  kPk2 (Ae ) + o((Ae )): Since sp() = fg, we have kk;2 1  j1j , so that

ke ; k2  (A)kPk (Ae ) + o((Ae )): 2 kk2 e Then Let e := m1 tr().

and

je ; j = m1 j tr(e ; )j  (e ; )  ke ; k2

je ; j  kAk2 kPk (Ae ) + o((Ae )): jj jj 2 We remark that kjAkj 2 2 [1; 2(A)]. Thus cond (A) := 2 (A) kPk2 is a condition number of A relative to the computation of .

6.4.2 Stability or Backward Error Analysis

Roughly speaking, the backward error analysis consists of showing that the approximate results given by the computer are in fact the exact

348

6. MATRIX COMPUTATIONS

results corresponding to some perturbed data. It gives a measure of the stability of the computation. Let c (
) denote the computed value or result of the element in the brackets. For example, given real numbers x and y, c (x + y) denotes the computed result of their sum, given columns x and y in Cj n1 , c (hx ; yi) denotes the computed value of the inner product hx ; yi, c (kyk2 ) denotes the computed value of the norm kyk2 , and c (Ax) denotes the computed result of the product Ax, where A 2 Cj nm . The backward error analysis addresses the following problem: Given normed linear spaces X and Y , a function f : X ! Y and an element x 2 X , nd h 2 X such that f (x + h) = c (f (x)) and khk  k4xk for each 4x 2 X such that f (x + 4x) = c (f (x)); where k
k denotes the norm on the space X . In most practical situations, we are not able to compute h and we restrict our attention to nding an element 4x 2 X such that f (x + 4x) = c (f (x)) and then estimating its norm in terms of some `reasonable' upper bound of the unit roundo of the computer, which we now de ne. Computers use a particular set of real numbers called oating point numbers, which we denote by IF . From a mathematical point of view, the arithmetic unit of a computer is characterized by four positive integers: p the precision, the arithmetic basis, L the under ow limit, U the over ow limit, and by the oating point function . Using the four positive integers mentioned above, let us de ne the following set: n IF :=  (c1 ;1 + : : : + cp ;p ) e : e 2 [ ; L; U ] ; o

ci 2 [ 0 ; [ for i 2 [ 1 ; p ] ; c1 6= 0 [ f0g: The set IF is nite, symmetric with respect to 0, and the statistical distribution of its elements is far from being uniform.

Example 6.25 A set of oating point numbers: 0 1

-

8

349

6.4. ERROR ANALYSIS

The above gure shows the set IF corresponding to p = 2, = 3,

L = 0 and U = 2. Let

m := ;1;L and M := U (1 ; ;p ): Then m is the smallest positive number in IF and M is the largest one. A oating point map

: [ ; M; ;m] [ [m; M ] [ f0g ! IF may be of two dierent kinds: chopping, which is denoted it by c , or rounding, which is denoted by r . These maps are de ned as follows. If m  x  M , there exist c1 ; c2 ; : : :; cp in [ 0 ; [ and e 2 [ ; L ; U ] such that c1 6= 0 and x;  x < x+ , where

x; := (c1 ;1 + : : : + cp ;p ) e ; x+ := (c1 ;1 + : : : + cp ;p ) e + e;p : Then

c (x) := x; ; 

< x+ ; x;

r (x) := xx; ifif xx ;;x;x  x ; x; : + + These functions are extended to [ ; M; ;m] as odd functions and at 0 they are de ned to be 0. Computers based on c are said to use the chopping arithmetic and those based on r are said to use the rounding arithmetic. We associate to the function , the unit roundo of the computer:

u := inf fv 2 IR : v > 0 and (1 + v) > 1g: Theorem 6.26

The unit roundo in chopping arithmetic is

uc = 1;p and the unit roundo in rounding arithmetic is ur = 21 1;p :

350

6. MATRIX COMPUTATIONS

Proof

The number following 1 in IF is ( ;1 + ;p ) 1 . Hence if 0 < v < 1;p , then c (1 + v) = 1 and if 0 < v < 21 1;p , then r (1 + v) = 1. Also,

c (1 + 1;p ) = r (1 + 12 1;p ) = 1 + 1;p 2 IF :

The proof is complete. In computers which optimize accuracy, uc can be much smaller than

1;p and ur can be much smaller than 12 1;p .

Theorem 6.27 For each x 2 [m; M [, there exists (x) 2 [ ; u; u] such that

(x) = (1 + (x))x:

Proof If m  x  M , then there exist c1 ; c2 ; : : :; cp in [ 0 ; [ and e 2 [ ; L ; U ] such that c1 = 6 0, c(x) = (c1 ;1 + : : : + cp ;p ) e and

c (x)  x < (c1 ;1 + : : : + cp ;p ) e + e;p : Hence If we de ne then

; e;p < c(x) ; x  0: (x) := c (xx) ; x ; e;p

e;p

;p

j(x)j = x  (x)  ;1 = uc : c In fact (x) 2 [ ; uc ; 0]. In the case of rounding arithmetic,

j r (x) ; xj  12 e;p : If we de ne

(x) := r (xx) ; x ;

351

6.4. ERROR ANALYSIS

then The theorem is proved.

j(x)j  12 1;p = ur :

Example 6.28 An example of a unit roundo:

The IEEE standard 754 single- and double-precision values of the unit roundo are Precision Size u Single Double

32 bits (:596)10;7 64 bits (:111)10;15

(See [44].) In order to estimate the roundo error propagation, we need a mathematical model for the elementary arithmetic operations performed by the computer. This means that we need an hypothesis concerning (x op y), when the operation op is addition, subtraction, multiplication or division. We shall assume that our ideal computer performs any of the four elementary operations op in such a way that for each pair (x; y) satisfying x op y 2 IF , there is  2 IR such that

(x op y) = (x op y)(1 + ) and jj  u:

p

We assume in the same context that for each x > 0 such that x 2 IF , there is  2 IR such that

p

p

( x) = x(1 + ) and jj  u: The function u , given by

u (k) := ku 1 ; ku for k 2 [0; 1=u[;

has proved to be useful. It was introduced in [42]. For each integer k, the symbol k will denote any complex number satisfying jk j  u (k): In the following discussion, we attempt to give `simple' bounds for the constants that appear in error estimates. More precisely, instead

352

6. MATRIX COMPUTATIONS

of producing an upper bound of the form  u (cj ), where  is a given positive real number, c is a `generic' constant (usually unspeci ed) and j a variable integer, we propose a `simpler' upper bound u (c0 k) in which c0 is an explicit integer and k a variable integer, possibly greater than j . In this sense, our bounds may not be `optimal', but our conclusions remain valid and consistent with [42], which is one of the main references on this subject. (See also [7].)

Lemma 6.29

The function u satis es the following inequalities:

u (k + 1)  u (k) + u;

u(k + j )  u (k) + u (j ) + u (k) u (j );

u (jk)  j u (k); provided it is well de ned.

Proof

First, if k + 1 < 1=u, then ku + u = (k + 1)u ; ku2  (k + 1)u = (k + 1): u 1 ; ku 1 ; ku 1 ; (k + 1)u Next, if j + k < 1=u, then 2

u(j ) + u (k) + u (j ) u (k) = j u(1 ; ku) + ku(1 ; j u) 2+ jku 1 ; j u ; ku + jku 2 u  j u + ku = (j + k): = 1 ;j uj u+ ;kuku; +jkjk u2 1 ; j u ; k u u Finally, if jk < 1=u, then

jku

jku

1 ; ku  1 ; jku = The proof is complete.

Lemma 6.30

u (jk):

The following results hold: (1 + j )(1 + k ) = 1 + j+k ; 1 + j 1 + k = 1 + maxfj;kg+k :

353

6.4. ERROR ANALYSIS

Also, for each positive integer k satisfying u (k)  3=4, and each real k  ;1, we have p j1 ; 1 + k j  jk j:

Proof

First, for j; k < 1=u, we have (1 + j )(1 + k ) ; 1 = j + k + j k : Next, let j + k < 1=u. Then by Lemma 6.29,

jj + k + j k j  u (j + k): Also,

j u + ku j ; j ;   1 ; j u 1 ; ku = (j + k)u ; 2jku2 : 1 + k 1 + k (1 ; j u)(1 ; 2ku) ku 1 ; ku If j > k, then (1 ; j u)(1 ; 2ku)  (1 ; (j + k)u)(1 ; ku) and (j + k)u ; 2jku2  (j + k)u(1 ; ku), and hence 1 + j (j + k)u

1 +  ; 1  1 ; (j + k )u = u (j + k ): k If j  k, then (j + k)u ; 2jku2  2(1 ; j u)ku and hence 1 + j  2k u = u (2k ): ; 1 1 +  1 ; 2ku k Using the Mean Value Theorem for derivatives, there exists  such that jj  k and p j1 ; 1 + k j = 2pj1k+j  : 1 +

Thus

1 =

k

p



j1 ; 1 + k j 

jk j = j j; k 2 1 ; 3=4 p

which completes the proof. The following results concern complex arithmetic and matrix operations. They allow us to estimate the propagation of roundo errors in

354

6. MATRIX COMPUTATIONS

computations. They will be used in the backward error analysis of the Basic QR Method.

Proposition 6.31 Let x, y be complex numbers. If c (x  y) exists, then there exists  2 Cj such that c (x  y) = (x  y)(1 + ); jj  u: If c (x  y) exists and 3u < 1, then there exists  2 Cj such that c (xy) = (xy)(1 + ); jj  u(3): ;
If c jyj2 exists, then there exists  2 IR such that ;
c jyj2 = jyj2(1 + ); jj  u(2): If c (x=y) exists, then there exists  2 Cj such that c (x=y) = (x=y)(1 + ); jj  u(6): Proof

Let us write a complex number x in its Cartesian form x = x< + i x= , where x< 2 IR is its real part and x= 2 IR its imaginary part. Then

c (x  y) = (x<  y<) + i (x=  y=) = (x<  y< )(1 + 1 ) + i (x=  y= )(1 + 2 ) for some 1 and 2 such that j1 j  u and j2 j  u. Suppose x  y 6= 0 and let  := c (x  yx);y(x  y) . Then jjjx  yj = jc (x  y) ; (x  y)j = q j(x<  y< )1 + i (x=  y= )2 j = (x<  y< )2 12 + (x=  y= )2 22  maxfj1j; j2 jg jx  yj  ujx  yj: Similarly, if c (x  y) exists, c (x  y) = (x< y<(1 + 1) ; x= y=(1 + 2))(1 + 3) +i (x< y= (1 + 4 ) + x= y< (1 + 5 ))(1 + 6 ):

355

6.4. ERROR ANALYSIS

Suppose x  y 6= 0 and let  := c (x  yx);y(x  y) . Then

jx  yjjj = jc (x  y) ; (x  y)j = jx< y< (1 + 3 + 1 3 ) ; x= y= (2 + 3 + 2 3 )

+i (x< y= (4 + 6 + 4 6 ) + x= y< (5 + 6 + 5 6 )j;

for some i such that ji j  u, i 2 [ 1; 6 ] . Hence

jx  yj2 jj2  (jx< y< j2 + jx= y= j2 + 4jx< y< x= y= j + jx< y= j2 + jx= y< j2 )u2 (2 + u)2 : But 2jx< y< x= y= j  jx< y< j2 + jx= y= j2 ; 2jx< y< x= y= j  jx< y= j2 + jx= y< j2 ; and

jx  yj2 = jx< y< j2 + jx= y= j2 + jx< y= j2 + jx= y< j2 ;

so that

jj2  2u2 (2 + u)2  u (3)2 ;

p

since 3u < 1 and the polynomial p(t) := 3t2 + 5t ; 2 + 3= 2 has two ;
negative roots. Also, if c jyj2 exists,

c ;jyj2
= c (y  y) = (y<2 (1 + 1) + y=2 (1 + 1))(1 + 1)  jyj2(1 + 2): Finally, if c (x=y) exists,   xy(1 + 3 ) (1 +  ) = x (1 +  ); c (x=y) = c xy = 1 6 yy yy(1 + 2 ) y

by Lemma 6.30.

Example 6.32 Backward error analysis of elementary operations:

The assertions c (x  y) = (x + 4x)  (y + 4y); where j4xj  ujxj; j4yj  ujyj; c (xy) = (x + 4x)y; where j4xj  u (3)jxj; c (x=y) = (x + 4x)=y; where j4xj  u (6)jxj; follow from the proof of Proposition 6.31.

356

6. MATRIX COMPUTATIONS

Remark 6.33 Backward error analysis of Gaussian Elimination with partial pivoting: Let A be a real matrix of order n and b be a real n1 column matrix. Suppose that the Gaussian Elimination Algorithm with partial pivoting produces a computed solution c (x) of the linear system Ax = b. Then there exists a matrix 4A independent of b such that (A + 4A)c (x) = b and k4Ak1  2n2 u (n)n kAk1 ;

where n is the growth factor. For proofs and details, see Sections 9.1 and 9.2 of [42]. In order to study the stability of the Basic QR Method, that is, the roundo error propagation of this method when it runs on a computer, we need some results about the numerical computation of Householder symmetries.

Lemma 6.34

Let x := [x(1); : : :; x(n)]> 2 Cj n1 and k 2 [ 1; n ; 1 ] . De ne xk :=

Then

j =k

x(j )ej :

c (kxk k2 ) = kxk k2 (1 + 3+n;k ); c (hy ; xk i) = hy ; xk i(1 + 3+n;k ) for each y 2 Cj n1:

Proof Since

n X

q

c (kxk k2 ) = c ;kxk k22
(1 + 1); c ;kxk k22
= ;jx(k)j2 + c ;kxk+1 k22

(1 + 1)

and by Proposition 6.31,

c ;jx(k)j2
= jx(k)j2 (1 + 2);

we get, by Lemma 6.30, c ;kxk k22
= jx(k)j2 (1 + 3) + c ;kxk+1 k22
(1 + 1) =

nX ;1 i=k

jx(i)j2 (1 + 3+i;k ) + jx(n)j2 (1 + 3+n;1;k );

357

6.4. ERROR ANALYSIS

that is

;




jc kxk k22 ; kxk k22 j  kxk k22 u (2 + n ; k): Thus c (kxk k2 ) = kxk k2 (1 + 3+n;k ). Similarly,





c (hy ; xk i) = y(k)x(k)(1 + 3) + c (hy ; xk+1 i) (1 + 1 ) = so that as stated.

nX ;1 i=k

y(i)x(i)(1 + 4+i;k ) + y(n)x(n)(1 + 3+n;k );

c (hy ; xk i) = hy ; xk i(1 + 3+n;k );

Proposition 6.35 > n1 Let x := [x(1); : : :; x(n)] 2 Cj be such that for some k 2 [ 1; n ; 1 ] , x(k) = 6 0. Let H := I ; vv be the Householder symmetry de ned in Lemma 6.4. Then

jc ( ) ; j  u (2(n ; k) + 9)j j; jc (v) ; vj  u (n ; k + 10)jvj; where jvj := [ jv(1)j; : : : ; jv(n)j ]> .

Proof

First note that is real:

= (jx(k)j1+ ) :

Next, by Lemma 6.34 and Lemma 6.30, c () = (1 + 3+n;k ) and hence c ( ) = c () (c (jx(k)1j)++c1 ()) (1 +  ) 2 1 +  1 = (1 +  3+n;k ) (jx(k )j(1 + 3 ) + (1 + 3+n;k )) (1 + 2 ) + 1 =  (jx(k)j + ) (11 + 3+n;k )2 (1 + 2 ) = (1 + 2(n;k)+9 ):

358

6. MATRIX COMPUTATIONS

Similarly, and

c (v(k)) = (x(k) + c (s))(1 + 1);   c (s) = c jxx((kk))j c ()(1 + 1)

=  jxx((kk))j (1 + 4 )(1 + 3+n;k )(1 + 3 ) = s(1 + n;k+10 );

since

c





x(k) x(k) x(k)(1 + 1 ) x( k ) jx(k)j = c (jx(k)j) (1 + 1 ) = jx(k)j(1 + 3 ) = jx(k)j (1 + 4 ):

The result now follows.

Proposition 6.36 Let y 2 Cj n1 and H be the Householder symmetry de ned in Lemma 6.4. Then there exists 4y 2 Cj n1 such that c (Hy) = H(y + 4y) and k4yk2  u(5(n ; k) + 37) kyk2 : Proof

We have

c (Hy) = (y ; c ( vv y))(1 + 1) = (y ; c ( )c (v)(1 + 1 )c (v y)(1 + 3 )) (1 + 1 ) = (y ; vv y(1 + 2(n;k)+9 )(1 + n;k+10 ) (1 + n;k+3 )(1 + 4 ))(1 + 1 ) = Hy(1 + 5(n;k)+37 ) = H(y + 4y);

where

kH4yk2  u (5(n ; k) + 37): But H is a unitary matrix, so that kH4yk2 = k4yk2 .

Proposition 6.37 Let A 2 Cj nn and H be the Householder symmetry de ned in Lemma

359

6.4. ERROR ANALYSIS 6.4. Then there exist E 2 Cj nn and 4A 2 Cj nn such that

c (HA) = H(A + E); kEkF  u (5(n ; k) + 37)kAkF ; c (HAH) = H(A + 4A)H; k4AkF  u(10(n ; k) + 74)kAkF : Proof

For each j 2 [ 1; n ] , de ne yj := Aej . By Proposition 6.36,

c (Hyj ) = H(yj + 4yj ); k4yj k2  u(5(n ; k) + 37) kyk2 : Hence

c (HA) = [c (Hy1); : : : ; c (Hyn)] = H[y1 + 4y1; : : : ; yn + 4yn] = H([y1 ; : : : ; yn ] + [4y1; : : : ; 4yn]): If we de ne

E := [4y1; : : : ; 4yn];

then

kEkF =

v uX u n t

j =1

k4yj k22  u (5(n ; k) + 37)kAkF :

Also,

c (AH) = (A + F)H with kFkF  u(5(n ; k) + 37)kAkF ; and

c (HAH) = H(c (AH) + G) with kGkF  u(5(n ; k)+37)kc (AH)kF : Hence

c (HAH) = H((A + F)H + GH2) = H(A + F + GH)H; since H2 = I. If we de ne

4A := F + GH; then

k4AkF  kFkF + kGHkF = kFkF + kGkF ; since kGHkF  kGkF kHk2 and kHk2 = 1. For the same reason, by Lemma 6.29, kGkF  u (5(n ; k)+37)kA + FkF  u (10(n ; k)+74)kAkF .

360

6. MATRIX COMPUTATIONS

Proposition 6.38 Let A 2 Cj nn and A0 := U AU be the upper Hessenberg matrix obtained in Proposition 6.7 with the help of Householder symmetries. Then there exist 4A 2 Cj nn and a unitary matrix V 2 Cj nn such that c (A0 ) is an upper Hessenberg matrix satisfying c (A0 ) = V(A + 4A)V; k4AkF  u(13n2)kAkF :

Proof

Let Hk denote the Householder symmetry which at step k annihilates the entries of column k having a row index greater than k + 1. Let us set A1 := c (H1 AH1 ) and A2 := c (H2 A1 H2 ): By Proposition 6.37, A1 = H1 (A + 4A1)H1 ; k4A1kF  u (10(n ; 1) + 74)kAkF ; A2 = H2 (A1 + E)H2 ; kEkF  u (10(n ; 1) + 74)kA1 kF ; so that A2 = H2 H1 (A + 4A1 + H1 EH1 )H1 H2 ; and since H1 is a unitary matrix, 4A2 := 4A1 + H1 EH1 satis es k4A2kF  k4A1kF + kEkF  u(10(n ; 1) + 74)kAkF + u (10(n ; 2) + 74)kA1kF : But k4A1kF  kAkF + k4A1kF  (1 + u (10(n ; 1) + 74))kAkF : Hence k4A2kF  u (10[(n ; 1) + (n ; 2)] + 128): Repeating this process, we obtain An;2 = Hn;2 Hn;3


H1 (A + 4A)H1


Hn;3 Hn;2 ; where 

k4AkF  u 74(n ; 2) + 10

nX ;2 k=1



(n ; k) kAkF  u (13n2)kAkF ;

361

6.4. ERROR ANALYSIS

since

74(n ; 2) + 10

nX ;2 k=1

(n ; k) = 5n2 + 69n ; 158  pn2

for each positive integer n, if p  13.

Theoremn6.39 Let A 2 Cj n and A0 := U AU be the upper Hessenberg matrix obtained in Proposition 6.7 with the help of Householder symmetries. Let k  1 be an integer and Ak the matrix de ned by k iterations of the Basic QR algorithm. Then there exist 4A 2 Cj nn and a unitary matrix V 2 Cj nn such that c (Ak ) is an upper Hessenberg matrix satisfying c (Ak ) = V(A + 4A)V; k4AkF  u(22(k + 1)n2)kAkF : Proof

It suces to analyze the backward error of one iteration of the Basic QR algorithm. Indeed,

c (Ak ) = c ;Qk c (Ak;1 )Qm
; where Qk is a product of k ; 1 Householder symmetries and A0 is an

upper Hessenberg matrix unitarily similar to A through Householder symmetries. Applying Proposition 6.38,

c (Ak ) = V (c (Ak;1 ) + 4Ak;1)V for some unitary matrix V and for some matrix 4Ak;1 satisfying 

k4Ak;1kF  u 74(n ; 1) + 10 Hence since, as before,

nX ;1 j =1



(n + 1 ; k) kc (Ak;1 )kF :

k4Ak;1kF  u (22(k + 1)n2 )kAkF

74(n ; 2) + 10

nX ;2 k=1

(n + 1 ; k) = 5n2 + 69n ; 74  pn2

for each positive integer n, if p  22.

362

6. MATRIX COMPUTATIONS

6.4.3 Relative Error in Spectral Computations and Stopping Criteria for the Basic QR Method

Forward and backward error analyses taken together allow us to bound the relative error of a computed eigenvalue or a computed basis for a spectral subspace. In fact, if the algorithm is backward stable, the computed results correspond to exact computations on some perturbed data, and then (Relative Error)  (Condition Number)  (Relative Perturbation). The backward error analysis provides a bound for the relative perturbation of the data, and the forward error analysis provides a bound for the condition number. However, the latter is usually expressed in terms of unknown quantities. That is why the preceding theoretical upper bound on the relative error is not useful in practice as a stopping criterion for a computer run. This remark justi es our suggestion of a residual test to stop the Simultaneous Iteration Method in subsection 6.3.2. Let us consider stopping criteria for the Basic QR Method. We rst take up a backward error point of view. When the method is applied to an upper irreducible Hessenberg matrix, Theorem 6.18 suggests that the diagonal entries of an iterate Ak are `good' approximate eigenvalues of A if the subdiagonal entries of Ak are `small enough'; and that if a subdiagonal entry of Ak becomes `almost' zero, we could decide to neglect it and then view Ak as a block triangular matrix and perform the next iterations in each diagonal block separately. This could be done using two dierent processors, for instance. The decoupling of the problem into two smaller problems is called de ation. These considerations are con rmed by the backward error standpoint as we see now. With the notations of Theorem 6.39, let Tk and Lk be the matrices de ned by 

i  j; Tk (i; j ) := c0 (Ak )(i; j ) ifotherwise, 

i = j + 1; Lk (i; j ) := c0 (Ak )(i; j ) ifotherwise. Since c (Ak ) is an upper Hessenberg matrix, we have

c (Ak ) = Tk + Lk :

363

6.4. ERROR ANALYSIS

De ne

Ae := VTk V :

Then Theorem 6.39 gives Ae = A + 4A ; VLk V ; where k4AkF  n;k ukAkF ; for some constant n;k of `moderate' size. Recall that kVLk V kF = kLk kF . We conclude that to guarantee stability of computations, we should stop the algorithm at an iteration k such that v un;1 uX t

j =1

jL(j + 1; j )j2  n;k ukAkF

for some constant n;k of the order of n;k . Then

kAe ; AkF  ( + )u: n;k n;k kAkF

This approach gives the type of stopping criteria used by several standard software packages. For instance, in EISPACK, if jLk (j + 1; j )j  u(jTk (j; j )j + jTk (j + 1; j + 1)j) for some `small' constant , then Lk (j +1; j ) is rede ned to be zero, and a de ation into two blocks takes place. Since the relative error of computed eigenvalues is bounded by the product of both the condition number and the relative perturbation of the data, stopping criteria arising from a backward error analysis will be meaningful only for a matrix which is well conditioned relative to the computation of its eigenvalues. This observation is con rmed by the following example: Let  be a \small" positive number,  a \large" positive number, and let e1 and e2 be real numbers. Suppose that 



Ak := e1 e : 2 This matrix has eigenvalues 1 := e1 + 2e2 +  ; 2 := e1 + 2e2 ;  ;

364 where

6. MATRIX COMPUTATIONS p

 := (e1 ; e2 )2 + 4:

If  is `small enough', we would like to conclude that the diagonal entries of Ak , that is, Ak (1; 1) = e1 and Ak (2; 2) = e2 , are `good' approximate eigenvalues of A. However, (i) if e1 = e2 = e, then 1 = e + 1, 2 = e ; 1 , with 1 = p, and (ii) if e1 6= e2 , then 1 = e1 + e2 + 2je1 ; e2 j2 ; 2 = e1 + e2 ; 2je1 ; e2 j2 ; where s 2 := 1 + (e 4;e )2 : 1

2

We observe that the diagonal entries of Ak are `good' approximate eigenp is small values if and only if enough in case (i), and if and only if p=je1 ; e2 j is small enough in case (ii). For this reason, many implementations of the QR Method begin by `scaling' A, that is, by dividing it by its own 1-norm, for example. Since (A)  kAk1 , the scaled matrix will have all its eigenvalues inside the unit circle. But then the scaling operation may make convergence slow. In fact, theoretical analysis suggests that the stopping criterion for the Basic QR Method from a relative error standpoint must take into account three aspects of the kth iterate Ak : 1. the size of the subdiagonal entries, 2. the size of the superdiagonal entries, and 3. the distance between two consecutive diagonal entries. The next theorem suggests a similar conclusion for an nn upper Hessenberg matrix with distinct diagonal entries.

Theorem n6.40 Let B 2 Cj n be an upper Hessenberg matrix such that all its diagonal entries are dierent. Let the matrix-valued function B : IR ! Cj nn be de ned by 8 < t B(i; j ) if i ; j = 1; B(t)(i; j ) := :

B(i; j ) otherwise.

365

6.4. ERROR ANALYSIS If  > 0 is a small enough positive number and nX ;1 j =1

jB(j + 1; j )j < ;

then there exist n dierentiable functions j j 2 [ 1; n ] ; t 2 [ ; 1; 1] 7! j (t) 2 C; such that, for each j 2 [ 1; n ] , j (t) is an eigenvalue of B(t) and j (0) = B(j; j ) for j 2 [ 1; n ] ; ; 1)B(1; 2) if j = 1; 0j (0) = BB(1(2; 1) ; B(2; 2) n ; 1)B(n ; 1; n) 0j (0) = BB(n(n; ; 1; n ; 1) ; B(n; n) if j = n; and B(j ; 1; j ) B(j +1; j )B(j; j +1) 0j (0) = BB(j(;j; 1j;;j 1) + ; 1) ; B(j; j ) B(j; j ) ; B(j +1; j +1) if 1<j
Proof

The eigenvalues of B(0) are its diagonal entries which are all distinct. When a polynomial of degree n has n distinct roots, these are dierentiable functions of the polynomial coecients. Hence the eigenvalues of a matrix are continuous functions of its entries. Thus for  small enough, B(1) has distinct eigenvalues. Let us de ne the matrix-valued function A : IR Cj ! Cj nn by 8 B(i; j ) ; x if i = j; > > > > <

A(t; x)(i; j ) := > t B(i; j ) > > > :

if i ; j = 1;

B(i; j ) otherwise. Then A(1; 0) = B and A(0; 0) is the upper triangular part of B. A is an in nitely dierentiable function and @ A (t; x) = ;I: @ A (t; x) = B ; A(0; 0);

@t

@x

Since det : Cj n1  : : :  Cj n1 ! Cj is an in nitely dierentiable function, the chain rule gives @ det A (t; x) = D(det(A(t; x))) @ A (t; x)

@x

@x

366

6. MATRIX COMPUTATIONS 2

3

B(1; 1) ; x





B(1; n) 6 tB(2; 1) B(2; n) 7 6 7 . . .. 6 7 .. .. 6 7 0 . 6 7 n X 6 7 . . . . . . 6 . . ej . B(j; n) 77 = det 6 7 .. .. .. j =1 6 6 7 . . . 6 7 6 7 . . .. .. 4 5 0





B(n; n) ; x

= =

2 n Y X 6 (B( 4

j =1 ``=1 6=j n Y X j =1 ``=1 6=j

3

`; `) ; x) + t
j (t; x)75

(B(`; `) ; x) + t
(t; x);

where

j
j (t; x)j  (2kBk?)n;1 ; j
(t; x)j  n(2kBk?)n;1 and

kBk? := maxfjB(i; j )j : i; j 2 [ 1; n ] g: Let j 2 [ 1; n ] . Since B(j; j ) = 6 B(k; k) for all k = 6 j, @ det A (0; B(j; j )) = Y (B(`; `) ; B(j; j )) 6= 0; @x

`=1 `=j

6

and there exists a dierentiable function j : ] ; 1; 1[ ! Cj such that det(A(t; j (t))) = 0 for jtj < ; j (0) = B(j; j ) and

@ det A (0;  (0)) j @t 0j (0) = ; @ det A (0;  (0)) : j @x

On the other hand, @ det A (t; x) = D(det(A(t; x))) @ A (t; x)

@t

@t

367

6.4. ERROR ANALYSIS

B(1; 1) ; x


0


B(1; n) 3 .. .. 6 7 6 tB(2; 1) . 0 . B(2; n) 77 6 .. 6 7 6 7 0 . 6 7 6 7 . . . n .. .. .. X 6 7 0 6 7 = det 6 7 . . . . 6 j =1 6 . B(j +1; j ) . B(j +1; n) 77 6 7 .. .. 6 7 . 0 . 6 7 6 7 . . . 4 .. .. .. 5 0


0


B(n; n) ; x 2

=

n X j =1

(;1)2j+1 B(j + 1; j )Dj ;

where 2

Dj :=

6 6 6 6 6 6 6 6 det 66 6 6 6 6 6 6 4

B(1; 1) ; x




. . . . . . 0

=;

n X j =1

. . .

0 . . . . . .

. . . . . .

B(1; j ; 1) . . .

. . .

. . .

. . .

B(j ; 1; j ; 1) ; x B(j ; 1; j +1) t B(j; j ; 1) B(j; j +1)


B(j; n) 0 t B(j +2; j +1)


B(j +2; n) 0 0 B(j +3; n)






. . .

. . .

0

0

. . .

. . .




B(

n; n)

;

e j ; tB(j + 2; j + 1)
 j ]; B(j + 1; j )[Aj Bj ; tB(j; j ; 1)


Aj and Bj are given by 2

B(1; 1) ; x 6 6 Aj := det 66 t B(2; 1) 4

3

B(1; j +1)


B(1; n)






B(1; j ; 1) B(2; j ; 1)

3

7 .. 7 . 7; 7 ... 5 t B(j ; 1; j ; 2) B(j ; 1; j ; 1) ; x

x

7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 5

368

6. MATRIX COMPUTATIONS 2

B(j +2; j +2) ; x 6 6 Bj := det 66 t B(j +3; j +2) 4




.. . ...

3

B(j +2; n) 7 B(j +3; n) 7 7; 7

t B(n; n ; 1) B(n; n) ; x

5

e j and
 j satisfy and


maxfj
e j j ; j
 j jg  (2kBk? )n;2 : Hence

0j (0) = BB(1(2; 1); 1);BB(1(2; 2) ; 2) if j = 1; n ; 1)B(n ; 1; n) 0j (0) = BB(n(n; ; 1; n ; 1) ; B(n; n)

if j = n;

and

B(j +1; j )B(j; j +1) B(j ; 1; j ) 0j (0) = BB(j(;j; 1j;;j 1) ; 1) ; B(j; j ) + B(j; j ) ; B(j +1; j +1) if 1<j
This completes the proof. In Theorem 6.40, let B denote the iterate Ak of the Basic QR algorithm and  > 0 satisfy nX ;1 j =1

jAk (j + 1; j )j < :

Then, for  small enough, jj (1) ; Ak (j; j )j  j0j (0)j + o(); j 2 [ 1; n ] : This leads to the following stopping criterion for the Basic QR Method which takes into account the total relative error: Let 21 10;p be an upper bound of the permitted relative error in the approximate eigenvalues. Assume that, by introducing shifts if necessary, j0j (1)j < jAk (j; j )j for j 2 [ 1; n ] ; where 0j (1) is given by Theorem 6.40 with B := Ak . Then, the algorithm should be stopped at an iteration k such that j0j (1)j 1 10;p for j 2 [ 1; n ] ; and < 0 jAk (j; j )j ; jj (1)j 2

369

6.4. ERROR ANALYSIS nX ;1 j =1

jAk (j + 1; j )j < 12 10;p:

This condition implies that the total relative error of each approximate eigenvalue Ak (j; j ) is bounded as follows:

jj (1) ; Ak (j; j )j < 1 10;p + o;10;p
; j 2 [ 1; n ] : jj (1)j 2 Similar criteria have been implemented in [7] and [10].

6.4.4 Relative Error in Solving a Sylvester Equation

Let m and n be positive integers such that m  n. Consider B 2 Cj nn and Z 2 Cj mm such that sp(B) \ sp(Z) = ;. Let y 2 Cj nm be given. By Proposition 1.50, the Sylvester equation Bx ; xZ = y

has a unique solution in Cj nm . In this section we give the forward error analysis as well as the backward error analysis for the computation of this solution. The forward error analysis of this problem can be carried out as follows. Consider the Frobenius norm k
kF on Cj nm and let k
k denote the subordinated operator norm on BL( Cj nm ). De ne a Sylvester operator S : Cj nm ! Cj nm by

S ( x ) := B x ; x Z for x 2 Cj nm : Then, along the lines of Remark 6.23,

(S ) := kSk kS ;1k is a condition number of S 2 BL( Cj nm ) relative to the equation S ( x ) = y . In [39], kS ;1 k is called the separation between the matrices B and Z. We have shown in Proposition 1.50 that sp(S ) = f ;  :  2 sp(B);  2 sp(Z)g. Thus

(S ) = maxfj ; j :  2 sp(B);  2 sp(Z)g and (S ;1 ) = minfj ; j :  21sp(B);  2 sp(Z)g :

370

6. MATRIX COMPUTATIONS

Hence

fj ; j :  2 sp(B);  2 sp(Z)g (S )  (S )(S ;1 )  max minfj ; j :  2 sp(B);  2 sp(Z)g :

This inequality shows that if B has at least two distinct eigenvalues and if Z changes in such a way that an eigenvalue of Z comes close to an eigenvalue of B, then (S ) tends to 1. In other words, for S to be well conditioned it is necessary that sp(B) and sp(Z) be well separated. The proof of Proposition 1.50 suggests the following method for solving the Sylvester equation S ( x ) = y . Compute a Schur form of Z, that is, nd a unitary matrix V 2 Cj mm such that T := V ZV is an upper triangular matrix. Let u := x V and v := y V. Then S ( x ) = y if and only if B u ; u T = v : Let N := T ; diag [T(1; 1); : : :; T(m; m)] m X b := 1 T(i; i) and

m i=1 Tb := bIm + N;

that is, Tb is obtained from T by replacing each of the diagonal entries of T by their arithmetic mean. Suppose that there exists a positive constant of `moderate' size such that the following `cluster condition' is satis ed: (C )

kTb ; TkF =

m X i=1

jT(i; i) ; bj2

1=2

 u:

Now, instead of solving the equation B u ; u T = v , we solve the perturbed equation B u ; u Tb = c ( v ). Let u := [u1 ; : : : ; um ] and c ( v ) := [c (v1 ); : : : ; c (vm )]. Then B u ; b u T = c ( v ) if and only if (B ; bIn )u1 = c (v1 ); (B ; bIn )uj = c (vj ) +

j ;1 X i=1

T(i; j )ui

for j = 2; : : : ; m:

Suppose that the upper triangular matrix T = V ZV is obtained as follows. First the matrix Z is transformed to a unitarily similar upper Hessenberg matrix Z0 by employing Householder symmetries (Proposition 6.7). Then ` iterations of the QR Method are performed on the

371

6.4. ERROR ANALYSIS

matrix Z0 so as to satisfy the stopping criterion from the backward error standpoint presented in Subsection 6.4.3; the QR factors in each such iteration are also computed by employing the Householder symmetries. Further, suppose that the linear system satis ed by uj , j 2 [ 1; m ] , is solved by Gaussian Elimination with partial pivoting. The backward error analysis of the above algorithm for solving a Sylvester equation can be developed as follows. As mentioned in Subsection 6.4.3, the matrix T satis es kVTV ; ZkF  ( + )u m;` m;` kZkF b  ; Z. for constants m;` and m;` of `moderate' size. Let 4Z := VTV Then, under the cluster condition (C ), k4ZkF  (m;` + m;` + )ukZkF : As stated in Remark 6.33, for each j 2 [ 1; m ] , the solution c (uj ) 2 Cj n1 , computed by Gaussian Elimination with partial pivoting, is such that there exists a matrix 4B 2 Cj nn which satis es (B + 4B ; bIn )c (u1 ) = c (v1 );

(B + 4B ;  c (uj ) = c (vj ) + bIn )

and Hence

j ;1 X i=1

T(i; j )c (ui )

for j = 2; : : : ; m;

k4Bk1  2n2 u (n)n kB ; bIn k1 :

(B + 4B)c ( u ) ; c ( u )Tb = c ( v ); and c ( u )V 2 Cj nm satis es (B + 4B)c ( u )V ; c ( u )V (Z + 4Z) = c ( v )V : The computed solution of the original Sylvester equation is then ;
c ( x ) := c c ( u )V : Since the matrices V and V are constructed by employing Householder symmetries, the proof of Proposition 6.37 shows that there are  y and w 2 Cj nm such that c ( v ) = ( y +  y )V and k y kF  c1 uk y kF ; c ;c ( u )V
= (c ( u ) + w )V and k w kF  c2ukc ( u )kF ;

372

6. MATRIX COMPUTATIONS

for some constants c1 and c2 of `moderate' size, depending only on n and m. Hence (B + 4B)c ( x ) ; c ( x )(Z + 4Z) = y + 4 y ; where Since

4 y = S ( w V ) +  y + O((4B) w ) + O( w (4Z)):

kS ( w V )kF  kSk k w kF and kc ( u )kF  kS ;1 k k y kF + O(u2 );

it follows that

k4 y kF  (c1 + c2 (S ))uk y kF + O(u2 ): Thus under the cluster condition stated above, the suggested algorithm for solving a Sylvester equation is backward stable, provided (S ) is of `moderate' size. The condition number (S ) plays a role in the backward error analysis because we have performed a change of variable in the original equation. Other numerical methods for solving Sylvester equations are presented in [26].

References

[1] Ahues M. (1987): A class of strongly stable operator approximations, J. Austral. Math. Soc. Ser. B, 28, 435-442. [2] Ahues M. (1989): Spectral condition numbers for defective eigenelements of linear operators in Hilbert spaces, Numer. Funct. Anal. and Optimiz. 10 (9 & 10), 843-861. [3] Ahues M., Aranchiba S. and Telias M. (1990): RayleighSchrodinger series for defective spectral elements of compact operators in Banach spaces First Part: Theoretical Aspects, Numer. Funct. Anal. Optimiz., 11, 839-850. [4] Ahues M. and Hocine F. (1994): A note on spectral approximation of linear operations, Appl. Math. Lett., 7, 63-66. [5] Ahues M. and Largillier A. (1990): Rayleigh-Schrodinger series for defective spectral elements of compact operators in Banach spaces Second Part: Numerical comparison with some inexact Newton methods, Numer. Funct. Anal. Optimiz., 11, 851-872. [6] Ahues M. and Largillier A. (1995): A variant of the xed tangent method for spectral computations on integral operators, Numer. Funct. Anal. and Optimiz. 16 (1 & 2), 1-17. [7] Ahues M., Largillier A. and Tisseur F. (1996): Stability and precision of the QR eigenvalue computations, Householder Meeting, Pontresina, Switzerland. [8] Ahues M. and Limaye B.V. (2000): Computation of spectral subspaces for weakly singular integral operators, to appear. 373

374

References

[9] Ahues M. and Telias M. (1986): Re nement methods of Newton type for approximate eigenelements of integral operators, SIAM J. Numer. Anal., 23, 144-159. [10] Ahues M. and Tisseur F. (1997): A new de ation criterion for the QR algorithm, UT CS-97-353, ( le: lawn122.ps), LAPACK. [11] Alam, R. (1995): Accelerated spectral approximation, Ph.D. Thesis, Indian Institute of Technology Bombay, India. [12] Alam R. (1998): On spectral approximation of linear operators, J. Math. Anal. Appl., 226, 229-244. [13] Alam R., Kulkarni R.P. and Limaye B.V. (1996): Boundedness of adjoint bases of approximate spectral subspaces and of associated block reduced resolvents, Numer. Funct. Anal. Optimiz., 17, 473501. [14] Alam R., Kulkarni R.P. and Limaye, B.V. (1998): Accelerated spectral approximation, Math. Comp., 67, 1401-1422. [15] Alam R., Kulkarni R.P. and Limaye, B.V. (2000): Accelerated spectral re nement. Part I: Simple eigenvalue, J. Austral. Math. Soc. Ser. B, 41, 487-507. [16] Alam R., Kulkarni R.P. and Limaye, B.V. (2000): Accelerated spectral re nement. Part II: Cluster of eigenvalues, ANZIAM J., 42, 224-243. [17] Anselone P.M. (1971): Collectively Compact Operator Approximation Theory and Applications to Integral Equations, Prentice-Hall, Englewood Clis, N.J. [18] Anselone P.M. (1981): Singularity subtraction in the numerical solution of integral equations, J. Austral. Math. Soc. Ser. B, 22, 408-418. [19] Anselone P.M. and Ansorge R. (1979): Compactness principle in nonlinear operator approximation theory, Numer. Funct. Anal. Optimiz., 1, 589-618. [20] Anselone P.M. and Treuden M.L. (1996): Spectral analysis of asymptotically compact operator sequences, Numer. Funct. Anal. Optimiz., 17, 679-690.

References

375

[21] Atkinson K.E. (1967): The numerical solution of eigenvalue problem for compact integral operators, Trans. Amer. Math. Soc., 129, 458-465. [22] Baker C.T.H. (1977): The Numerical Treatment of Integral Equations, Oxford Univ. Press, London. [23] Bouldin R. (1990): Operator approximations with stable eigenvalues, J. Austral. Math. Soc., Ser. A, 49, 250-257. [24] Brakhage H. (1961): Zur Feherabschatzung fur die numerische Eigenwertbestimmung bei Integral-gleichungen, Numer. Math., 3, 174-179. [25] Chatelin F. (1983): Spectral Approximation of Linear Operators, Academic Press, New York. [26] Chatelin F. (1993): Eigenvalues of Matrices, (English edition) John Wiley and Sons, Chichester. [27] Conte S.D. and de Boor C. (1980): Elementary Numerical Analysis: An Algorithmic Approach, Third Edition, McGraw Hill, New York. [28] Cryer C.W. (1982): Numerical Functional Analysis, Oxford Univ. Press, London. [29] de Boor C. (1978): A Practical Guide to Splines, Springer-Verlag, New York. [30] de Boor C. and Swartz B. (1980): Collocation approximation to eigenvalues of an ordinary dierential equation: The principle of the thing, Math. Comp., 35, 679-694. [31] Dellwo D.R. and Friedman M.B. (1984): Accelerated spectral analysis of compact operators, SIAM J. Numer. Anal., 21, 1115-1131. [32] Dellwo D.R. (1989): Accelerated spectral re nement with applications to integral equations, SIAM J. Numer. Anal., 26, 1184-1193. [33] Deshpande L.N. and Limaye B.V. (1989): A xed point technique to re ne a simple approximate eigenvalue and a corresponding eigenvector, Numer. Funct. Anal. Optimiz., 10, 909-921. [34] Deshpande L.N. and Limaye B.V. (1990): On the stability of singular nite-rank methods, SIAM J. Numer. Anal., 27, 792-803.

376

References

[35] Devore R.A. and Lorentz G.G. (1991): Constructive Approximation, Springer-Verlag, Berlin. [36] Diestel J. (1984): Sequences and Series in Banach Spaces, Springer-Verlag, New York. [37] Erxiong J. and Zhenyue Z.(1985): A new shift of the QL algorithm for irreducible symmetric tridiagonal matrices, Linear Algebra Appl., 65, 261-272. [38] Gohberg I., Goldberg S. and Kaashoek M.A. (1990): Classes of Linear Operators, Vol. 1, Birkhauser Verlag, Basel. [39] Golub G.H. and Van Loan C. (1996): Matrix Computations, Third Edition, The Johns Hopkins University Press, Baltimore. [40] Grigorie R.D. (1975): Diskrete Approximation von Eigenwert problemen I: Qualitative Konvergenz, Numer. Math., 24, 355-374. [41] Henrici P. (1962): Bounds for iterates, inverses, spectral variation and elds of values of nonnormal matrices, Numer. Math. 4, 24-40. [42] Higham N. (1996): Accuracy and Stability of Numerical Algorithms, SIAM, Philadelphia. [43] Horn R. and Johnson C.R. (1985): Matrix Analysis, Cambridge Univ. Press, Cambridge. [44] IEEE standard for binary oating point arithmetic, ANSI/IEEE Standard 754: (1985), Institute of Electrical and Electronics Engineers, New York. Reprinted in SIGPLAN Notices, 22, (2), 9-25. [45] Kantorovich L.V. and Krylov V.I. (1958): Approximate Methods of Higher Analysis, Interscience, New York. [46] Kato T. (1960): Estimation of iterated matrices with application to the von Neumann condition, Numer. Math., 2, 22-29. [47] Kato T. (1976): Perturbation Theory for Linear Operators, Second Edition, Springer-Verlag, Berlin. [48] Kreyszig E. (1989): Introductory Functional Analysis with Applications, John Wiley and Sons, New York. [49] Kulkarni R.P. and Limaye B.V. (1989): On the error estimates for the Rayleigh-Schrodinger series and the Kato-Rellich perturbation series, J. Austral. Math. Soc. Ser. A, 46, 456-468.

References

377

[50] Kulkarni R.P. and Limaye B.V. (1990): Solution of a Schrodinger equation by iterative re nement, J. Austral. Math. Soc. Ser. B, 32, 115-132. [51] Largillier A. (1996): Bounds for relative errors of complex matrix factorizations, Appl. Math. Lett., 6, 79-84. [52] Largillier A. and Levet M. (1993): A note on a collectively compact approximation for weakly singular integral operators, Appl. Math. Lett., 4, 87-90. [53] Largillier A. and Limaye B.V. (1996): Finite-rank methods and their stability for coupled systems of operator equations, SIAM J. Numer. Anal., 33, 707-728. [54] Limaye B.V. (1987): Spectral Perturbation and Approximation with Numerical Experiments, Proceedings of the Centre for Mathematical Analysis, Vol. 13, Austral. Nat. Univ., Canberra. [55] Limaye B.V. (1996): Functional Analysis, Second Edition, New Age International, New Delhi. [56] Limaye B.V. and Nair M.T. (1986): Rayleigh-Schrodinger procedure for iterative re nement of computed eigenelements under strong approximation, in the Proceedings of the International Conference on `Methods of Functional Analysis in Approximation Theory', (C.A. Micchelli, D.V. Pai, B.V. Limaye, eds.), International Series of Numerical Mathematics, Vol. 76, 371-388, Birkhauser Verlag, Basel. [57] Limaye B.V. and Nair M.T. (1989): On the accuracy of the Rayleigh-Schrodinger approximations, J. Math. Anal. Appl., 139, 413-431. [58] Limaye B.V. and Nair M.T. (1990): Eigenelements of perturbed operators, J. Austral. Math. Soc. Ser. A, 49, 138-148. [59] Limaye B.V. and Nair M.T. (1995): On the multiplicities and the ascent of an eigenvalue of a linear map, Math. Student, 64, 162-166. [60] Michael E. (1951): Topologies on spaces of subsets, Trans. Amer. Math. Soc., 71, 152-182. [61] Nair M.T. (1992): On strongly stable approximations, J. Austral. Math. Soc. Ser. A, 52, 251-260.

378

References

[62] Osborn J.E. (1975): Spectral approximation for compact operators, Math. Comp., 29, 712-725. [63] Parlett B.N. (1980): The Symmetric Eigenvalue Problem, PrenticeHall, Englewood Clis, N.J. [64] Redont P. (1979): Application de la theorie de la perturbation des operateurs lineaires a l'obtention de bornes d'erreur sur les elements propres et a leur calcul, These de docteur-ingenieur, Universite de Grenoble I, France. [65] Schumaker L.L. (1981): Spline Functions: Basic Theory, John Wiley and Sons, New York. [66] Sloan, I.H. (1976): Convergence of degenerate kernel methods, J. Austral. Math. Soc., Ser. B, 19, 422-431. [67] Sloan I.H. (1976): Iterated Galerkin method for eigenvalue problems, SIAM J. Numer. Anal., 13, 753-760. [68] Vainikko G.M. (1969): The compact approximation principle in the theory of approximation methods (in Russian), Z. Vycisl. Mat. i Mat. Fiz., 9, 739-761. [69] Vainikko G.M. (1977): U ber die Konvergenz und Divergenz von Naherungs-methoden bei Eigenwert Problemen, Math. Nachr., 78, 145-164. [70] Wang J.Y. (1976): On the numerical computation of eigenvalues and eigenfunctions of compact integral operators using spline functions, J. Inst. Maths Applics, 18, 177-188. [71] Watkins D.S. (1991): Fundamentals of Matrix Computations, John Wiley and Sons, New York. [72] Watson G.A. (1980): Approximation Theory and Numerical Methods, John Wiley and Sons, Chichester. [73] Whitley R. (1986): The stability of nite rank methods with applications to integral equations, SIAM J. Numer. Anal., 23, 118-134. [74] Wilkinson J.H. (1965): The Algebraic Eigenvalue Problem, Oxford Univ. Press, London.

Index

1-norm, 51

Cauchy's Integral Formul, 13 Cauchy's Theorem, 13 Cauchy's Theorem for Multiply Connected Domains, 13 Cayley-Hamilton Theorem, 39 characteristic equation, 3 characteristic polynomial, 3 Cholesky factor, 319 chopping, 349 collectively compact convergence, 71 collocation approximation, 191 compact operator, 40 companion matrix, 325 complex rotation, 322 Compound Gauss Two-Point Rule, 215 Compound Mid-Point Rule, 215 Compound Simpson Rule, 216 Compound Trapezoidal Rule, 215 condition number, 344 conjugate Jordan curve, 43 conjugate-linear functional, 41 conjugate-transpose matrix, 5 continuity of the spectrum, 70 convergent sequence of quadrature formul, 203 coordinate matrix, 2

 -convergence, 72 F-norm, 49 1-norm, 51 2-norm, 51

a posteriori error estimates, 231 acceleration, 114 adjoint matrix, 5 adjoint operator, 41 adjoint space, 41 algebraic multiplicity, 34 analytic, 12 annihilator, 42 arithmetic basis, 348 ascent, 36 B-splines, 195 backward error, 347 Banach space, 9 Basic QR Method, 336 Bernstein polynomial, 202 block reduced resolvent operator, 61 block resolvent operator, 58 bounded linear operator, 9 Cauchy contour, 24 Cauchy domain, 24 379

380 cubic spline function with knots, 194 decomposition, 21 defective, 36 de ation, 362 degenerate, 200 degenerate kernel approximation, 200 derivative, 12 domain, 13 dominant spectral value, 111 Double Iteration, 117 eigenequation, 2 eigenspace, 2 eigenvalue, 1 eigenvector, 2 elementary Cauchy domain, 24 Elementary Iteration, 115 exterior of a Cauchy contour, 25 exterior of a Jordan curve, 12 nite element approximation, 198 First Neumann Expansion, 16 First Resolvent Identity, 11 Fixed Slope Newton Method, 116

oating point function, 348

oating point number, 348 forward error, 344 Fourier Expansion, 46 Frechet derivative, 116 Fredholm approximation, 204 Fredholm integral operator, 199 Frobenius norm, 49 Galerkin approximation, 186 gap, 93 Gauss-Legendre Rule, 216 Gelfand-Mazur Theorem, 18 generalized eigenvalue problem, 197

INDEX

generalized eigenvector, 35 geometric multiplicity, 2 grade of the generalized eigenvector, 35 Gram matrix, 50 Gram product, 50 Gram-Schmidt Process, 45 Green Function, 303 Haar functions, 190 hat functions, 193 Hermitian matrix, 5 higher order spectral approximation, 149 Hilbert space, 44 Householder symmetry, 319 identity matrix, 3 identity operator, 1 inner product, 3 interior of a Cauchy contour, 25 interior of a Jordan curve, 12 interpolatory projection, 191 Inverse Iteration Algorithm, 335 Inverse Power Method, 114 Inverse Power Method with shift, 114 irreducible upper Hessenberg matrix, 324 iterative re nement, 113 Jordan canonical form, 40 Jordan curve, 12 Kantorovich-Krylov approximation, 224 Krylov sequence, 331 Lagrange interpolatory projection, 195 Lagrange polynomials, 195 Laurent Expansion, 16

INDEX

Laurent's Theorem, 14 left eigenvector, 334 left shift operator, 19 Legendre polynomials, 190 length of a Cauchy contour, 82 Liouville's Theorem, 15 lower Hessenberg matrix, 324 lower semicontinuity of the spectrum, 70 lower triangular matrix, 3 mesh, 192 modulus of continuity, 209

381 power series, 14 precision, 348 projection, 21 projection approximation, 186 Property L, 70 Property U, 70 QR Factorization, 317 QR factors, 318 QR Method, 336 quadrature formula, 203 quasinilpotent, 33 quasiuniform partitions, 194

orthogonal, 3 orthogonal complement, 44 orthogonal projection, 45 orthonormal, 3 orthonormal basis, 45 over ow limit, 348

radius of convergence, 14 rate of convergence, 333 Rectangular Rule, 209 recti able, 12 reduced resolvent operator, 32 relative residual, 303 relatively compact, 40 residual, 115 resolvent operator, 9 resolvent set, 9 Riemann sum, 209 Riesz Representation Theorem, 45 right shift operator, 106 rounding, 349

piecewise quadratic interpolatory projection, 242 pointwise convergence, 70 Polar Factorization, 6 Polarization Identity, 44 Polya's Theorem, 203 polynomial eigenvalue problem, 149 positive de nite, 5 positively oriented, 12 Power Method, 114

saw-tooth functions, 189 Schauder basis, 187 Schur form, 5 Schur's Theorem, 4 Schwarz Inequality, 44 Second Neumann Expansion, 77 Second Resolvent Identity, 77 selfadjoint operator, 46 semisimple, 36 separable space, 46 separation between matrices, 369

natural extension, 49 nilpotent, 33 nodes, 190 norm, 9 norm convergence, 70 normal matrix, 5 normal operator, 46 Nystrom approximation, 204

382 sesquilinear functional, 197 shift, 333 simple curve, 12 simple eigenvalue, 36 simply connected, 13 Simultaneous Inverse Iteration, 342 Singularity Subtraction Technique, 224 Sloan approximation, 186 Spectral Decomposition Theorem, 30 spectral projection, 31 spectral radius, 10 spectral set, 23 spectral set of nite type, 36 spectral subspace, 31 spectral value, 10 spectral value of nite type, 34 spectrum, 10 square root, 5 stability, 348 standard basis, 316 strongly coercive, 197 Sturm-Liouville Problem, 303 subordinated operator norm, 9 Sylvester equation, 56 Taylor Expansion, 16 Taylor's Theorem, 14 trigonometric functions, 190 under ow limit, 348 unit roundo, 348 unitary factor map, 318 unitary matrix, 5 unitary operator, 46 upper Hessenberg matrix, 324 upper semicontinuity of the spectrum, 70

INDEX

upper triangular block diagonal form, 38 upper triangular factor map, 318 upper triangular matrix, 3 weakly posed generalized eigenvalue problem, 198 weakly singular, 222 weights, 203