A Polynomial Approach to Linear Algebra (Second Edition)

Universitext

Universitext Series Editors: Sheldon Axler San Francisco State University Vincenzo Capasso Universit`a degli Studi di Milano Carles Casacuberta Universitat de Barcelona Angus J. MacIntyre Queen Mary, University of London Kenneth Ribet University of California, Berkeley Claude Sabbah ´ CNRS, Ecole Polytechnique Endre S¨uli University of Oxford Wojbor A. Woyczynski Case Western Reserve University

Universitext is a series of textbooks that presents material from a wide variety of mathematical disciplines at master’s level and beyond. The books, often well class-tested by their author, may have an informal, personal even experimental approach to their subject matter. Some of the most successful and established books in the series have evolved through several editions, always following the evolution of teaching curricula, to very polished texts. Thus as research topics trickle down into graduate-level teaching, first textbooks written for new, cutting-edge courses may make their way into Universitext.

For further volumes: http://www.springer.com/series/223

Paul A. Fuhrmann

A Polynomial Approach to Linear Algebra Second Edition

123

Paul A. Fuhrmann Ben-Gurion University of the Negev Beer Sheva Israel

ISSN 0172-5939 e-ISSN 2191-6675 ISBN 978-1-4614-0337-1 e-ISBN 978-1-4614-0338-8 DOI 10.1007/978-1-4614-0338-8 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2011941877 Mathematics Subject Classification (2010): 15-02, 93-02 © Springer Science+Business Media, LLC 2012 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

To Nilly

Preface

Linear algebra is a well-entrenched mathematical subject that is taught in virtually every undergraduate program, both in the sciences and in engineering. Over the years, many texts have been written on linear algebra, and therefore it is up to the author to justify the presentation of another book in this area to the public. I feel that my jusification for the writing of this book is based on a different choice of material and a different approach to the classical core of linear algebra. The main innovation in it is the emphasis placed on functional models and polynomial algebra as the best vehicle for the analysis of linear transformations and quadratic forms. In pursuing this innovation, a long standing trend in mathematics is being reversed. Modern algebra went from the specific to the general, abstracting the underlying unifying concepts and structures. The epitome of this trend was represented by the Bourbaki school. No doubt, this was an important step in the development of modern mathematics, but it had its faults too. It led to several generations of students who could not compute, nor could they give interesting examples of theorems they proved. Even worse, it increased the gap between pure mathematics and the general user of mathematics. It is the last group, made up of engineers and applied mathematicians, which is concerned not only in understanding a problem but also in its computational aspects. A very similar development occurred in functional analysis and operator theory. Initially, the axiomatization of Banach and Hilbert spaces led to a search for general methods and results. While there were some significant successes in this direction, it soon became apparent, especially in trying to understand the structure of bounded operators, that one has to be much more specific. In particular, the introduction of functional models, through the work of Livsic, Beurling, Halmos, Lax, de Branges, Sz.-Nagy and Foias, provided a new approach to structure theory. It is these ideas that we have taken as our motivation in the writing of this book. In the present book, at least where the structure theory is concerned, we look at a special class of shift operators. These are defined using polynomial modular arithmetic. The interesting fact about this class is its property of universality, in the

vii

viii

Preface

sense that every cyclic operator is similar to a shift and every linear operator on a finite-dimensional vector space is similar to a direct sum of shifts. Thus, the shifts are the building blocks of an arbitrary linear operator. Basically, the approach taken in this book is a variation on the study of a linear transformation via the study of the module structure induced by it over the ring of polynomials. While module theory provides great elegance, it is also difficult to grasp by students. Furthermore, it seems too far removed from computation. Matrix theory seems to be at the other extreme, that is, it is too much concerned with computation and not enough with structure. Functional models, especially the polynomial models, lie on an intermediate level of absraction between module theory and matrix theory. The book includes specific chapters devoted to quadratic forms and the establishment of algebraic stability criteria. The emphasis is shared between the general theory and the specific examples, which are in this case the study of the Hankel and Bezout forms. This general area, via the work of Hermite, is one of the roots of the theory of Hilbert spaces. I feel that it is most illuminating to see the Euclidean algorithm and the associated Bezout identity not as isolated results, but as an extremely effective tool in the development of fast inversion algorithms for structured matrices. Another innovation in this book is the inclusion of basic system-theoretic ideas. It is my conviction that it is no longer possible to separate, in a natural way, the study of linear algebra from the study of linear systems. The two topics have benefited greatly from cross-fertilization. In particular, the theory of finite-dimensional linear systems seems to provide an unending flow of problems, ideas, and concepts that are quickly assimilated in linear algebra. Realization theory is as much a part of linear algebra as is the long familiar companion matrix. The inclusion of a whole chapter on Hankel norm approximation theory, or AAK theory as it is commonly known, is also a new addition as far as linear algebra books are concerned. This part requires very little mathematical knowledge not covered in the book, but a certain mathematical maturity is assumed. I believe it is very much within the grasp of a well-motivated undergraduate. In this part several results from early chapters are reconstructed in a context in which stability is central. Thus the rational Hardy spaces enter, and we have analytic models and shifts. Lagrange and Hermite interpolation are replaced by Nevanlinna-Pick interpolation. Finally, coprimeness and the Bezout identity reappear, but over a different ring. I believe the study of these analogies goes a long way toward demonsrating to the student the underlying unity of mathematics. Let me explain the philosophy that underlies the writing of this book. In a way, I share the aim of Halmos (1958) in trying to treat linear transformations on finitedimensional vector spaces by methods of more general theories. These theories were functional analysis and operator theory in Hilbert space. This is still the case in this book. However, in the intervening years, operator theory has changed remarkably. The emphasis has moved from the study of self-adjoint and normal operators to the study of non-self-adjoint operators. The hope that a general structure theory for linear operators might be developed seems to be too naive. The methods utilizing

Preface

ix

Riesz-Dunford integrals proved to be too restrictive. On the other hand, a whole new area centering on the theory of invariant subspaces and the construction and study of functional models was developed. This new development had its roots not only in pure mathematics but also in many applied areas, notably scattering, network and control theories, and some areas of stochastic processes such as estimation and prediction theories. I hope that this book will show how linear algebra is related to other, more advanced, areas of mathematics. Polynomial models have their root in operator theory, especially that part of operator theory that centered on invariant subspace theory and Hardy spaces. Thus the point of view adopted here provides a natural link with that area of mathematics, as well as those application areas I have already mentioned. In writing this book, I chose to work almost exclusively with scalar polynomials, the one exception in this project being the invariant factor algorithm and its application to structure theory. My choice was influenced by the desire to have the book accessible to most undergraduates. Virtually all results about scalar polynomial models have polynomial matrix generalizations, and some of the appropriate references are pointed out in the notes and remarks. The exercises at the end of chapters have been chosen partly to indicate directions not covered in the book. I have refrained from including routine computational problems. This does not indicate a negative attitude toward computation. Quite to the contrary, I am a great believer in the exercise of computation and I suggest that readers choose, and work out, their own problems. This is the best way to get a better grasp of the presented material. I usually use the first seven chapters for a one-year course on linear algebra at Ben Gurion University. If the group is a bit more advanced, one can supplement this by more material on quadratic forms. The material on qudratic forms and stability can be used as a one-semester course of special topics in linear algebra. Also, the material on linear systems and Hankel norm approximations can be used as a basis for either a one term course or a seminar. Paul A. Fuhrmann

Preface to the Second Edition

Linear algebra is one of the most active areas of mathematics, and its importance is ever increasing. The reason for this is, apart from its intrinsic beauty and elegance, its usefulness to a large array of applied areas. This is a two-way road, for applications provide a great stimulus for new research directions. However, the danger of a tower-of-Babel phenomenon is ever present. The broadening of the field has to confront the possibility that, due to differences in terminology, notation, and concepts, the communication between different parts of linear algebra may break down. I strongly believe, based on my long research in the theory of linear systems, that the polynomial techniques presented in this book provide a very good common ground. In a sense, the presentation here is just a commercial for subsequent publications stressing extensions of the scalar techniques to the context of polynomial and rational matrix functions. Moreover, in the fifteen years since the original publication of this book, my perspective on some of the topics has changed. This, at least partially, is due to the mathematical research I was doing during that period. The most significant changes are the following. Much greater emphasis is put on interpolation theory, both polynomial and rational. In particular, we also approach the commutant lifting theorem via the use of interpolation. The connection between the Chinese remainder theorem and interpolation is explained, and an analytic version of the theorem is given. New material has been added on tensor products, both of vector spaces and of modules. Because of their importance, special attention is given to the tensor products of quotient polynomial modules. In turn, this leads to a conceptual clarification of the role of Bezoutians and the Bezout map in understanding the difference between the tensor products of functional models taken with respect to the underlying field and those taken with respect to the corresponding polynomial ring. This enabled the introduction of some new material on model reduction. In particular, some connections between the polynomial Sylvester equation and model reduction techniques, related to interpolation on the one hand and projection methods on the other, are clarified. In the process of adding material, I also tried to streamline theorem statements and proofs and generally enhance the readability of the book. It is my hope that this effort was at least partially successful. xi

xii

Preface to the Second Edition

I am greatly indebted to my friends and colleagues Uwe Helmke and Abie Feintuch for reading parts of the manuscript and making useful suggestions and to Harald Wimmer for providing many useful references to the history of linear algebra. Special thanks to my beloved children, Amir, Oded, and Galit, who not only encouraged and supported me in the effort to update and improve this book, but also enlisted the help of their friends to review the manuscript. To these friends, Shlomo Hoory, Alexander Ivri, Arie Matsliah, Yossi Richter, and Patrick Worfolk, go my sincere thanks. Paul A. Fuhrmann

Contents

1

Algebraic Preliminaries .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.2 Sets and Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.3 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4 Rings and Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.1 The Integers .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.2 The Polynomial Ring . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.3 Formal Power Series. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.4 Rational Functions .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.5 Proper Rational Functions.. . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.6 Stable Rational Functions . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.7 Truncated Laurent Series . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.5 Modules .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.6 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.7 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

1 1 1 3 8 11 11 20 22 23 24 25 26 31 32

2

Vector Spaces.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.2 Vector Spaces .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.3 Linear Combinations .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.4 Subspaces .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.5 Linear Dependence and Independence . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.6 Subspaces and Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.7 Direct Sums. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.8 Quotient Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.9 Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.10 Change of Basis Transformations . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.11 Lagrange Interpolation .. . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.12 Taylor Expansion .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.13 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.14 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

33 33 33 36 36 37 40 41 43 45 46 48 51 52 52 xiii

xiv

Contents

3

Determinants .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.2 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.3 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.4 The Sylvester Resultant .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.5 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.6 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

55 55 55 60 62 64 66

4

Linear Transformations.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.2 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.3 Matrix Representations . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.4 Linear Functionals and Duality . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.5 The Adjoint Transformation . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.6 Polynomial Module Structure on Vector Spaces .. . . . . . . . . . . . . . . . . . . 4.7 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.8 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

67 67 67 75 79 85 88 94 95

5

The Shift Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.2 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.3 Circulant Matrices .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.4 Rational Models .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.5 The Chinese Remainder Theorem and Interpolation . . . . . . . . . . . . . . . 5.5.1 Lagrange Interpolation Revisited . . . . . .. . . . . . . . . . . . . . . . . . . . 5.5.2 Hermite Interpolation.. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.5.3 Newton Interpolation .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.6 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.7 Universality of Shifts. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.8 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.9 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

97 97 97 109 111 118 119 120 121 122 127 131 133

6

Structure Theory of Linear Transformations . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.2 Cyclic Transformations . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.2.1 Canonical Forms for Cyclic Transformations .. . . . . . . . . . . . 6.3 The Invariant Factor Algorithm.. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.4 Noncyclic Transformations . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.5 Diagonalization .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.6 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.7 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

135 135 135 140 145 147 151 154 158

7

Inner Product Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.2 Geometry of Inner Product Spaces . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3 Operators in Inner Product Spaces . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.1 The Adjoint Transformation . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

161 161 161 166 166

Contents

xv

7.3.2 Unitary Operators.. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.3 Self-adjoint Operators . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.4 The Minimax Principle .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.5 The Cayley Transform.. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.6 Normal Operators.. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.7 Positive Operators . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.8 Partial Isometries . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 7.3.9 The Polar Decomposition . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Singular Vectors and Singular Values . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Unitary Embeddings . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

169 173 176 176 178 180 182 183 184 187 190 193

8

Tensor Products and Forms . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.2 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.2.1 Forms in Inner Product Spaces. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.2.2 Sylvester’s Law of Inertia . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3 Some Classes of Forms . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3.1 Hankel Forms .. . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3.2 Bezoutians . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3.3 Representation of Bezoutians . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3.4 Diagonalization of Bezoutians . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3.5 Bezout and Hankel Matrices . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3.6 Inversion of Hankel Matrices . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.3.7 Continued Fractions and Orthogonal Polynomials.. . . . . . . 8.3.8 The Cauchy Index . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.4 Tensor Products of Models . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.4.1 Bilinear Forms . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.4.2 Tensor Products of Vector Spaces . . . . .. . . . . . . . . . . . . . . . . . . . 8.4.3 Tensor Products of Modules . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.4.4 Kronecker Product Models .. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.4.5 Tensor Products over a Field . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.4.6 Tensor Products over the Ring of Polynomials.. . . . . . . . . . . 8.4.7 The Polynomial Sylvester Equation . . .. . . . . . . . . . . . . . . . . . . . 8.4.8 Reproducing Kernels . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.4.9 The Bezout Map . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.5 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 8.6 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

195 195 196 196 199 204 205 209 213 217 223 230 237 248 254 254 255 259 260 261 263 266 270 272 274 277

9

Stability .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.2 Root Location Using Quadratic Forms .. . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.3 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.4 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

279 279 279 293 294

7.4 7.5 7.6 7.7

xvi

Contents

10 Elements of Linear System Theory .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.2 Systems and Their Representations . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.3 Realization Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.4 Stabilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.5 The Youla–Kucera Parametrization . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.6 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.7 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

295 295 296 300 314 319 321 324

11 Rational Hardy Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2 Hardy Spaces and Their Maps . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2.1 Rational Hardy Spaces . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2.2 Invariant Subspaces.. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2.3 Model Operators and Intertwining Maps .. . . . . . . . . . . . . . . . . 11.2.4 Intertwining Maps and Interpolation . .. . . . . . . . . . . . . . . . . . . . 11.2.5 RH∞ + -Chinese Remainder Theorem . . .. . . . . . . . . . . . . . . . . . . . 11.2.6 Analytic Hankel Operators and Intertwining Maps .. . . . . . 11.3 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.4 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

325 325 326 326 334 337 344 353 354 358 359

12 Model Reduction.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.1 Introduction .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2 Hankel Norm Approximation .. . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2.1 Schmidt Pairs of Hankel Operators .. . .. . . . . . . . . . . . . . . . . . . . 12.2.2 Reduction to Eigenvalue Equation .. . . .. . . . . . . . . . . . . . . . . . . . 12.2.3 Zeros of Singular Vectors and a Bezout equation .. . . . . . . . 12.2.4 More on Zeros of Singular Vectors . . . .. . . . . . . . . . . . . . . . . . . . 12.2.5 Nehari’s Theorem.. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2.6 Nevanlinna–Pick Interpolation .. . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2.7 Hankel Approximant Singular Values and Vectors . . . . . . . 12.2.8 Orthogonality Relations . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2.9 Duality in Hankel Norm Approximation . . . . . . . . . . . . . . . . . . 12.3 Model Reduction: A Circle of Ideas. . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.3.1 The Sylvester Equation and Interpolation . . . . . . . . . . . . . . . . . 12.3.2 The Sylvester Equation and the Projection Method.. . . . . . 12.4 Exercises .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.5 Notes and Remarks .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .

361 361 362 363 369 370 376 378 379 381 383 385 392 392 394 397 400

References .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 403 Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 407

Chapter 1

Algebraic Preliminaries

1.1 Introduction This book emphasizes the use of polynomials, and more generally, rational functions, as the vehicle for the development of linear algebra and linear system theory. This is a powerful and elegant idea, and the development of linear theory is leaning more toward the conceptual than toward the technical. However, this approach has its own weakness. The stumbling block is that before learning linear algebra, one has to know the basics of algebra. Thus groups, rings, fields, and modules have to be introduced. This we proceed to do, accompanied by some examples that are relevant to the content of the rest of the book.

1.2 Sets and Maps Let S be a set. If between elements of the set a relation a  b is defined, so that either a  b holds or not, then we say we have a binary relation. If a binary relation in S satisfies the following conditions: 1. a  a holds for all a ∈ S, 2. a  b ⇒ b  a, 3. a  b and b  c ⇒ a  c, then we say we have an equivalence relation in S. The three conditions are referred to as reflexivity, symmetry, and transitivity respectively. For each a ∈ S we define its equivalence class by Sa = {x ∈ S | x  a}. Clearly Sa ⊂ S and Sa = 0. / An equivalence relation leads to a partition of the set S. By a partition of S we mean a representation of S as the disjoint union of subsets. Since clearly, using transitivity, either Sa ∩ Sb = 0/ or Sa = Sb , and S = ∪a∈S Sa , the set of

P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 1, © Springer Science+Business Media, LLC 2012

1

2

1 Algebraic Preliminaries

equivalence classes is a partition of S. Similarly, any partition S = ∪α Sα defines an equivalence relation by letting a  b if for some α we have a, b ∈ Sα . A rule that assigns to each member a ∈ A a unique member b ∈ B is called a f

map or a function from A into B. We will denote this by f : A −→ B or A −→ B. We denote by f (A) the image of the set A defined by f (A) = {y | y ∈ B, ∃x ∈ A s.t. y = f (x)}. The inverse image of a subset M ⊂ B is defined by f −1 (M) = {x | x ∈ A, f (x) ∈ M}. A map f : A −→ B is called injective or 1-to-1 if f (x) = f (y) implies x = y. A map f : A −→ B is called surjective or onto if f (A) = B, i.e., for each y ∈ B there exists an x ∈ A such that y = f (x). A map f is called bijective if it is both injective and surjective. Given maps f : A −→ B and g : B −→ C, we can define a map h : A −→ C by letting h(x) = g( f (x)). We call this map h the composition or product of the maps f f

g

h

and g. This will be denoted by h = g ◦ f . Given three maps A −→ B −→ C −→ D, we compute h ◦ (g ◦ f )(x) = h(g( f (x))) and (h ◦ g) ◦ f (x) = h(g( f (x))). So the product of maps is associative, i.e., h ◦ (g ◦ f ) = (h ◦ g) ◦ f . Due to the associative law of composition, we can write h ◦ g ◦ f , and more generally fn ◦ · · · ◦ f1 , unambiguously. Given a map f : A −→ B, we define an equivalence relation  in A by letting x1  x2 ⇔ f (x1 ) = f (x2 ). Thus the equivalence class of a is given by Aa = {x | x ∈ A, f (x) = f (a)}. We will denote by A/  the set of equivalence classes and refer to this as the quotient set by the equivalence relation. Next we define three transformations f

f

f

1 2 3 A/R −→ f (A) −→ B A −→

with the fi defined by f1 (a) = Aa , f2 (Aa ) = f (a), f 3 (b) = b, b ∈ f (A). Clearly the map f1 is surjective, f2 is bijective and f 3 injective. Moreover, we have f = f3 ◦ f2 ◦ f1 . This factorization of f is referred to as the canonical factorization. The canonical factorization can be described also via the following commutative diagram:

1.3 Groups

3

A

f

- B 6 f3

f1 ? A/R

f2

- f (A)

We note that f2 ◦ f1 is surjective, whereas f3 ◦ f2 is injective.

1.3 Groups Given a set M, a binary operation in M is a map from M × M into M. Thus, an ordered pair (a, b) is mapped into an element of M denoted by ab. A set M with an associative binary operation is called a semigroup. Thus if a, b ∈ M we have ab ∈ M, and the associative rule a(bc) = (ab)c holds. Thus the product a1 · · · an of elements of M is unambiguously defined. We proceed to define the notion of a group, which is the cornerstone of most mathematical structures. Definition 1.1. A group is a set G with a binary operation, called multiplication, that satisfies 1. a(bc) = (ab)c, i.e., the associative law. 2. There exists a left identity, or unit element, e ∈ G, i.e., ea = a for all a ∈ G. 3. For each a ∈ G there exists a left inverse, denoted by a−1 , which satisfies a−1 a = e. A group G is called abelian if the group operation is commutative, i.e., if ab = ba holds for all a, b ∈ G. In many cases, an abelian group operation will be denoted using the additive notation, i.e., a + b rather than ab, as in the case of the group of integers Z with addition as the group operation. Other useful examples are R, the set of all real numbers under addition, and R+ , the set of all positive real numbers with multiplication as the group operation. Given a nonempty set S, the set of all bijective mappings of S onto itself forms a group with the group action being composition. The elements of G are called permutations of S. If S = {1, . . . , n}, then the group of permutations of S is called the symmetric group of degree n and denoted by Sn .

4

1 Algebraic Preliminaries

Theorem 1.2. 1. Let G be a group and let a be an element of G. Then a left inverse a−1 of a is also a right inverse. 2. A left identity is also a right identity. 3. The identity element of a group is unique. Proof. 1. We compute (a−1 )−1 a−1 aa−1 = ((a−1 )−1 a−1 )(aa−1 ) = e(aa−1 ) = aa−1 = (a−1 )−1 (a−1 a)a−1 = (a−1 )−1 (ea−1 ) = (a−1 )−1 a−1 = e. So in particular, aa−1 = e. 2. Let a ∈ G be arbitrary and let e be a left identity. Then aa−1 a = a(a−1 a) = ae = (aa−1)a = ea = a. Thus ae = a for all a. So e is also a right identity. 3. Let e, e be two identities in G. Then, using the fact that e is a left identity and e a right identity, we get e = ee = e .  In a group G, equations of the form axb = c are easily solvable, with the solution given by x = a−1 cb−1 . Also, it is easily checked that we have the following rule for inversion: −1 (a1 · · · an )−1 = a−1 n · · · a1 . Definition 1.3. A subset H of a group G is called a subgroup of G if it is a group with the composition rule inherited from G. Thus H is a subgroup if with a, b ∈ H, we have ab ∈ H and a−1 ∈ H. This can be made a bit more concise. Lemma 1.4. A subset H of a group G is a subgroup if and only if with a, b ∈ H, also ab−1 ∈ H. Proof. If H is a subgroup of G, then with a, b ∈ H, it contains also b−1 and hence also ab−1 . Conversely, if a, b ∈ H implies ab−1 ∈ H, then b−1 = eb−1 ∈ H and hence also ab = a(b−1 )−1 ∈ H, i.e., H is a subgroup of G.  Given a subgroup H of a group G, we say that two elements a, b ∈ G are Hequivalent, and write a  b, if b−1 a ∈ H. It is easily checked that this is a bona fide equivalence relation in G, i.e., it is a reflexive, symmetric, and transitive relation. We denote by Ga the equivalence class of a, i.e., Ga = {x | x ∈ G, x  a} If we denote by aH the set {x | ah, h ∈ H}, then Ga = aH. We will refer to these as right equivalence classes or as right cosets. In a completely analogous way, left equivalence classes, or left cosets, Ha are defined.

1.3 Groups

5

Given a subgroup H of G, it is not usually the case that the sets of left and right cosets coincide. If this is the case, then we say that H is a normal subgroup of G. Assuming that a left coset aH is also a right coset Hb, we clearly have a = ae ∈ H and hence a ∈ Hb so necessarily Hb = Ha. Thus, for a normal subgroup, aH = Ha for all a ∈ G. Equivalently, H is normal if and only if for all a ∈ G we have aHa−1 = H. Given a subgroup H of a group G, then any two cosets can be bijectively mapped onto each other. In fact, the map φ (ah) = bh is such a bijection between aH and bH. In particular, cosets aH all have the same cardinality as H. We will define the index of a subgroup H in G, and denote it by iG (H), as the number of left cosets. This will be denoted by [G : H]. Given the trivial subgroup E = {e} of G, the left and right cosets consist of single elements. Thus [G : E], is just the number of elements of the group G. The number [G : E] will be refered to also as the order o(G) of the group G. We can proceed now to study the connection between index and order. Theorem 1.5 (Lagrange). Given a subgroup H of a group G, we have [G : H][H : E] = [G : E], or o(G) = iG (H)o(H).  Homomorphisms are maps that preserve given structures. In the case of groups G1 and G2 , a map φ : G1 −→ G2 is called a group homomorphism if, for all g1 , g2 ∈ G1 , we have φ (g1 g2 ) = φ (g1 )φ (g2 ). Lemma 1.6. Let G1 and G2 be groups with unit elements e and e respectively. Let φ : G1 −→ G2 be a homomorphism. Then 1. φ (e) = e . 2. φ (x−1 ) = (φ (x))−1 . Proof. 1. For any x ∈ G1 , we compute φ (x)e = φ (x) = φ (xe) = φ (x)φ (e). Multiplying by φ (x)−1 , we get φ (e) = e . 2. We compute e = φ (e) = φ (xx−1 ) = φ (x)φ (x−1 ). This shows that φ (x−1 ) = (φ (x))−1 .



A homomorphism φ : G1 −→ G2 that is bijective is called an isomorphism. In this case G1 and G2 will be called isomorphic. An example of a nontrivial isomorphism is the exponential function x → ex , which maps R (with addition as the operation) isomorphically onto R+ (with multiplication as the group operation).

6

1 Algebraic Preliminaries

The general canonical factorization of maps discussed in Section 1.2 can now be applied to the special case of group homomorphisms. To this end we define, for a homomorphism φ : G1 −→ G2 , the kernel and image of φ by Ker φ = φ −1 {e } = {g ∈ G1 | φ (g) = e }, and

Im φ = φ (G1 ) = {g ∈ G2 | ∃g ∈ G1 , φ (g) = g }.

The kernel of a group homomorphism has a special property. Lemma 1.7. Let φ : G −→ G be a group homomorphism and let N = Ker φ . Then N is a normal subgroup of G. Proof. Let x ∈ G and n ∈ N. Then

φ (xnx−1 ) = φ (x)φ (n)φ (x−1 ) = φ (x)e φ (x−1 ) = φ (x)φ (x−1 ) = e . So xnx−1 ∈ N, i.e., xNx−1 ⊂ N for every x ∈ G. This implies N ⊂ x−1 Nx. Since this inclusion holds for all x ∈ G, we get N = xNx−1 , or Nx = xN, i.e., the left and right cosets are equal. So N is a normal subgroup.  Note now that given x, y ∈ G and a normal subgroup N of G, we can define a product in the set of all cosets by xN · yN = xyN.

(1.1)

Theorem 1.8. Let N ⊂ G be a normal subgroup. Denote by G/N the set of all cosets and define the product of cosets by (1.1). Then G/N is a group called the factor group, or quotient group, of G by N. Proof. Clearly (1.1) shows that G/N is closed under multiplication. To check associativity, we note that (xN · yN) · zN = (xyN) · zN = (xy)zN = x(yz)N = xN · (yzN) = xN · (yN · zN). For the unit element e of G we have eN = N, and eN · xN = (ex)N = xN. So eN = N is the unit element in G/N. Finally, given x ∈ G, we check that the inverse element is (xN)−1 = x−1 N.  Theorem 1.9. N is a normal subgroup of the group G if and only if N is the kernel of a group homomorphism.

1.3 Groups

7

Proof. By Lemma 1.7, it suffices to show that if N is a normal subgroup, then it is the kernel of a group homomorphism. We do this by constructing such a homomorphism. Let G/N be the factor group of G by N. Define π : G −→ G/N by

π (g) = gN.

(1.2)

Clearly (1.1) shows that π is a group homomorphism. Moreover, π (g) = N if and only if gN = N, and this holds if and only if g ∈ N. Thus Ker π = N.  The map π : G −→ G/N defined by (1.2) is called the canonical projection. A homomorphism φ whose kernel contains a normal subgroup can be factored through the factor group. Proposition 1.10. Let φ : G −→ G be a homomorphism with Ker φ ⊃ N, where N is a normal subgroup of G. Let π be the canonical projection of G onto G/N. Then there exists a unique homomorphism φ : G/N −→ G for which φ = φ ◦ π , or equivalently, the following diagram is commutative:

φ

- G 

G

π

φ ? G/N

Proof. We define a map φ : G/N −→ G by φ (xN) = φ (x). This map is well defined, since Ker φ ⊃ N. It is a homomorphism because φ is, and of course,

φ (x) = φ (xN) = φ (π (x)) = (φ ◦ π )(x). Finally, φ is uniquely defined by φ (xN) = φ (x).



We call φ the induced map by φ on G/N. As a corollary we obtain the following important result, the prototype of many others, which classifies images of group homomorphisms. Theorem 1.11. Let φ : G −→ G be a surjective group homomorphism with Ker φ = N. Then G is isomorphic to the factor group G/N. Proof. The induced map is clearly injective. In fact, if φ (xN) = e , we get φ (x) = e , or x ∈ Ker φ = N. It is also surjective by the assumption that φ is surjective. So we conclude that φ is an isomorphism. 

8

1 Algebraic Preliminaries

A group G is cyclic if all its elements are powers of one element a ∈ G, i.e., of the form an . In this case, a is called a generator of G. Clearly, Z is a cyclic group with addition as group operation. Defining nZ = {nk | k ∈ Z}, it is obvious that nZ is a normal subgroup of Z. We denote by Zn the quotient group Z/nZ. We can identify Zn with the set {0, . . . , n − 1} with the group operation being addition modulo n.

1.4 Rings and Fields Most of the mathematical structures that we will encounter in this book have, unlike groups, two operations, namely addition and multiplication. The simplest examples of these are rings and fields. These are introduced in this section. Definition 1.12. A ring R is a set with two laws of composition called addition and multiplication that satisfy, for all elements of R, the following. 1. Laws of addition: a. Associative law: a + (b + c) = (a + b) + c. b. Commutative law: a + b = b + a. c. Solvability of the equation a + x = b. 2. Laws of multiplication: a. Associative law: a(bc) = (ab)c. b. Distributive laws: a(b + c) = ab + ac, (b + c)a = ba + ca. 3. R is called a commutative ring if the commutative law ab = ba holds for all a, b ∈ R. Law 1 (c) implies the existence of a unique zero element, i.e., an element 0 ∈ R satisfying 0 + a = a + 0 = a for all a ∈ R. We call R a ring with identity if there exists an element 1 ∈ R such that for all a ∈ R, we have 1a = a1 = a. An element a in a ring with identity has a right inverse b if ab = 1 and a left inverse if ba = e. If a has both left and right inverses they must be equal, and then we say that a is invertible and denote its inverse by a−1 . A field is a commutative ring with identity in which every nonzero element is invertible. Definition 1.13. Let R1 and R2 be two rings. A ring homomorphism is a function φ : R1 −→ R2 that satifies

φ (x + y) = φ (x) + φ (y), φ (xy) = φ (x)φ (y). If R1 and R2 are rings with identities, then we require also φ (e) = e .

1.4 Rings and Fields

9

As in the case of groups, the kernel of a ring homomorphism φ : R1 −→ R2 is defined by Ker φ = φ −1 (0) = {x ∈ R1 | φ (x) = 0}. The kernel of a ring homomorphism has special properties. In fact, if x, y ∈ Ker φ and r ∈ R1 , then also x + y, rx ∈ Ker φ . This leads to the following definition. Definition 1.14. A subset J of a ring R is called a left ideal if x, y ∈ J and r ∈ R implies x + y, rx ∈ J. Thus, a subset J of R is a left ideal if it is an additive subgroup of R and RJ ⊂ J. If R contains an identity then RJ = J. For right ideals we replace the second condition by JR = J. A two-sided ideal, or just an ideal, is a subset of R that is both a left and right ideal. The sum of a finite number of ideals J1 , . . . , Jr in R is defined as the set J1 + · · · + Jr = {a1 + · · · + ar | ai ∈ Ji }. Proposition 1.15. Let R be a ring. Then 1. The sum of a finite number of left ideals is a left ideal. 2. The intersection of any set of left ideals in R is a left ideal. Proof. 1. Let J1 , . . . , Jk be left ideals in R. Then J = J1 + · · · + Jk = {a1 + · · · + ak | ai ∈ Ji }. Clearly if ai , bi ∈ Ji and r ∈ R, then k

k

k

i=1

i=1

i=1

k

k

i=1

i=1

∑ ai + ∑ bi = ∑ (ai + bi) ∈ J,

and

r ∑ ai = ∑ rai ∈ J. 2. Let J = ∩α Jα with Jα left ideals. If a, b ∈ J then a, b ∈ Jα for all α . Hence a + b ∈ Jα for all α , which implies a + b ∈ J. A similar argument holds to show that ra ∈ J.  Given a two-sided ideal J in a ring R, we can construct the quotient ring, denoted by R/J, whose elements are the cosets a + J by defining the operations of addition and multiplication by ⎧ ⎨ (a + J) + (b + J) = (a + b) + J, ⎩

(a + J)(b + J)

= ab + J.

It is easy to check that with the arithmetic operations so defined, R/J is indeed a ring. The following theorem gives a complete characterization of ideals. It is the counterpart, in the setting of rings, of Theorem 1.9. Theorem 1.16. Let R be a ring. A subset J ⊂ R is a two-sided ideal if and only if it is the kernel of a ring homomorphism.

10

1 Algebraic Preliminaries

Proof. We saw already that the kernel of a ring homomorphism is a two-sided ideal. Conversely, let J be a two-sided ideal. We define the canonical projection π : R −→ R/J by π (a) = a + J. It is easy to check that π is a surjective ring homomorphism, with Ker π = J.  An ideal in a ring R that is generated by a single element, i.e., of the form J = (d) = {rd | r ∈ R}, is called a principal ideal. The element d is called a generator of the ideal. More generally, given a1 , . . . , ak ∈ R, the set J = (ai , . . . , ak ) = {∑ki=1 ri ai | ri ∈ R} is obviously an ideal. We say that a1 , . . . , ak are generators of this ideal. In a ring R a nonzero element a is called a zero divisor if there exists another nonzero element b ∈ R such that ab = 0. A commutative ring without a zero divisors is called an entire ring or an integral domain. A commutative ring R with identity and no zero divisors is called a principal ideal domain, or PID for short, if every ideal in R is principal. In a ring R we have a division relation. If c = ab we say that a divides c or a is a left divisor or left factor of c. Given a1 , . . . , an ∈ R, we say that a is a common left divisor of the ai if it is a left divisor of all ai . We say that a is a greatest common left divisor, or g.c.l.d., if it is a common left divisor and is divisible by any other common left divisor. Two greatest common left divisors differ by a right factor that is invertible in R. We say that a1 , . . . , an ∈ R are left coprime if any greatest common left divisor is invertible in R. Right divisors are similarly defined. If R is a commutative ring, a greatest common left divisor is also a greatest common right divisor and will be refered to as a greatest common divisor, or a g.c.d. for short. Proposition 1.17. Let R be a principal ideal domain with identity. Then a1 , . . . , an ∈ R are coprime if and only if there exist bi ∈ R for which the Bezout identity a1 b 1 + · · · + a n b n = 1

(1.3)

holds. Proof. If there exist bi ∈ R for which the Bezout identity holds, then any common divisor of the ai is a divisor of 1, hence necessarily invertible. Conversely, we consider the ideal J generated by the ai . Since R is a principal ideal domain, J is generated by a single element d, which necessarily is invertible. So 1 ∈ J and hence there exist bi such that the Bezout identity (1.3) holds.  We present now a few examples of rings. We pay special attention to the ring of polynomials, due to the central role it plays in this book. Many of the results concerning ideals, factorizations, the Chinese remainder theorem, etc. hold also in the ring of integers. We do not give for those separate proofs nor, for the sake of concreteness, do we give proofs in the general context of Euclidean domains.

1.4 Rings and Fields

11

1.4.1 The Integers The set of integers Z is a commutative ring under the usual operations of addition and multiplication.

1.4.2 The Polynomial Ring A polynomial is an expression of the form n

p(z) = ∑ ai zi ,

ai ∈ F, 0 ≤ n ∈ Z.

i=0

The numbers ai are called the polynomial coefficients. We shall denote by F[z] the set of all polynomials with coefficients in the field F, i.e., F[z] = {∑ni=0 ai zi | ai ∈ F, 0 ≤ n ∈ Z}. Two polynomials are called equal if all their coefficients coincide. If p(z) = ∑ni=0 ai zi and an = 0, then we say that n is the degree of the polynomial p(z), which we denote by deg p. The coefficient an is called the leading coefficient of the polynomial p(z). We define the degree of the zero polynomial to be −∞. For a polynomial p(z) = ∑ni=0 ai zi , we assume ak = 0 for k > n = deg p. Given two i polynomials p(z) = ∑ni=0 ai zi and q(z) = ∑m i=0 bi z , we define their sum p(z)+q(z) by max(m,n)

∑

(p + q)(z) =

(ai + bi)zi .

(1.4)

i=0

The product, p(z)q(z), is defined by (pq)(z) =

m+n

∑ ci zi ,

(1.5)

i=0

where ci =

i

∑ a j bi− j .

(1.6)

j=0

It is easily checked that with these operations of addition and multiplication, F[z] is a commutative ring with an identity and has no zero divisors. The next theorem sums up the most elementary properties of polynomials. Theorem 1.18. Let p(z), q(z) be polynomials in F[z]. Then 1. deg(pq) = deg p + degq. 2. deg(p + q) ≤ max{deg p, deg q}. Proof. 1. If p(z) or q(z) is the zero polynomial, then both sides of the equality are equal to −∞. So we assume that both p(z) and q(z) are nonzero. Let

12

1 Algebraic Preliminaries

i p(z) = ∑ni=0 ai zi and q(z) = ∑m  0 i=0 bi z , with an , bm = 0. Then cn+m = an bm = but ck = 0 for k > m + n. 2. This is immediate. 

Note that the inequality deg(p + q) < max{deg p, deg q} may occur due to cancellations. Corollary 1.19. If p(z), q(z) ∈ F[z] and p(z)q(z) = 0, then p(z) = 0 or q(z) = 0. Proof. If p(z)q(z) = 0, then −∞ = deg pq = deg p + degq. So either deg p = −∞ or deg q = −∞.



In F[z], as in the ring of integers Z, we have a process of division with remainder. i Proposition 1.20. Given a nonzero polynomial p(z) = ∑m i=0 pi z with pm = 0, then an arbitrary polynomial q(z) ∈ F[z] can be written uniquely in the form

q(z) = a(z)p(z) + r(z)

(1.7)

with degr < deg p. Proof. If the degree of q(z) is less than the degree of p(z), then we write q(z) = 0 · p(z) + q(z) and this is the required representation. So we may assume degq = n ≥ m = deg p. The proof will proceed by induction on the degree of q(z). We assume that for all polynomials of degree less than n such a representation exists. Clearly, n−m q1 (z) = q(z) − qn p−1 p(z) m z

is a polynomial of degree ≤ n − 1. Hence by the induction hypothesis, q1 (z) = a1 (z)p(z) + r1 (z), with deg(r1 ) < deg(p). But this implies n−m + a1 (z))p(z) + r(z) = a(z)p(z) + r(z), q(z) = (qn p−1 m z

with

n−m a(z) = qn p−1 + a1 (z). m z

To show uniqueness, let q(z) = a1 (z)p(z) + r1 (z) = a2 (z)p(z) + r2 (z). This implies (a1 (z) − a2 (z))p(z) = r2 (z) − r1 (z).

1.4 Rings and Fields

13

A consideration of the degrees of both sides shows that, necessarily, they are both equal to zero. Hence, the uniqueness of the representation (1.7) follows.  The properties of the degree function in the ring F[z] can be abstracted to general rings. Definition 1.21. A ring R is called a Euclidean ring if there exists a function δ from the set of nonzero elements in R into the set of nonnegative integers that satisfies 1. For all a, b = 0, we have ab = 0 and δ (ab) ≥ δ (a). 2. For all f , g ∈ R, with g = 0, there exist a, r ∈ R such that f = ag + r and δ (r) < δ (g). We can define δ (0) = −∞. Obviously, with this definition, Proposition 1.20 implies that the ring of polynomials F[z] is a Euclidean ring. We note that in a Euclidean ring there are no zero divisors. It is convenient to have a notation for the remainder of a polynomial f (z) divided by q(z). If f (z) = a(z)q(z) + r(z) with deg r < deg q, we shall write r = πq f . We give several properties of the operation of taking a remainder. Lemma 1.22. Let q(z), a(z), b(z) ∈ F[z] with q(z) nonzero. Then

πq (aπq b) = πq (ab).

(1.8)

Proof. Let b(z) = b1 (z)q(z) + πq (z)b(z). Then a(z)b(z) = a(z)b1 (z)q(z) + a(z)πq b. Obviously, πq (ab1 q) = 0, and hence (1.8) follows.  Corollary 1.23. Given polynomials ai (z) ∈ F[z], i = 1, . . . , k, then

πq (a1 · · · ak ) = πq (a1 πq (a2 · · · πq (ak ) · · · )). 

Proof. By induction.

The following result simplifies in some important cases the computation of the remainder. Lemma 1.24. Let f (z), p(z), q(z) ∈ F[z], with p(z), q(z) nonzero. Then

π pq(p f ) = pπq ( f ).

(1.9)

Proof. Let r = πq f , i.e., for some polynomial a(z), we have f (z) = a(z)q(z) + r(z) and deg r < deg q. The representation of f (z) implies p(z) f (z) = a(z)(p(z)q(z)) + p(z)r(z). Since deg(pr) = deg p + degr < deg p + degq = deg(pq), it follows that π pq (p f ) = pr = pπq f , and hence (1.9) holds.



14

1 Algebraic Preliminaries

Definition 1.25. Let p(z), q(z) ∈ F[z]. We say that p(z) divides q(z), or that p(z) is a factor of q(z), and write p(z) | q(z), if there exists a polynomial a(z) such that q(z) = p(z)a(z). If p(z) ∈ F[z] and p(z) = ∑ni=0 ai zi , then p(z) defines a function on F given by n

p(α ) = ∑ ai α i ,

α ∈ F.

i=0

Then p(α ) is called the value of p(z) at α . An element α ∈ F is called a zero of p(z) if p(α ) = 0. We never identify the polynomial with the function defined by it. Theorem 1.26. Let p(z) ∈ F[z]. Then α is a zero of p(z) if and only if (z − α ) | p(z). Proof. If (z − α ) | p(z), then p(z) = (z − α )a(z), and hence p(α ) = 0. Conversely, by the division rule, we have p(z) = a(z)(z − α ) + r(z) with r(z) necessarily a constant. Substituting α in this equality implies r = 0.



Theorem 1.27. Let p(z) ∈ F[z] be a polynomial of degree n. Then p(z) has at most n zeros in F. Proof. The proof is by induction. The statement is certainly true for zero-degree polynomials. Assume that we have proved it for all polynomials of degree less than n. Suppose that p(z) is a polynomial of degree n. Either it has no zeros and the statement holds, or there exists a zero α . But then p(z) = (z − α )a(z), and a(z) has, by the induction hypothesis, at most n − 1 zeros.  Theorem 1.28. Let F be a field and F[z] the ring of polynomials over F. Then F[z] is a principal ideal domain. Proof. We have already shown that F[z] is a commutative ring with identity that contains no zero divisors. Let J be any ideal in F[z]. If J = {0}, then J is generated by 0. So let us assume that J = {0}. Let d(z) be any nonzero polynomial in J of minimal degree. We will show that J = dF[z]. To this end, let f (z) ∈ J be arbitrary. By the division rule of polynomials we have f (z) = a(z)d(z) + r(z) with deg r < deg d. Now r(z) = f (z) − a(z)d(z) ∈ J, since both f (z) and d(z) are in J. Since d(z) was a nonzero element of smallest degree, we must have r(z) = 0. So f (z) ∈ dF[z] and hence J ⊂ dF[z]. Conversely, since d(z) ∈ J, we have dF[z] ⊂ J, and so equality follows.  Definition 1.29. 1. Let p1 (z), . . . , pn (z) ∈ F[z]. A polynomial d(z) ∈ F[z] will be called a greatest common divisor of p1 (z), . . . , pn (z) ∈ F[z] if a. We have the division relation d(z) | pi (z), for all i = 1, . . . , n. b. If d1 (z) | pi (z), for all i = 1, . . . , n, then d1 (z) | d(z).

1.4 Rings and Fields

15

2. Let p1 (z), . . . , pn (z) ∈ F[z]. A polynomial d(z) ∈ F[z] will be called a least common multiple of p1 (z), . . . , pn (z) ∈ F[z] if a. We have the division relation pi (z) | d(z), for all i = 1, . . . , n. b. If pi (z) | d (z), for all i = 1, . . . , n, then d | d . Given polynomials p1 (z), p2 (z), we denote by p1 (z) ∧ p2 (z) their greatest common divisor and by p1 (z)∨ p2 (z) their least common multiple. It is easily shown that a greatest common divisor is unique up to a constant multiple. We will say that polynomials p1 (z), . . . , pn (z) ∈ F[z] are coprime if 1 is a greatest common divisor. The availability of the division process in F[z] leads directly to the Euclidean algorithm. This gives an algorithm for the computation of a g.c.d. of two polynomials. Theorem 1.30. Let p(z), q(z) ∈ F[z]. Set q−1 (z) = q(z) and q0 (z) = p(z). Define inductively, using the division rule for polynomials, a sequence of polynomials qi by qi+1 (z) = ai+1 (z)qi (z) − qi−1 (z)

(1.10)

with deg qi+1 < deg qi . Let qs (z) be the last nonzero remainder, i.e., qs+1 (z) = 0. Then qs (z) is a g.c.d. of q(z) and p(z). Proof. First we show that qs (z) is a common divisor of q(z) and p(z). Indeed, since qs+1 (z) = 0, we have qs−1 (z) = as+1 (z)qs (z), that is, qs (z) | qs−1 (z). Assume that we proved qs (z) | qs−1 (z), . . . , qi (z). Since qi−1 (z) = ai+1 (z)qi (z) − qi+1 (z) it follows that qs (z) | qi−1 (z) and hence, by induction qs (z) divides all qi (z), and in particular q0 (z), q−1 (z). So qs (z) is a common divisor of q(z) and p(z). Let J(qi , qi−1 ) = {aqi + bqi−1 | a(z), b(z) ∈ F[z]} be the ideal generated by qi (z), qi−1 (z). Clearly, equation (1.10) shows, again using an induction argument, that qi+1 (z) ∈ J(qi , qi−1 ) ⊂ J(qi−1 , qi−2 ) ⊂ · · · ⊂ J(q0 , q−1 ). In particular, qs (z) ∈ J(q0 , q−1 ). So there exist polynomials a(z), b(z) such that qs (z) = a(z)p(z) + b(z)q(z). This shows that any common divisor r(z) of p(z) and q(z) divides also qs (z). Thus qs (z) is a g.c.d. of p(z) and q(z).  We remark that the polynomials a(z), b(z) in the representation qs (z) = a(z)p(z)+ b(z)q(z) can be easily calculated from the polynomials ai (z). We will return to this in Chapter 8. Corollary 1.31. Let p(z), q(z) ∈ F[z]. Then 1. p(z) and q(z) are coprime if and only if the Bezout equation a(z)p(z) + b(z)q(z) = 1 is solvable in F[z].

(1.11)

16

1 Algebraic Preliminaries

2. A solution pair a(z), b(z) is uniquely determined if we require additionally that deg a < deg q. In that case, we have also deg b < deg p.

Proof. 1. Solvability of (1.11) is clearly a sufficient condition for the coprimeness of p(z), q(z). Necessity is a consequence of Theorem 1.30. 2. The solution polynomials a(z), b(z) of the Bezout equation (1.11) are not unique. In fact, if a(z), b(z) is a solution pair and r(z) ∈ F[z] is arbitrary, then also (a(z) − r(z)q(z)), (b(z) + r(z)p(z)) is a solution. By the division rule, we can choose r(z) such that deg(a − rq) < deg q, which also implies deg(b + rp) < deg p. This proves uniqueness.  In view of Theorem 1.16, the easiest way to construct ideals is by taking sums and intersections of kernels of ring homomorphisms. The case of interest for us is for the ring of polynomials. Definition 1.32. We define, for each α ∈ F, a map φα : F[z] −→ F by

φα (p) = p(α ).

(1.12)

Theorem 1.33. A map φ : F[z] −→ F is a ring homomorphism if and only if φ (p) = p(α ) for some α ∈ F. Proof. If φα is defined by (1.12), then clearly it is a ring homomorphism. Conversely, let φ : F[z] −→ F be ring homomorphism. Set α = φ (z). Then, given p(z) = ∑ki=0 pi zi , we have k

k

k

k

i=0

i=0

i=0

i=0

φ (p) = φ ∑ pi zi = ∑ pi φ (zi ) = ∑ pi φ (z)i = ∑ pi α i = p(α ).  Corollary 1.34. Given α1 , . . . , αn ∈ F, the set Jα1 ,...,αn = {p(z) ∈ F[z] | p(α1 ) = · · · = p(αn ) = 0} n (z − α ). is an ideal in F[z]. Moreover, Jα1 ,...,αn = dF[z], where d(z) = Πi=1 i

Proof. For φα defined by (1.12), we have Ker φα = {p ∈ F[z] | p(α ) = 0} = Jα , which is an ideal. Clearly Jα1 ,...,αn = ∩ni=1 Jαi , and the intersection of ideals is an ideal. Obviously, for d(z) defined as above and an arbitrary polynomial f (z) we have (d f )(αi ) = d(αi ) f (αi ) = 0, so d(z) f (z) ∈ Jα1 ,...,αn . Conversely, if g(z) ∈ Jα1 ,...,αn , we have g(αi ) = 0, and hence g(z) is divisible by z − αi . Since the αi are distinct, g(z) is divisible by d(z), or g(z) = d(z) f (z). 

1.4 Rings and Fields

17

Proposition 1.35. Let d(z) ∈ F[z]. Then dF[z] = {d(z)p(z) | p(z) ∈ F[z]} is an ideal. The next, important, result relates the generator of an ideal to division properties. Theorem 1.36. The ideal J generated by p1 (z), . . . , pn (z) ∈ F[z], namely J=



 ,

n

∑ ri (z)pi (z) | ri (z) ∈ F[z]

(1.13)

i=1

has the representation J = dF[z] if and only if d(z) is a greatest common divisor of p1 (z), . . . , pn (z) ∈ F[z] . Proof. By Theorem 1.28, there exists a d(z) ∈ J such that J = (d). Since pi (z) ∈ J, there exist polynomials qi (z) such that pi (z) = d(z)qi (z) for i = 1, . . . , n. So d(z) is a common divisor of the pi (z). We will show that it is maximal. Assume that d (z) is another common divisor of the pi (z), i.e., pi (z) = d (z)si (z). Since d(z) ∈ J, there exist polynomials ai (z) such that d(z) = ∑ni=1 ai (z)pi (z). Therefore n

n

n

i=1

i=1

i=1

d(z) = ∑ ai (z)pi (z) = ∑ ai (z)d (z)qi (z) = d (z) ∑ ai (z)qi (z). But this means that d (z) | d(z), and so d(z) is a g.c.d.



Corollary 1.37. Let p1 (z), . . . , pn (z) ∈ F[z], and let d(z) be their greatest common divisor. Then there exist polynomials a1 (z), . . . , an (z) ∈ F[z] such that n

d(z) = ∑ ai (z)pi (z).

(1.14)

i−1

Proof. Let d(z) be the g.c.d. of the pi (z). Obviously, d(z) ∈ J = {∑ni=1 ri (z)pi (z) | ri (z) ∈ F[z]}. Therefore there exist ai (z) ∈ F[z] for which (1.14) holds.  Corollary 1.38. Polynomials p1 (z), . . . , pn (z) ∈ F[z] are coprime if and only if there exist polynomials a1 (z), . . . , an (z) ∈ F[z] such that n

∑ ai (z)pi (z) = 1.

(1.15)

i=1

Equation (1.15), one of the most important equations in mathematics, will be refered to as the Bezout equation. The importance of polynomials in linear algebra stems from the strong connection between factorization of polynomials and the structure of linear transformations. The primary decomposition theorem is of particular applicability.

18

1 Algebraic Preliminaries

Definition 1.39. A polynomial p(z) ∈ F[z] is factorizable, or reducible, if there exist polynomials f (z), g(z) ∈ F[z] of degree ≥ 1 such that p(z) = f (z)g(z). If p(z) is not factorizable, it is called a prime or an irreducible polynomial. Note that the reducibility of a polynomial is dependent on the field F. Theorem 1.40. Let p(z), f (z), g(z) ∈ F[z], and assume that p(z) is irreducible and p(z) | ( f (z)g(z)). Then either p(z) | f (z) or p(z) | g(z). Proof. Assume that p(z) | ( f (z)g(z)) but p(z) does not divide f (z). Then a g.c.d. of p(z) and f (z) is 1. There exist therefore polynomials a(z), b(z) such that the Bezout equation 1 = a(z) f (z) + b(z)p(z) holds. From this it follows that g(z) = a(z)( f (z)g(z)) + (b(z)g(z))p(z). This implies p(z) | g(z).



Corollary 1.41. Let p(z) be an irreducible polynomial and assume p(z) | ( f1 (z) · · · fn (z)). Then there exists an index i for which p(z) | f i (z). 

Proof. By induction.

Lemma 1.42. Let p(z) and q(z) be coprime. Then if p(z) | q(z)s(z), it follows that p(z) | s(z). Proof. Follows immediately from Theorem 1.40.



Lemma 1.43. Let p(z), q(z) be coprime. Then p(z)q(z) is their l.c.m. Proof. Clearly, p(z)q(z) is a common multiple. Let s(z) be an arbitrary common multiple. In particular, we can write s(z) = q(z)t(z). Since p(z) and q(z) are coprime and p(z) | s(z), it follows that p(z) | t(z) or that t(z) = p(z)t (z). Thus s(z) = (p(z)q(z))t (z), and therefore p(z)q(z) is a least common multiple.  A polynomial p(z) ∈ F[z] is called monic if its highest nonzero coefficient is 1. Theorem 1.44. Let p(z) be a monic polynomial in F[z]. Then p(z) has a unique, up to ordering, factorization into a product of prime monic polynomials. Proof. We prove the theorem by induction on the degree of p(z). If deg p = 1, the statement is trivially true. Assume that we proved the statement for all polynomials of degree < n. Let p(z) be of degree n. Then either p(z) is prime or p(z) = f (z)g(z) with 1 < deg f , deg g < n. By the induction hypothesis, both f (z) and g(z) are decomposable into a product of prime monic polynomials. This implies the existence of a decomposition of p(z) into the product of monic primes. It remains to prove uniqueness. Suppose p1 (z), . . . , pm (z) and q1 (z), . . . , qn (z) are all prime and monic and p1 (z) · · · pm (z) = q1 (z) · · · qn (z).

1.4 Rings and Fields

19

Clearly, pm (z) | q1 (z) · · · qn (z), so, by Corollary 1.41, there exists an i such that pm (z) | qi (z). Since both are monic and irreducible, it follows that pm (z) = qi (z). Without loss of generality we may assume pm (z) = qn (z) Since there are no zero divisors in F[z], we get p1 (z) · · · pm−1 (z) = q1 (z) · · · qn−1 (z). We complete the proof by using the induction hypothesis.  Corollary 1.45. Given a monic polynomial p(z) ∈ F[z]. There exist monic primes pi (z) and positive integers ni , i = 1, . . . , s, such that p = p1 (z)n1 · · · ps (z)ns .

(1.16)

The primes pi (z) and the integers ni are uniquely determined. Proof. Follows from the previous theorem.



The factorization in (1.16) is called the primary decomposition of p(z). The monicity assumption is necessary only to get uniqueness. Without it, the theorem still holds, but the primes are determined only up to constant factors. The next result relates division in the ring of polynomials to the geometry of ideals. We saw already, in Proposition 1.15, that in a ring the sum and intersection of ideals are also ideals. the next proposition makes this specific for the ring of polynomials. Proposition 1.46. 1. Let p(z), q(z) ∈ F[z]. Then qF[z] ⊂ pF[z] if and only if p(z) | q(z). 2. Let pi (z) ∈ F[z] for i = 1, . . . , n. Then ∩ni=1 pi F[z] = pF[z], where p(z) is the l.c.m. of the pi (z). 3. Let pi (z) ∈ F[z] for i = 1, . . . , n. Then n

∑ pi F[z] = pF[z],

i=1

where p(z) is the g.c.d. of the pi (z). Proof. 1. Assume qF[z] ⊂ pF[z]. Thus there exists a polynomial f (z) for which q(z) = p(z) f (z), i.e., p(z) | q(z). Conversely, assume p(z) | q(z), i.e., q(z) = p(z) f (z) for some polynomial f (z). Then qF[z] = {q · g | g ∈ F[z]} = {p f q | g ∈ F[z]} ⊂ {ph | h ∈ F[z]} = pF[z]. 2. By Theorem 1.28, the intersection ∩m i=1 pi F[z] = pF[z] is a principal ideal, i.e., for some p(z) ∈ F[z] we have ∩m p F[z] = pF[z]. Clearly, pF[z] ⊂ pi F[z] for all i i=1 i. By part (i), this implies pi (z) | p(z). So p(z) is a common multiple of the pi (z).

20

1 Algebraic Preliminaries

Suppose q(z) is any common multiple, i.e., q(z) = pi (z)qi (z) and so qF[z] ⊂ pi F[z] for all i, which implies that qF[z] ⊂ ∩m i=1 pi F[z] = pF[z]. But this inclusion shows that p(z) | q(z), and hence p(z) is the l.c.m of the pi (z). 3. Again, by Theorem 1.28, ∑m i=1 pi F[z] is a principal ideal, i.e., for some p(z) ∈ F[z] we have ∑m p F[z] = pF[z]. Obviously, pi F[z] ⊂ pF[z] for all i, and hence i i=1 p(z) | pi (z). Thus p(z) is a common divisor for the pi (z). Let q(z) be any other common divisor. Then pi F[z] ⊂ qF[z], and hence pF[z] = ∑m i=1 pi F[z] ⊂ qF[z], which shows that q(z) | p(z), that is, p(z) is the g.c.d.  For the case of two polynomials we have the following. Proposition 1.47. Let p(z), q(z) be nonzero monic polynomials and let r(z) and s(z) be their g.c.d. and l.c.m. respectively, both taken to be monic. Then we have p(z)q(z) = r(z)s(z). Proof. Write p(z) = r(z)p1 (z), q(z) = r(z)q1 (z), with p1 (z), q1 (z) coprime and monic. Clearly s(z) = r(z)p1 (z)q1 (z) is a common multiple of p(z), q(z). Let s (z) be any other multiple. Since p(z) | s (z), we have s (z) = r(z)p1 (z)t(z) for some polynomial t(z). Since q(z) | s (z), we have q1 (z) | p1 (z)t(z). Since p1 (z), q1 (z) are coprime, it follows from Lemma 1.42 that q1 (z) | t(z). This shows that s(z) = r(z)p1 (z)q1 (z) is the unique monic l.c.m. of p(z) and q(z). The equality p(z)q(z) = r(z)s(z) is now obvious. 

1.4.3 Formal Power Series For a given field F, we denote by F[[z]] the set of all formal power series, i.e., of formal sums of the form f (z) = ∑∞j=0 f j z j . Addition and multiplication are defined by ∞

∞

∞

j=0

j=0

j=0

∑ f j z j + ∑ g j z j = ∑ ( f j + g j )z j

and

∞

∞

∞

j=0

j=0

k=0

∑ f j z j · ∑ g j z j = ∑ hk z k ,

with hk =

(1.17)

(1.18)

k

∑ f j gk− j .

(1.19)

j=0

With these operations, F[[z]] is a ring with identity 1. An element f (z) = ∑∞j=0 f j z j ∈ F[[z]] is invertible if and only if f0 = 0. To see this let g(z) = ∑∞j=0 g j z j . Then g(z) is an inverse of f (z) if and only if

1.4 Rings and Fields

21

1 = ( f g)(z) =



∞



k

∑ ∑ f j gk− j

k=0

zk .

j=0

This is equivalent to the solvability of the infinite system of equations k

∑ f j gk− j =

j=0



1 0

k = 0, k > 0.

The first equation is f0 g0 = 1, which shows the necessity of the condition f0 = 0. This is also sufficient, since the system of equations can be solved recursively. The following result analyzes the ideal structure in F[[z]]. Proposition 1.48. J ⊂ F[[z]] is a nonzero ideal if and only if for some nonnegative integer n, we have J = zn F[[z]]. Thus F[[z]] is a principal ideal domain. Proof. Clearly, any set of the form zn F[[z]] is an ideal. To prove the converse, we set, for f (z) = ∑∞j=0 f j z j ∈ F[[z]],

δ(f) =

min{n | fn = 0} −∞

f=  0 f =0

Let now f ∈ J be any nonzero element that minimizes δ ( f ). Then f (z) = zn h(z) with h invertible. Thus zn belongs to J and generates it.  In the sequel we find it convenient to work with the ring F[[z−1 ]] of formal power series in z−1 . We study now an important construction that allows us to construct from some ring larger rings. The prototype of this situation is the construction of the field of rational numbers out of the ring of integers. Given rings R and R, we say that R is embedded in R if there exists an injective homomorphism of R into R. A set S in a ring R with identity is called a multiplicative set if 0 ∈ / S, 1 ∈ S, and a, b ∈ S implies also ab ∈ S. Given a commutative ring with identity R and a multiplicative set S in R, we proceed to construct a new ring. Let M be the set of ordered paits (r, s) with r ∈ R and s ∈ S. We introduce a relation in M by saying that (r, s)  (r , s ) if there exists σ ∈ S for which σ (s r − sr ) = 0. We claim that this is indeed an equivalence relation. Reflexivity and symmetry are trivial to check. To check transitivity, assume σ (s r − sr ) = 0 and τ (s

r − s

) = 0 with σ , τ ∈ S. We compute

τσ s (s

r) = τ s

σ (s r) = τ s

σ (sr ) = σ sτ (s

r ) = σ sτ (s r

) = σ s τ (sr

), or τσ s (s

r − sr

) = 0. We denote by r/s the equivalence class of the pair (r, s). We denote by R/S the set of all equivalence classes in M . In R/S we define operations of addition and

22

multiplication by

1 Algebraic Preliminaries

rs + sr r r , + = s s ss

(1.20) rr r r · = . s s ss It can be checked that these operations are well defined, i.e., they are independent of the equivalence class representatives. Also, it is straightforward to verify that with these operations R/S is a commutative ring. This is called a ring of quotients of R. Of course, there may be many such rings, depending on the multiplicative sets we are taking. In case R is an entire ring, then the map φ : R −→ R/S defined by φ (r) = r/1 is an injective ring homomorphism. In this case more can be said. Theorem 1.49. Let R be an entire ring. Then R can be embedded in a field. Proof. Let S = R − {0}, that is, S is the set of all nonzero elements in R. Obviously S is a multiplicative set. We let F = R/S. Then F is a commutative ring with identity. To show that F is a field, it suffices to show that every nonzero element is invertible. If a/b = 0, this implies a = 0 and hence (a/b)−1 = b/a. Moreover, the map φ given before provides an embedding of R in F.  The field F constructed by the previous theorem is called the field of quotients of R. For our purposes the most important example of a field of quotients is that of the field of rational functions, denoted by F(z), which is obtained as the field of quotients of the ring of polynomials F[z]. Let us consider now an entire ring that is also a principal ideal domain. Let F be the field of fractions of R. Given any f ∈ F, we can consider J = {r ∈ R | r f ∈ R}. Obviously, J is an ideal, hence generated by an element q ∈ R that is uniquely defined up to an invertible factor. Thus q f = p for p ∈ R and f = p/q. Obviously, p, q are coprime, and f = p/q is called a coprime factorization of f . We give now a few examples of rings and fields that are a product of the process of taking a ring of quotients.

1.4.4 Rational Functions For a field F we saw that the ring of polynomials F[z] is a principal ideal domain. Its field of quotients is called the field of rational functions and denoted by F(z). Its elements are called rational functions. Every rational function has a representation of the form p(z)/q(z) with p(z), q(z) coprime. We can make the coprime factorization unique if we require the polynomial q(z) to be monic. By Proposition 1.20, every polynomial p(z) has a unique representation in the form p(z) = a(z)q(z) + r(z) with degr < deg q. This implies

1.4 Rings and Fields

23

r(z) p(z) = a(z) + . q(z) q(z)

(1.21)

A rational function r(z)/q(z) is called proper if deg r ≤ degq and strictly proper if deg r < deg q. The set of all strictly proper rational functions is denoted by F− (z). Thus we have F(z) = F[z] ⊕ F−(z). Here the direct sum representation refers to the uniqueness of the representation (1.21).

1.4.5 Proper Rational Functions We denote by F pr (z) the subring of F(z) defined by F pr (z) =

p(z) f (z) ∈ F(z) | f (z) = , deg p ≤ deg q . q(z)

(1.22)

It is easily checked that F pr (z) is a commutative ring with identity. An element f (z) ∈ F pr (z) is a unit if and only if in any representation f (z) = p(z)/q(z), we have deg p = deg q. We define the relative degree ρ by ρ (p/q) = deg q − deg p, and ρ (0) = −∞. Theorem 1.50. F pr (z) is a Euclidean domain with respect to the relative degree. Proof. We verify the conditions of a Euclidean ring as given in Definition 1.21. Let ρ be the relative degree. Let f (z) = p1 (z)/q1 (z) and g(z) = p2 (z)/q2 (z) be nonzero. Then  p1 p 2 = deg(q1 q2 ) − deg(p1 p2 ) ρ ( f g) = ρ q1 q2 = deg q1 + degq2 − deg p1 − deg p2 = (deg q1 − deg p1 ) + (degq2 − deg p2 ) = ρ ( f ) + ρ (g) ≥ ρ ( f ). Next, we turn to division. If ρ ( f ) < ρ (g), we write f (z) = 0 · g(z) + f (z). If g(z) = 0 and ρ ( f ) ≥ ρ (g), we show that g(z) divides f (z). Let σ and τ be the relative degrees of f (z) and g(z); thus σ ≥ τ . So we can write f (z) =

1 f1 (z), zσ

g(z) =

1 g1 (z), zτ

with f1 (z), g1 (z) units in F pr (z), i.e., satisfying ρ ( f1 ) = ρ (g1 ) = 0. Then we can write

24

1 Algebraic Preliminaries

 f (z) = Of course

1 g (z)−1 f1 (z) zσ − τ 1

 1 1 −1 g1 (z) g1 (z) f1 (z) . zτ zσ −τ

belongs to F pr (z) and has relative degree σ − τ .



Since F pr (z) is Euclidean, it is a principal ideal domain. We proceed to characterize all ideals in F pr (z). Theorem 1.51. A subset J ⊂ F pr (z) is a nonzero ideal if and only if it is of the form J = z1σ F pr (z) for some nonnegative integer σ . Proof. Clearly, J = z1σ F pr (z) is an ideal. Conversely, let J be a nonzero ideal. Let f (z) be a nonzero element in J of least relative degree, say σ . We may assume without loss of generality that f (z) = z1σ . By the proof of Theorem 1.50, f (z) divides every element with relative degree greater than or equal to σ . So f (z) is a generator of J.  Heuristically speaking, and adopting the language of complex analysis, we see that ideals are determined by the zeros at infinity, multiplicities counted. In terms of singularities, F pr (z) is the set of all rational functions that have no singularity (pole) at infinity. In comparison, F[z] is the set of all rational functions whose only singularity is at infinity. Of course, there are many intermediate situations and we turn to these.

1.4.6 Stable Rational Functions Fix the field in what follows to be the real field R. We can identify many subrings of R(z) simply by taking the ring of quotients of R[z] with respect to multiplicative sets that are smaller than the set of all nonzero polynomials. We shall say that a polynomial is stable if all its (complex) zeros are in a given subset Σ of the complex plane. It is antistable if all its zeros lie in the complement of Σ . We will assume now that the domain of stability is the open left half-plane. Let S be the set of all stable polynomials. Then S is obviously a multiplicative subset of R[z]. The ring of quotients S = R[z]/S ⊂ R(z) is called the ring of stable rational functions. As a ring of fractions it is a commutative ring with identity. An element f (z) ∈ S is a unit if in an irreducible representation f (z) = p(z)/q(z) the numerator p(z) is stable. Thus we may say that f (z) ∈ S is a unit if it has no zeros in the closed left half-plane. Such functions are sometimes called minimum phase functions. From the point of view of complex analysis, the ring of stable rational functions is the set of rational functions that have all their singularities in the open left halfplane or at the point at infinity. In S we define a degree function δ as follows. Let f (z) = p(z)/q(z) with p(z), q(z) coprime. We let δ ( f ) be the number of antistable zeros of p(z), and hence of f (z). Note that δ does not take into account zeros at infinity.

1.4 Rings and Fields

25

Theorem 1.52. S is a Euclidean ring with respect to the degree function δ . Proof. That δ ( f g) = δ ( f ) + δ (g) ≥ δ ( f ) is obvious. Let now f (z), g(z) ∈ S with g(z) = 0. Assume without loss of generality that δ ( f ) ≥ δ (g). Let g(z) = αg (z)/βg (z) with αg (z), βg (z) coprime polynomials. Similarly, let f (z) = α f (z)/ β f (z) with α f (z), β f (z) coprime. Factor αg (z) = α+ (z)α− (z), with α− (z) stable and α+ (z) antistable. Then g(z) =

α+ (z)α− (z) α− (z)(z + 1)ν α+ (z) α+ (z) = e(z) · , = · ν βg (z) βg (z) (z + 1) (z + 1)ν

with e(z) a unit and ν = δ (g). Since β f (z) is stable, β f (z), α+ (z) are coprime in R[z] and there exist polynomials φ (z), ψ (z) for which φ (z)α+ (z) + ψ (z)β f (z) = α f (z)(z+ 1)ν −1 , and we may assume without loss of generality that deg ψ < deg α+ . Dividing both sides by β f (z)(z + 1)ν −1 , we get

α f (z) = β f (z)



φ (z + 1) β f (z)



α+ (z) (z + 1)ν −1



 +

ψ (z) , (z + 1)ν −1

and we check that 

δ

α+ (z + 1)ν −1

≤ deg ψ < deg α+ = δ (g). 

Ideals in S can be represented by generators that are, up to unit factors, antistable polynomials. For two antistable polynomials p1 (z), p2 (z) we have p2 S ⊂ p1 S if and only if p1 (z) | p2 (z). More generally, for f 1 (z), f2 (z) ∈ S , we have f2 S ⊂ f1 S if and only if f1 (z) | f2 (z). However, the division relation f 1 (z) | f2 (z) means that every zero of f 1 (z) in the closed right half-plane is also a zero of f2 (z) with at least the same multiplicity. Here zeros should be interpreted as complex zeros. Since the intersection of subrings is itself a ring, we can consider the ring R pr (z) ∩ S . This is equivalent to excluding from S elements with a singularity at infinity. We denote this ring by RH∞ + . Because of its great importance to systemtheoretic problems, we shall defer the study of this ring to Chapters 11 and 12.

1.4.7 Truncated Laurent Series Let F be a field. We denote by F((z−1 )) the set of all formal sums of the form nf f (z) = ∑ j=−∞ f j z j with n f ∈ Z. The operations of addition and multiplication are defined by

26

1 Algebraic Preliminaries max{n f ,ng }

∑

( f + g)(z) =

( f j + g j )z j ,

j=−∞

and

{n f +ng }

( f g)(z) =

∑

hk zk ,

k=−∞

with hk =

∞

∑

f j gk− j .

j=−∞

Notice that the last sum is well defined, since it contains only a finite number of nonzero terms. We can check that all the field axioms are satisfied; in particular, all nonzero elements are invertible. We call this the field of truncated Laurent series. Note that F((z−1 )) can be considered the field of fractions of F[[z−1 ]]. Clearly, F(z) can be considered a subfield of F((z−1 )). We introduce for later use the projection maps n

n

f f π+ ∑ j=−∞ f j z j = ∑ j=0 f jz j, nf −1 j π− ∑ j=−∞ f j z = ∑ j=−∞ f j z j .

(1.23)

1.5 Modules The module structure is one of the most fundamental algebraic concepts. Most of the rest of this book is, to a certain extent, an elaboration on the module theme. This is particularly true for the case of linear transformations and linear systems. Let R be a ring with identity. A left module M over the ring R is a commutative group together with an operation of R on M that for all r, s ∈ R and x, y ∈ M, satisfies r(x + y) = rx + ry, (r + s)x = rx + sx, r(sx) = (rs)x, 1x = x. Right modules are defined similarly. Let M be a left R-module. A subset N of M is a submodule of M if it is an additive subgroup of M that further satisfies RN ⊂ M. Given two left R-modules M1 and M2 , a map φ : M1 −→ M2 is an R-module homomorphism if for all x, y ∈ M1 and r ∈ R,

φ (x + y) = φ x + φ y, φ (rx) = rφ (x).

1.5 Modules

27

Given an R-module homomorphism φ : M1 −→ M2 , then Ker φ and Im φ are submodules of M1 and M2 respectively. Given R-modules M0 , . . . , Mn , a sequence of R-module homomorphisms φi−1

φi

−→ Mi−1 −→ Mi −→ Mi+1 −→ is called an exact sequence if Im φi−1 = Ker φi . An exact sequence of the form φ

ψ

0 −→ M1 −→ M2 −→ M3 −→ 0 is called a short exact sequence. This means that φ is injective and ψ is surjective. Given modules Mi over the ring R, we can make the Cartesian product M1 × · · · × Mk into an R-module by defining, for mi , ni ∈ Mi and r ∈ R, (m1 , . . . , mk ) + (n1 , . . . , nk ) = (m1 + n1 , . . . , mk + nk ), r(m1 , . . . , mk ) = (rm1 , . . . , rmk ).

(1.24)

Given a module M and submodules Mi , we say that M is the direct sum of the Mi , and write M = M1 ⊕ · · · ⊕ Mk , if every m ∈ M has a unique representation of the form m = m1 + · · · + mk with mi ∈ Mi . Given a submodule N of a left R-module M, we can construct a module structure in the same manner in which we constructed quotient groups. We say that two elements x, y ∈ M are equivalent if x − y ∈ N. The equivalence class of x is denoted by [x]N = x + N. The set of equivalence classes is denoted by M/N. We make M/N into an R-module by defining, for all r ∈ R and x, y ∈ M, [x]N + [y]N = [x + y]N , r[x]N = [rx]N .

(1.25)

It is easy to check that these operations are well defined, that is, independent of the equivalence class representative. We state without proof the following. Proposition 1.53. With the operations defined in (1.25), the set of equivalence classes M/N is a module over R. This is called the quotient module of M by N. We shall make use of the following result. This is the counterpart of Theorem 1.9. Proposition 1.54. Let M be a module over the ring R. Then N ⊂ M is a submodule if and only if it is the kernel of a R-module homomorphism. Proof. Clearly the kernel of a module homomorphism is a submodule. Conversely, let π : M −→ M/N be the canonical projection. Then π is a module homomorphism and Ker π = N. 

28

1 Algebraic Preliminaries

Note that with i : N −→ M the natural embedding, then i

π

{0} −→ N −→ M −→ M/N −→ {0} is a short exact sequence. Proposition 1.55. Let M, N be R-modules and let φ : M −→ N be a surjective Rmodule homomorphism. Then we have the module isomorphism N  M/Ker φ . Proof. Let π be the canonical projection of N onto N/Ker φ , that is, π (m) = m + Ker φ . We define now a map τ : N/Ker φ −→ M by τ (m + Ker φ ) = φ (m). The map τ is well defined. For if m1 , m2 are representatives of the same coset, then m1 − m2 ∈ Ker φ . This implies φ (m1 ) = φ (m2 ). That τφ is a homomorphism follows from the fact that φ is one. Clearly, the surjectivity of φ implies that of τ . Finally, τ (m + Ker φ ) = 0 if and only if φ (m) = 0, i.e., m ∈ Ker φ . Thus τ is also injective, hence an isomorphism.  Let M be a module over the ring R and let S ⊂ M. By a linear combination, with coefficients in R, we mean a sum ∑α ∈S aα mα , where aα ∈ R, mα ∈ M and S a finite subset of S. The elements aα are called the coefficients of the linear combination. A subset S of M is called linearly independent if whenever ∑α ∈S aα mα = 0, we have aα = 0 for all α ∈ S . A subset S of M generates M if the set of all finite linear combinations of elements in S is equal to M. Equivalently, if the smallest submodule of M that includes S is M itself. S is a basis for M if it is non-empty subset, linearly independent and generates M. We end by giving a few examples of important module structures. Every abelian group G is a module over the ring of integers Z. Every ring is a module over itself, as well as over any subring. A vector space, as we shall see in Chapter 2, is a module over a field. Thus F(z) is a module over F[z]; F((z−1 )) is a module over each of the subrings F[z] and F[[z−1 ]]; z−1 F[[z−1 ]] has an induced F[z]-module structure, being isomorphic to F((z−1 ))/F[z]. This module structure is defined by ∞

hj = j j=1 z

z· ∑

∞

h j+1 . j j=1 z

∑

As F-vector spaces, we have the direct sum representation F((z−1 )) = F[z] ⊕ z−1F[[z−1 ]].

(1.26)

We denote by π+ and π− the projections of F((z−1 )) on F[z] and z−1 F[[z−1 ]] respectively, i.e., given by

1.5 Modules

29 N

∑

π−

h jz j =

j=−∞ N

π+

∑

−1

∑

h jz j

j=−∞

h jz j =

j=−∞

N

∑ h jz j

(1.27)

j=0

Clearly, π+ and π− are complementary projections, i.e., satisfy π±2 = π± and π+ + π− = I. Since F((z−1 )) is a 1-dimensional vector space over the field F((z−1 )), every F((z−1 ))-linear map from F((z−1 )) to F((z−1 )) has a representation (LA f )(z) = A(z) f (z),

(1.28)

for some A(z) ∈ F((z−1 )). The map LA defined by (1.28) is called the Laurent operator, and A(z) is called the symbol of the Laurent operator LA . Of course in terms of the expansions of A(z) and f (z), we have g = LA f = ∑k gk zk with gk = ∑∞j=−∞ A j fk− j . The sum is well defined, since there are only a finitely many nonzero terms in it. Of special importance is the Laurent operator acting in F((z−1 )) with symbol z. Because of the natural interpretation in terms of the expansion coefficients, we call it the bilateral shift, or simply the shift, and denote it by S. The shift S is clearly an invertible map in F((z−1 )), and we have (S f )(z) = z f (z), (S

−1

f )(z) = z−1 f (z).

(1.29)

The subsets F[z] and F[[z−1 ]] of F((z−1 )) are closed under addition and multiplication; hence they inherit natural ring structures. Units in F[[z−1 ]] are biproper, i.e., are proper and have a proper inverse. Similarly F((z−1 )) is a module over both F[z] and F[[z−1 ]]; F[z] ⊂ F((z−1 )) is an F[z] submodule and similarly z−1 F[[z−1 ]] ⊂ F((z−1 )) is an F[[z−1 ]] submodule. Furthermore, F((z−1 )) becomes a module over various rings including F, F[z], F((z−1 )) and F[[z−1 ]]. Various module structures will be of interest in the sequel. In particular, we note the following. Proposition 1.56. 1. F[z] is an F[z]-submodule of F((z−1 )). 2. F[[z−1 ]] and z−1 F[[z−1 ]] are F[[z−1 ]] submodules of F((z−1 )). 3. As F[z] modules we have the following short exact sequence of module homomorphisms: j

π

0 −→ F[z] −→ F((z−1 )) −→ F((z−1 ))/F[z] −→ 0, with j the embedding of F[z] into F((z−1 )) and π the canonical projection onto the quotient module. 

30

1 Algebraic Preliminaries

Elements of F((z−1 ))/F[z] are equivalence classes, and two elements are in the same equivalence class if and only if they differ in their polynomial terms only. For h ∈ F((z−1 )), we denote by [h]F[z] its equivalence class. A natural choice of representative in each equivalence class is the one element whose polynomial terms are all zero. This leads to the isomorphism F((z−1 ))/F[z]  z−1 F[[z−1 ]],

(1.30)

given by [h]F[z] → π− h. There are important linear transformations induced by S in the spaces F[z] and z−1 F[[z−1 ]]. Noting that F[z] is an F[z]-submodule of F((z−1 )), then F[z] is Sinvariant. Thus, we can define a map S+ : F[z] −→ F[z] by restricting S to F[z], i.e., S+ = S | F[z].

(1.31)

Similarly, S induces a map in the quotient module F((z−1 ))/F[z] given by [h]F[z] → [Sh]F[z]. Using the isomorphism (1.33), we also define S− : z−1 F[[z−1 ]] −→ z−1 F[[z−1 ]] by S− h = π− zh.

(1.32)

We note that S+ is injective but not surjective, whereas S− is surjective but not injective. Also codim Im S+ = dim Ker S− = dim F = 1. We will refer to S+ as the forward shift operator and to S− as the backward shift operator. Clearly, F((z−1 )), being a module over F((z−1 )), is also a module over any subring of F((z−1 )). In particular, it has a module structure over F[z]m×m and F[[z−1 ]], as well as over the rings F[z] and F[[z−1 ]]. We can state the following. Proposition 1.57. 1. F[z] is a left F[z] submodule, and hence also a left F[z] submodule of F((z−1 )). 2. z−1 F[[z−1 ]] is an F[[z−1 ]] submodule and hence also an F[[z−1 ]] submodule of F((z−1 )). 3. We have the isomorphism z−1 F[[z−1 ]]  F((z−1 ))/F[z],

(1.33)

and z−1 F[[z−1 ]] has therefore the naturally induced F[z]-module structure. Also, it has an F[z]-module structure given, for A ∈ F[z], by A · h = π− Ah,

h ∈ z−1 F[[z−1 ]].

(1.34)

4. Given A ∈ F[z] p×m, with the F[z]-module structures, the map X : z−1 F[[z−1 ]] −→ z−1 F[[z−1 ]] p defined by h → π− Ah is a module homomorphism. 5. F[z] has an F[[z−1 ]]-module structure given, for A ∈ F[[z−1 ]], by A · f = π+ A f ,

f ∈ F[z].

(1.35)

1.6 Exercises

31

1.6 Exercises 1. Show that the order o(Sn ) of the symmetric group Sn is n!. 2. Show that any subgroup of a cyclic group is cyclic, and so is any homomorphic image of G. 3. Show that every group of order ≤5 is abelian. 4. Prove the isomorphism C  R[z]/(z2 + 1)R[z]. 5. Given a polynomial p(z) = ∑nk=0 pk zk , we define its formal derivative by p (z) = ∑nk=0 kpk zk−1 . Show that (p(z) + q(z)) = p (z) + q (z), (p(z)q(z)) = p(z)q (z) + q(z)p (z), (p(z)m ) = mp (z)p(z)m−1 . Show that over a field of characteristic 0, a polynomial p(z) factors into the product of distinct irreducible factors if and only if the greatest common divisor of p(z) and p (z) is 1. 6. Let M, M1 be R-modules and N ⊂ M a submodule. Let π : M −→ M/N be the canonical projection and φ : M −→ M1 an R-homomorphism. Show that if Ker φ ⊃ N, there exists a unique R-homomorphism, called the induced homomorphism, φ |M/N : M/N −→ M1 for which φ |M/N = φ ◦ π . Show that Ker φ |M/N = (Ker φ )/N, Im φ |M/N = Im φ . 7. Let M be a module and Mi submodules. Show that if

M = M1 + · · · + Mk Mi ∩ ∑ j=i M j = {0},

then the map φ : M1 × · · · × Mk −→ M1 + · · · + Mk defined by

φ (m1 , . . . , mk ) = m1 + · · · + mk is a module isomorphism. 8. Let M be a module and Mi submodules. Show that if M = M1 + · · · + Mk and M1 ∩ M2 = 0, (M1 + M2 ) ∩ M3 = 0, .. . (M1 + · · · + Mk−1 ) ∩ Mk = 0, then M = M1 ⊕ · · · ⊕ Mk .

32

1 Algebraic Preliminaries

9. Let M be a module and K, L submodules. a. Show that (K + L)/K  L/(K ∩ L). b. If K ⊂ L ⊂ M, then

M/L  (M/K)/(L/K).

10. A module M over a ring R is called free if it is the zero module or has a basis. Show that if M is a free module over a principal ideal domain R having n basis elements and N is a submodule, then N is free and has at most n basis elements. 11. Show that F[z]n , z−1 F[[z−1 ]]n , F((z−1 ))n are F[z]-modules.

1.7 Notes and Remarks The development of algebra can be traced to antiquity, in particular to the Babylonians and the Chinese. For an interesting survey of early non-European mathematics, see Joseph (2000). Galois, one of the more brilliant and colorful mathematicians, introduced the term group in the context of solvability of polynomial equations by radicals. This was done in 1830, but Galois’ writings were first published, by Liouville, only in 1846, fifteen years after Galois died in a duel. The abstract definition of a group is apparently due to Cayley. The development of ring theory as a generalization of integer arithmetic owes much to the efforts to prove Fermat’s last theorem. Although, in the context of algebraic number theory, ideals appear already in the work of Kummer, the concepts of rings and fields are probably due to Dedekind, Kronecker, and Hilbert. An axiomatic approach to the theory of rings and modules was first developed by E. Noether. Modern expositions of algebra all stem from the classical book by van der Waerden (1931), which in turn was based on lectures by E. Noether and E. Artin. Incidentally, this seems to be the first book having a chapter devoted to linear algebra. For a modern, general book on algebra, Lang (1965) is recommended. Our emphasis on the ring of polynomials is not surprising, considering their wellentrenched role in the study of linear transformations in finite-dimensional vector spaces. The similar exposure given to the field of rational functions, and in particular to the subrings of stable rational functions and bounded stable rational functions, is motivated by the role they play in system theory. This will be treated in Chapters 10 to 12. For more information in this direction, the reader is advised to consult Vidyasagar (1985).

Chapter 2

Vector Spaces

2.1 Introduction Vector spaces provide the setting in which the rest of the topics that are to be presented in this book are developed. The content is geometrically oriented and we focus on linear combinations, linear independence, bases, dimension, coordinates, subspaces, and quotient spaces. Change of basis transformations provide us with a first instance of linear transformations.

2.2 Vector Spaces Vector spaces are modules over a field F. For concreteness, we give an ab initio definition. Definition 2.1. Let F be a field. A vector space V over F is a set, whose elements are called vectors, that satisfies the following set of axioms: 1. For each pair of vectors x, y ∈ V there exists a vector x + y ∈ V , called the sum of x and y, and the following holds: a. The commutative law: x + y = y + x. b. The associative law: x + (y + z) = (x + y) + z. c. There exists a unique zero vector 0 that satisfies 0 + x = x + 0, for all x ∈ V . d. For each x ∈ V there exists a unique element −x such that P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 2, © Springer Science+Business Media, LLC 2012

33

34

2 Vector Spaces

x + (−x) = 0, that is, V is a commutative group under addition. 2. For all x ∈ V and α ∈ F there exists a vector α x ∈ V , called the product of α and x, and the following are satisfied: a. The associative law:

α (β x) = (αβ )x.

b. For the unit 1 ∈ F and all x ∈ V we have 1 · x = x. 3. The distributive laws: a. (α + β )x = α x + β x, b. α (x + y) = α x + α y. Examples: ⎧⎛ ⎞ ⎫ ⎪ ⎪ ⎪ a1 ⎪ ⎪ ⎪ ⎪ ⎜ . ⎟ ⎪ ⎪ ⎪ ⎨⎜ ⎟ ⎬ ⎜ ⎟ n F = ⎜ . ⎟ |ai ∈ F . ⎜ ⎟ ⎪ ⎪ ⎪ ⎪ ⎪⎝ . ⎠ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ a ⎭ n

1. Let

We define the operations of addition and multiplication by a scalar α ∈ F by ⎛

⎞ ⎛ ⎞ ⎛ ⎞ a1 b1 a 1 + b1 ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ . ⎟ + ⎜ . ⎟ = ⎜ . ⎟, ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎝ . ⎠ ⎝ . ⎠ ⎝ . ⎠ an bn an + bn

⎞ ⎛ ⎞ α a1 a1 ⎜ . ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ α ⎜ . ⎟ = ⎜ . ⎟. ⎜ ⎟ ⎜ ⎟ ⎝ . ⎠ ⎝ . ⎠ α an an ⎛

With these definitions, Fn is a vector space. 2. An m × n matrix over the field F is a set of mn elements ai j arranged in rows and columns, i.e., ⎞ ⎛ a11 . . . a1n ⎜ . ... . ⎟ ⎟ ⎜ ⎟ ⎜ A = ⎜ . . . . . ⎟. ⎟ ⎜ ⎝ . ... . ⎠ am1 . . . amn We denote by Fm×n the set of all such matrices. We define in Fm×n the operations of addition and multiplication by scalars by

2.2 Vector Spaces

35

(ai j ) + (bi j ) = (ai j + bi j ), α (ai j ) = (α ai j ). These definitions make Fm×n into a vector space. Given the matrix A = (ai j ), ˜ as the n × m matrix given by we define its transpose, which we denote by A, (a˜i j ) = (a ji ). Given matrices A ∈ F p×m , B ∈ Fm×n , we define the product AB ∈ F p×n of the matrices by (AB)i j =

m

∑ aik bk j .

(2.1)

k=1

It is easily checked that matrix multiplication is associative and distributive, i.e., we have A(BC) = (AB)C, A(B1 + B2 ) = AB1 + AB2 , (A1 + A2)B = A1 B + A2 B.

(2.2)

In Fn×n we define the identity matrix In by In = (δi j ).

(2.3)

Here δi j denotes the Kronecker delta function, defined by 

δi j =

0 if i = j, 1 if i = j.

(2.4)

3. The rings F[z], F((z−1 )), F[[z]], F[[z−1 ]] are all vector spaces over F. So is F(z) the field of rational functions. 4. Many interesting examples are obtained by considering function spaces. A typical example is CR (X ), the space of all real-valued continuous functions on a topological space X . With addition and multiplication by scalars are defined by ( f + g)(x) = f (x) + g(x), (α f )(x) = α f (x), CR (X) is a vector space. We note the following simple rules. Proposition 2.2. Let V be a vector space over the field F. Let α ∈ F and x ∈ V . Then 1. 0x = 0. 2. α x = 0 implies α = 0 or x = 0.

36

2 Vector Spaces

2.3 Linear Combinations Definition 2.3. Let V be a vector space over the field F. Let x1 , . . . , xn ∈ V and α1 , . . . , αn ∈ F. The vector α1 x1 + · · · + αn xn is an element of V and is called a linear combination of the vectors x1 , . . . , xn . The scalars α1 , . . . , αn are called the coefficients of the linear combination. A vector x ∈ V is called a linear combination of the vectors x1 , . . . , xn if there exist scalars α1 , . . . , αn for which x = ∑ni=1 αi xi . A linear combination in which all coefficients are zero is called a trivial linear combination.

2.4 Subspaces Definition 2.4. Let V be a vector space over the field F. A nonempty subset M of V is called a subspace of V if for any pair of vectors x, y ∈ M and any pair of scalars α , β ∈ F, we have α x + β y ∈ M . Thus a subset M of V is a subspace if and only if it is closed under linear combinations. An equivalent description of subspaces is subsets closed under addition and multiplication by scalars. Examples: 1. For an arbitrary vector space V , V itself and {0} are subspaces. These are called the trivial subspaces. 2. Let ⎧⎛ ⎞ ⎫ ⎪ ⎪ a1 ⎪ ⎪ ⎪ ⎪ ⎪ ⎜ ⎟ ⎪ ⎪ ⎪ . ⎨⎜ ⎟ ⎬ ⎜ ⎟ M = ⎜ . ⎟ |an = 0 . ⎜ ⎟ ⎪ ⎪ ⎪ ⎪ ⎪⎝ . ⎠ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ⎭ an Then M is a subspace of Fn . 3. Let A be an m × n matrix. Then M = {x ∈ Fn |Ax = 0} is a subspace of Fn . This is the space of solutions of a system of linear homogeneous equations. Theorem 2.5. Let {Mα }α ∈A be a collection of subspaces of V . Then M = ∩α ∈A Mα is a subspace of V . Proof. Let x, y ∈ M and α , β ∈ F. Clearly, M ⊂ Mα for all α , and therefore x, y ∈ Mα , and since Mα is a subspace, we have that α x + β y belongs to Mα , for all α , and hence to the intersection. So α x + β y ∈ M . 

2.5 Linear Dependence and Independence

37

Definition 2.6. Let S be a subset of a vector space V . The set L(S), or span (S), the subspace spanned by S, is defined as the intersection of the (nonempty) set of all subspaces containing S. This is therefore the smallest subspace of V containing S. Theorem 2.7. Let S be a subset of a vector space V . Then span (S), the subspace spanned by S, is the set of all finite linear combinations of elements of S. Proof. Let M = {∑ni=1 αi xi |αi ∈ F, xi ∈ S, n ∈ N}. Clearly S ⊂ M , and M is a subspace of V , since linear combinations of linear combinations of elements of S are also linear combinations of elements of S. Thus span (S) ⊂ M . Conversely, we have S ⊂ span (S). So necessarily span (S) contains all finite linear combinations of elements of S. Hence M ⊂ span (S), and equality follows.  We saw that if M1 , . . . , Mk are subspaces of V , then so is M = ∩ki=1 Mi . This, in general, is not true for unions of subspaces. The natural concept is that of sum of subspaces. Definition 2.8. Let M1 , . . . , Mk be subspaces of a vector space V . The sum of p these subspaces, ∑i=1 Mi , is defined by p



p

∑ Mi = ∑ αi xi |αi ∈ F, xi ∈ Mi

i=1

 .

i=1

2.5 Linear Dependence and Independence Definition 2.9. Vectors x1 , . . . , xk in a vector space V are called linearly dependent if there exist α1 , . . . , αk ∈ F, not all zero, such that

α1 x1 + · · · + αk xk = 0. Vectors x1 , . . . , xk in a vector space V are called linearly independent if they are not linearly dependent. Thus x1 , . . . , xk are linearly dependent if there exists a nontrivial, vanishing linear combination. On the other hand, x1 , . . . , xk are linearly independent if and only if α1 x1 + · · · + αk xk = 0 implies α1 = · · · = αk = 0. That is, the only vanshing linear combination of linearly independent vectors is the trivial one. We note that, as a consequence of Proposition 2.2, a set containing a single vector x ∈ V is linearly independent if and only if x = 0. Definition 2.10. Vectors x1 , . . . , xk in a vector space V are called a spanning set if V = span (x1 , . . . , xk ). Theorem 2.11. Let S ⊂ V . 1. If S is a spanning set and S ⊂ S1 ⊂ V , then S1 is also a spanning set.

38

2 Vector Spaces

2. If S is a linearly independent set and S0 ⊂ S, then S0 is also a linearly independent set. 3. If S is linearly dependent and S ⊂ S1 ⊂ V , then S1 is also linearly dependent. 4. Every subset of V that includes the zero vector is linearly dependent. As a consequence, a spanning set must be sufficiently large, whereas for linear independence the set must be sufficiently small. The case in which these two properties are in balance is of special importance and this leads to the following. Definition 2.12. A subset B of vectors in V is called a basis if 1. B is a spanning set. 2. B is linearly independent. V is called a finite-dimensional space if there exists a basis in V having a finite number of elements. Note that we defined the concept of finite dimensionality before having defined dimension. Example: Let V = Fn . Let ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ 1 0 0 ⎜0⎟ ⎜1⎟ ⎜.⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜.⎟ ⎜0⎟ ⎜.⎟ e1 = ⎜ ⎟ , e2 = ⎜ ⎟ , . . . , en = ⎜ ⎟ . ⎜.⎟ ⎜.⎟ ⎜.⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎝.⎠ ⎝.⎠ ⎝0⎠ 0 0 1 Then B = {e1 , . . . , en } is a basis for Fn . The following result is the main technical instrument in the study of bases. Theorem 2.13 (Steinitz). Let x1 , . . . , xm ∈ V and let e1 , . . . , e p be linearly independent vectors that satisfy ei ∈ span (x1 , . . . , xm ) for all i. Then there exist p vectors in {xi }, without loss of generality we may assume they are the first p vectors, such that span (e1 , . . . , e p , x p+1 , . . . , xm ) = span (x1 , . . . , xm ). Proof. We prove this by induction on p. For p = 1, we must have e1 = 0 by linear independence. Therefore there exist αi such that e1 = ∑m i=1 αi xi . Necessarily αi = 0 for some i. Without loss of generality we assume α1 = 0. Therefore we can write m

x1 = α1−1 e1 − α1−1 ∑ αi xi . i=2

This means that x1 ∈ span (e1 , x2 , . . . , xm ). Of course we also have xi ∈ span (e1 , x2 , . . . , xm ) for i = 2, . . . , m. Therefore

2.5 Linear Dependence and Independence

39

span (x1 , x2 , . . . , xm ) ⊂ span (e1 , x2 , . . . , xm ). On the other hand, by our assumption, e1 ∈ span (x1 , x2 , . . . , xm ) and hence span (e1 , x2 , . . . , xm ) ⊂ span (x1 , x2 , . . . , xm ). From these two inclusion relations, the following equality follows: span (e1 , x2 , . . . , xm ) = span (x1 , x2 , . . . , xm ). Assume that we have proved the assertion for up to p − 1 elements and assume that e1 , . . . , e p are linearly independent vectors that satisfy ei ∈ span (x1 , . . . , xm ) for all i. By the induction hypothesis, we have span (e1 , . . . , e p−1 , x p , . . . , xm ) = span (x1 , x2 , . . . , xm ). Therefore, e p ∈ span (e1 , . . . , e p−1 , x p , . . . , xm ), and hence there exist αi such that e p = α1 x1 + · · · + α p−1 + α p x p + · · · + αm xm . It is impossible that α p , . . . , αm are all 0, for that implies that e p is a linear combination of e1 , . . . , e p−1 , contradicting the assumption of linear independence. So at least one of these numbers is nonzero, and without loss of generality, reordering the elements if necessary, we assume α p = 0. Now x p = α p−1 (α1 e1 + · · · + α p−1 e p−1 − e p + α p+1x p+1 + · · · + αn xn ), that is, x p ∈ span (e1 , . . . , e p x p+1 , . . . , xm ). Therefore, span (x1 , x2 , . . . , xm ) ⊂ span (e1 , . . . , e p x p+1 , . . . , xm ) ⊂ span (x1 , x2 , . . . , xm ), and hence the equality span (e1 , . . . , e p x p+1 , . . . , xm ) = span (x1 , x2 , . . . , xm ).  Corollary 2.14. The following assertions hold. 1. Let {e1 , . . . , en } be a basis for the vector space V , and let { f1 , . . . , fm } be linearly independent vectors in V . Then m ≤ n. 2. Let {e1 , . . . , en } and { f1 , . . . , fm } be two bases for V . Then n = m. Proof. 1. Apply Theorem 2.13.

40

2 Vector Spaces

2. By the first part we have both m ≤ n and n ≤ m, so equality follows.



Thus two different bases in a finite-dimensional vector space have the same number of elements. This leads us to the following definition. Definition 2.15. Let V be a vector space over the field F. The dimension of V is defined as the number of elements in an arbitrary basis. We denote the dimension of V by dim V . Theorem 2.16. Let V be a vector space of dimension n. Then 1. Every subset of V containing more than n vectors is linearly dependent. 2. A set of p < n vectors in V cannot be a spanning set.

2.6 Subspaces and Bases Theorem 2.17. 1. Let V be a vector space of dimension n and let M be a subspace. Then dim M ≤ dim V . 2. Let {e1 , . . . , e p } be a basis for M . Then there exist vectors {e p+1 , . . . , en } in V such that {e1 , . . . , en } is a basis for V . Proof. It suffices to prove the second assertion. Let {e1 , . . . , e p } be a basis for M anf { f1 , . . . , fn } a basis for V . By Theorem 2.13 we can replace p of the fi by the e j , j = 1, . . . , p, and get a spanning set for V . But a spanning set with n elements is necessarily a basis for V .  From two subspaces M1 , M2 of a vector space V we can construct the subspaces M1 ∩ M2 and M1 + M2 . The next theorem studies the dimensions of these subspaces. For the sum of two subspaces of a vector space V we have the following. Theorem 2.18. Let M1 , M2 be subspaces of a vector space V . Then dim(M1 + M2 ) + dim(M1 ∩ M2 ) = dim M1 + dimM2 .

(2.5)

Proof. Let {e1 , . . . , er } be a basis for M1 ∩ M2 . Then there exist vectors { fr+1 , . . . , f p } and {gr+1, . . . , gq } such that the set {e1 , . . . , er , fr+1 , . . . , f p } is a basis for M1 and the set {e1 , . . . , er , gr+1 , . . . , gq } is a basis for M2 . We will proceed to show that the set {e1 , . . . , er , fr+1 , . . . , f p , gr+1 , . . . , gq } is a basis for M1 + M2 . Clearly {e1 , . . . , er , fr+1 , . . . , f p , gr+1 , . . . , gq } is a spanning set for M1 + M2 . So it remains to show that it is linearly independent. Assume that they are linearly dependent, i.e., there exist αi , βi , γi such that r

p

i=1

i=r+1

∑ αi ei + ∑

βi f i +

q

∑

i=r+1

γi gi = 0,

(2.6)

2.7 Direct Sums

or

41

q

∑



γi g i = −

i=r+1

r

p

i=1

i=r+1

∑ αi ei + ∑



βi f i .

So it follows that ∑qi=r+1 γi gi ∈ M1 . On the other hand, ∑qi=r+1 γi gi ∈ M2 as a linear q combination of some of the basis elements of M2 . Therefore ∑i=r+1 γi gi belongs to M1 ∩ M2 and hence can be expressed as a linear combination of the ei . So there exist numbers ε1 , . . . , εr such that r

q

i=1

i=r+1

∑ εi ei + ∑

γi gi = 0.

However, this is a linear combination of the basis elements of M2 ; hence εi = γ j = 0. Now (2.6) reduces to r

p

i=1

i=r+1

∑ αi ei + ∑

βi fi = 0.

From this, we conclude by the same reasoning that αi = β j = 0. This proves the linear independence of the vectors {e1 , . . . , er , fr+1 , . . . , f p , gr+1 , . . . , gq }, and so they are a basis for M1 + M2 . Now dim(M1 + M2 ) = p + q − r = dim M1 + dim M2 − dim(M1 ∩ M2 ). 

2.7 Direct Sums Definition 2.19. Let Mi , i = 1, . . . , p, be subspaces of a vector space V . We say that p ∑i=1 Mi is a direct sum of the subspaces Mi , and write M = M1 ⊕ · · · ⊕ M p , if p p for every x ∈ ∑i=1 Mi there exists a unique representation x = ∑i=1 xi with xi ∈ Mi . Proposition 2.20. Let M1 , M2 be subspaces of a vector space V . Then M = M1 ⊕ M2 if and only if M = M1 + M2 and M1 ∩ M2 = {0}. Proof. Assume M = M1 ⊕ M2 . Then for x ∈ M we have x = x1 + x2, with xi ∈ Mi . Suppose there exists another representation of x in the form x = y1 + y2 , with yi ∈ Mi . From x1 + x2 = y1 + y2 we get z = x1 − y1 = y2 − x2 . Now x1 − y1 ∈ M1 and y2 − x2 ∈ M2 . So, since M1 ∩ M2 = {0}, we have z = 0, that is, x1 = y1 and x 2 = y2 . Conversely, suppose every x ∈ M has a unique representation x = x1 + x2 , with xi ∈ Mi . Thus M = M1 + M2 . Let x ∈ M1 ∩ M2 . Then x = x + 0 = 0 + x, which implies, by the uniqueness of the representation, that x = 0. 

42

2 Vector Spaces

We consider next some examples. 1. The space F((z−1 )) of truncated Laurent series has the spaces F[z] and F[[z−1 ]] as subspaces. We clearly have the direct sum decomposition F((z−1 )) = F[z] ⊕ z−1 F[[z−1 ]].

(2.7)

The factor z−1 guarantees that the constant elements appear in one of the subspaces only. 2. We denote by Fn [z] the space of all polynomials of degree < n, i.e., Fn [z] = {p(z) ∈ F[z]| deg p < n}. The following result is based on Proposition 1.46. Proposition 2.21. Let p(z), q(z) ∈ F[z] with deg p = m and deg q = n. Let r(z) be the g.c.d. of p(z) and q(z) and let s(z) be their l.c.m. Let deg r = ρ . Then 1. pFn [z] + qFm[z] = rFm+n−ρ [z]. 2. pFn [z] ∩ qFm [z] = sFρ [z]. Proof. 1. We know that with r(z) the g.c.d. of p(z) and q(z), pF[z] + qF[z] = rF[z]. So, given f (z), g(z), ∈ F[z], there exists h(z) ∈ F[z] such that p(z) f (z) + q(z)g(z) = r(z)h(z). Now we take remainders after division by p(z)q(z), i.e., we apply the map π pq. Now π pq p f = pπq f and π pq qg = qπ p g. Finally, since by Proposition 1.47 we have p(z)q(z) = r(z)s(z), it follows that π pq rh = πrs rh = rπs h. Now deg s = deg p + deg q − deg r = n + m − ρ . So we get the equality pFm [z] + qFm[z] = rFm+n−ρ [z]. 2. We have pF[z]∩qF[z] = sF[z]. Again we apply π pq. If f ∈ pF[z] then f = p f  , and hence π pq p f  = pπq f  ∈ pFn [z]. Similarly, if f (z) ∈ qF[z] then f (z) = q(z) f  (z) and π pq f = π pq · q(z) f  (z) = q(z)π p f  ∈ qFm [z]. On the other hand, f (z) = s(z)h(z) implies π pqsh = πrs sh = sπr h ∈ sFρ [z].  Corollary 2.22. Let p(z), q(z) ∈ F[z] with deg p = m and degq = n. Then p(z) ∧ q(z) = 1 if and only if Fm+n [z] = pFn [z] ⊕ qFm [z]. We say that the polynomials p1 (z), . . . , pk (z) are mutually coprime if for i = j, pi (z) and p j (z) are coprime. Theorem 2.23. Let p1 (z), . . . , pk (z) ∈ F[z] with deg pi = ni with ∑ki=1 ni = n. Then the pi (z) are mutually coprime if and only if Fn [z] = π1 (z)Fn1 [z] ⊕ · · · ⊕ πk (z)Fnk [z], where the π j (z) are defined by

πi (z) = Π j=i p j (z).

2.8 Quotient Spaces

43

Proof. The proof is by induction. Assume the pi (z) are mutually coprime. For k = 2 the result was proved in Proposition 2.21. Assume it has been proved for positive integers ≤ k − 1. Let τi (z) = Π k−1 j=1 p j (z). Then j=i

Fn1 +···+nk−1 [z] = τ1 (z)Fn1 [z] ⊕ · · · ⊕ τk−1 (z)Fnk−1 [z]. Now πk (z) ∧ pk (z) = 1, so Fn [z] = πk (z)Fnk [z] ⊕ pk (z)Fn1 +···+nk−1 [z]  = pk (z) τ1 (z)Fn1 [z] ⊕ · · · ⊕ τk−1 (z)Fnk−1 [z] ⊕ πk (z)Fnk (z) = π1 (z)Fn1 [z] ⊕ · · · ⊕ πk (z)Fnk [z]. Conversely, if Fn [z] = π1 (z)Fn1 [z] ⊕ · · · ⊕ πk (z)Fnk [z], then there exist polynomials fi (z) such that 1 = ∑ πi fi . The coprimeness of the πi (z) implies the mutual coprimeness of the pi (z).  Corollary 2.24. Let p(z) = p1 (z)n1 · · · pk (z)nk be the primary decomposition of p(z), with deg pi = ri and n = ∑ki=1 ni . Then Fn [z] = p2 (z)n2 · · · pk (z)nk Fr1 n1 [z] ⊕ · · · ⊕ p1 (z)n1 · · · pk−1 (z)nk−1 Frk nk [z]. Proof. Follows from Theorem 2.23, replacing pi (z) by pi (z)ni .



2.8 Quotient Spaces We begin by introducing the concept of codimension. Definition 2.25. We say that a subspace M ⊂ X has codimension k, denoted by codim M = k, if 1. There exist k vectors {x1 , . . . , xk }, linearly independent over M , i.e., for which ∑ki=1 αi xi ∈ M if and only if αi = 0 for all i = 1, . . . , k. 2. X = L(M , x1 , . . . , xk ). Now let X be a vector space over the field F and let M be a subspace. In X we define a relation x y if x − y ∈ M . (2.8) It is easy to check that this is indeed an equivalence relation, i.e., it is reflexive, symmetric, and transitive. We denote by [x]M = x + M = {x + m|m ∈ M } the equivalence class of x ∈ X . We denote by X /M the set of equivalence classes with respect to the equivalence relation induced by M as in (2.8).

44

2 Vector Spaces

So far, X /M is just a set. We introduce in X /M two operations, addition and multiplication by a scalar, as follows: [x]M + [y]M = [x + y]M , x, y ∈ X , α [x]M = [α x]M . Proposition 2.26. 1. The operations of addition and multiplication by a scalar are well defined, i.e., independent of the representatives x, y. 2. With these operations, X /M is a vector space over F. 3. If M has codimension k in X , then dim X /M = k. Proof. 1. Let x x and y y. Thus [x]M = [x ]M . This means that x = x+m1 , y = y + m2 with mi ∈ M . Hence x + y = x + y + (m1 + m2 ), which shows that [x + y ]M = [x + y]M . Similarly, α x = α x + α m. So α x − α x = α m ∈ M ; hence [α x ]M = [α x]M . 2. The axioms of a vector space for X /M are easily shown to result from those in X . 3. Let x1 , . . . , xk be linearly independent over M and such that L(x1 , . . . , xk , M ) = X . We claim that {[x1 ], . . . , [xk ]} are linearly independent, for if ∑ni=1 αi [xi ] = 0, it follows that [∑ αi xi ]M = 0, i.e., ∑ni=1 αi xi ∈ M . Since x1 , . . . , xn are linearly independent over M , necessarly αi = 0, i = 1, . . . , k. Let now [x]M be an arbitrary equivalence class in X /M . Assume x ∈ [x]M . Then there exist αi ∈ F such that x = α1 x1 + · · · + αn xn + m for some m ∈ M . This implies [x]M = α1 [x1 ]M + · · · + αk [xk ]M . So indeed {[x1 ]M , . . . , [xk ]M } is a basis for X /M , and hence dim X /M = k. Corollary 2.27. Let q ∈ F[z] be a polynomial of degree n. Then 1. qF[z] is a subspace of F[z] of codimension n. 2. dim F[z]/qF[z] = n. Proof. The polynomials 1, z, . . . , zn−1 are obviously linearly independent over qF[z]. Moreover, applying the division rule of polynomials, it is clear that 1, z, . . . , zn−1 together with qF[z] span all of F[z].  Proposition 2.28. Let F((z−1 )) be the space of truncated Laurent series and F[z] and z−1 F[[z−1 ]] the corresponding subspaces. Then we have the isomorphisms F[z] F((z−1 ))/z−1 F[[z−1 ]] and

z−1 F[[z−1 ]] F((z−1 ))/F[z].

Proof. Follows from the direct sum representation (2.7).



2.9 Coordinates

45

2.9 Coordinates Lemma 2.29. Let V be a finite-dimensional vector space of dimension n and let B = {e1 , . . . , en } be a basis for V . Then every vector x ∈ V has a unique representation as a linear combination of the ei . That is, n

x = ∑ αi ei .

(2.9)

i=1

Proof. Since B is a spanning set, such a representation exists. Since B is linearly independent, the representation (2.9) is unique.  Definition 2.30. The scalars α1 , . . . , αn will be called the coordinates of x with respect to the basis B, and we will use the notation ⎛

⎞ α1 ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ B [x] = ⎜ . ⎟ . ⎜ ⎟ ⎝ . ⎠ αn The vector [x]B will be called the coordinate vector of x with respect to B. We will always write it in column form. The map x → [x]B is a map from V to Fn . Proposition 2.31. Let V be a finite-dimensional vector space of dimension n and let B = {e1 , . . . , en } be a basis for V . The map x → [x]B has the following properties: 1. 2.

[x + y]B = [x]B + [y]B . [α x]B = α [x]B .

3. [x]B = 0 if and only if x = 0. 4. For every α1 , . . . , αn ∈ F there exists a vector x ∈ V for which ⎞ α1 ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ B [x] = ⎜ . ⎟ . ⎜ ⎟ ⎝ . ⎠ αn ⎛

Proof. 1. Let x = ∑ni=1 αi ei , y = ∑ni=1 βi ei . Then

46

2 Vector Spaces n

n

n

i=1

i=1

i=1

x + y = ∑ αi ei + ∑ βi ei = ∑ (αi + βi )ei . So

⎞ ⎛ ⎞ ⎛ ⎞ α1 + β1 α1 β1 ⎜ ⎟ ⎜ ⎟ ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ [x + y]B = ⎜ . ⎟ = ⎜ . ⎟ + ⎜ . ⎟ = [x]B + [y]B . ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎠ ⎝ . ⎠ ⎝ . ⎠ ⎝ . αn + βn αn βn ⎛

2. Let α ∈ F. Then α x = ∑ni=1 ααi ei , and therefore ⎛

⎞ ⎛ ⎞ αα1 α1 ⎜ . ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ [α x]B = ⎜ . ⎟ = α ⎜ . ⎟ = α [x]B . ⎜ ⎟ ⎜ ⎟ ⎝ . ⎠ ⎝ . ⎠ ααn αn 3. If x = 0 then x = ∑ni=1 0ei , and ⎛ ⎞ 0 ⎜.⎟ ⎜ ⎟ ⎜ ⎟ [x]B = ⎜ . ⎟ . ⎜ ⎟ ⎝.⎠ 0 Conversely, if [x]B = 0 then x = ∑ni=1 0ei = 0. 4. Let α1 , . . . , αn ∈ F. Then we define a vector x ∈ V by x = ∑ni=1 αi ei . Then clearly ⎛

⎞ α1 ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ [x]B = ⎜ . ⎟ . ⎜ ⎟ ⎝ . ⎠ αn



2.10 Change of Basis Transformations Let V be a finite-dimensional vector space of dimension n and let B = {e1 , . . . , en } and B1 = { f1 , . . . , fn } be two different bases in V . We will explore the connection between the coordinate vectors with respect to the two bases.

2.10 Change of Basis Transformations

47

For each x ∈ V there exist unique α j , β j ∈ F for which x=

n

n

j=1

j=1

∑ α je j = ∑ β j f j .

(2.10)

Since for each j = 1, . . . , n, the basis vector e j has a unique expansion as a linear combination of the fi , we set n

e j = ∑ ti j f i . i=1

numbers, which we arrange in an n × n matrix. We denote 1 . We refer to this matrix as the basis transformation matrix this matrix by [I]B B from the basis B to the basis B1 . Substituting back into (2.10), we get These equations define n2

x = ∑ni=1 βi fi = ∑nj=1 α j e j = ∑nj=1 α j ∑ni=1 ti j fi = ∑nj=1 ∑ni=1 ti j α j fi = ∑ni=1 (∑nj=1 ti j α j ) fi . Equating coefficients of the fi , we obtain

βi =

n

∑ ti j α j .

j=1

Thus we conclude that B 1 [x]B1 = [I]B B [x] .

So the basis transformation matrix transforms the coordinate vector with respect to the basis B to the coordinate vector with respect to the basis B1 . Theorem 2.32. Let V be a finite-dimensional vector space of dimension n and let B = {e1 , . . . , en }, B1 = { f1 , . . . , fn }, and B2 = {g1 , . . . , gn } be three bases for V . Then we have B2 B1 2 [I]B B = [I]B1 [I]B . B2 B1 2 Proof. Let [I]B B = (ri j ), [I]B1 = (si j ), [I]B = (ti j ). This means that

⎧ ⎪ e j = ∑ni=1 ri j gi , ⎪ ⎪ ⎪ ⎪ ⎨ fk = ∑ni=1 sik gi , ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ e j = ∑ni=1 tk j fk . Therefore ri j = ∑ni=1 sik tk j .



48

2 Vector Spaces

Corollary 2.33. Let V be a finite-dimensional vector space with two bases B and B1 . Then B −1 1 [I]B B = ([I]B1 ) .

Proof. Clearly [I]B B = I, where I denotes the identity matrix. By Theorem 2.32, we get B1 B I = [I]B B = [I]B1 [I]B .



1 Theorem 2.34. The mapping in Fn defined by the matrix [I]B B as

B [x]B → [I]B1 [x]B = [x]B1

has the following properties: B1 B B1 B B B 1 [I]B B ([x] + [y] ) = [I]B [x] + [I]B [y] , B1 B B 1 [I]B B (α [x] ) = α [I]B [x] .

Proof. We compute B1 B B B B1 1 [I]B B ([x] + [y] ) = [I]B [x + y] = [x + y] B

B

= [x]B1 + [y]B1 = [I]B1 [x]B + [I]B1 [y]B . Similarly, B1 B B B1 1 [I]B B (α [x] ) = [I]B [α x] = [α x] B

= α [x]B1 = α [I]B1 [x]B .



These properties characterize linear maps. We will discuss those in detail in Chapter 4.

2.11 Lagrange Interpolation Let Fn [z] = {p(z) ∈ F[z]| deg p < n}. Clearly, Fn [z] is an n-dimensional subspace of F[z]. In fact, Bst = {1, z, . . . , zn−1 } is a basis for Fn [z], and we will refer to this as i the standard basis of Fn [z]. Obviously, if p(z) = ∑n−1 i=0 pi z , then

2.11 Lagrange Interpolation

49

⎛

⎞ p0 ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ st [p] = ⎜ . ⎟ . ⎜ ⎟ ⎝ . ⎠ pn−1 We exhibit next another important basis that is intimately related to polynomial interpolation. To this end we introduce The Lagrange interpolation problem: Given distinct numbers αi ∈ F, i = 1, . . . , n and another set of arbitrary numbers ci ∈ F, i = 1, . . . , n, find a polynomial p(z) ∈ Fn [z] such that p(αi ) = ci , i = 1, . . . , n. We can replace this problem by a set of simpler Special Interpolation Problems: Find polynomials li (z) ∈ Fn [z] such that for all i = 1, . . . , n, li (α j ) = δ i j ,

j = 1, . . . , n.

Here δi j is the Kronecker delta function. If li (z) are such polynomials, then a solution to the Lagrange interpolation problem is given by n

p(z) = ∑ ci li (z). i=1

This is seen by a direct evaluation at all αi . The existence and properties of the solution to the special interpolation problem are summarized in the following. Proposition 2.35. Given distinct numbers αi ∈ F, i = 1, . . . , n, let the Lagrange interpolation polynomials li (z) be defined by li (z) =

Π j=i (z − α j ) . Π j=i (αi − α j )

Then 1. The polynomials li (z) are in Fn [z] and li (α j ) = δi j ,

j = 1, . . . , n.

2. The set {l1 (z), . . . , ln (z)} forms a basis for Fn [z]. 3. For all p(z) ∈ Fn [z] we have n

p(z) = ∑ p(αi )li (z). i=1

(2.11)

50

2 Vector Spaces

Proof. 1. Clearly, li (z) has degree n − 1; hence li (z) ∈ Fn [z]. They are well defined, since by the distinctness of the αi , all the denominators are different from zero. Moreover, by construction, we have li (α j ) = δi j ,

j = 1, . . . , n.

2. We show first that the polynomials l1 (z), . . . , ln (z) are linearly independent. For this, assume that ∑ni=1 ci li (z) = 0. Evaluating at α j , we get n

n

i=1

i=1

0 = ∑ ci li (α j ) = ∑ ci δi j = c j . Since dim Fn [z] = n, they actually form a basis. 3. Since {l1 (z), . . . , ln (z)} forms a basis for Fn [z], then for every p(z) ∈ Fn [z] there exist c j such that n

p(z) = ∑ ci li (z). i=1

Evaluating at α j , we get n

n

i=1

i=1

p(α j ) = ∑ ci li (α j ) = ∑ ci δi j = c j .  The solution of the Lagrange interpolation problem presented above was achieved by an ad hoc construction of the Lagrange polynomials. In Chapter 5, we will see that more general interpolation problems can be solved by applying the Chinese remainder theorem. Corollary 2.36. Let α1 , . . . , αn ∈ F be distinct. Let Bin = {l1 (z), . . . , ln (z)} be the Lagrange interpolation basis and let Bst be the standard basis in Fn [z]. Then the change of basis transformation from the standard basis to the interpolation basis is given by ⎛ ⎞ 1 α1 . . α1n−1 ⎜. . .. . ⎟ ⎜ ⎟ ⎜ ⎟ [I]in (2.12) . ⎟. st = ⎜ . . . . ⎜ ⎟ ⎝. . .. . ⎠ 1 αn . . αnn−1 Proof. From the equality (2.11) we get as special cases, for i = 0, . . . , n − 1, z j = j  ∑ni=1 αi li (z), and (2.12) follows. The matrix in (2.12) is called the Vandermonde matrix.

2.12 Taylor Expansion

51

2.12 Taylor Expansion We proceed to study, algebraically, the Taylor expansion of polynomials, and we do it within the framework of change of basis transformations. Let p(z) ∈ Fn [z], i.e., deg p < n, and let α ∈ F. We would like to have a representation of p(z) in the form p(z) =

n−1

∑ p j (z − α ) j .

j=0

We refer to this as the Taylor expansion of p(z) at the point α . Naturally, the j standard representation of a polynomial, namely p(z) = ∑n−1 j=0 p j z , is the Taylor expansion at 0. Proposition 2.37. Given a positive integer n and α ∈ F. Then 1. The set of polynomials Bα = {1, z − α , . . . , (z − α )n−1 } forms a basis for Fn [z]. 2. For every p(z) ∈ Fn [z] there exist unique numbers pi,α such that p(z) =

n

∑ p j,α (z − α ) j .

(2.13)

j=0

3. The change of basis transformation is given by ⎛

⎞ (−α )n−1 1 −α α 2 ⎜ 0 1 −2α ⎟ . ⎜ ⎟ ⎜ ⎟ 1 . ⎜. ⎟ st [I]α = ⎜ ⎟, ⎜. ⎟ . . ⎜ ⎟ ⎝. . −(n − 1)α ⎠ 0 . . .0 1 i.e., the ith column entries come from the binomial expansion of (z − α )i−1 . Proof. 1. In Bα we have one polynomial of each degree from 0 to n − 1; hence these polynomials are linearly independent. Since dimFn [z] = n, they form a basis. 2. Follows from the fact that Bα is a basis. 3. We use the binomial expansion of (z − α )i−1 .  The Taylor expansion can be generalized by replacing the basis {(z − λ )i }n−1 i=0 n−1 with {Π i−1 j=0 (z − λ j )}i=0 , where the λi ∈ F are assumed to be distinct. The proof of the following proposition is analogous to the preceeding one and we omit it. Proposition 2.38. Let λi ∈ F, i = 0, . . . , n − 1, be distinct. Then

52

2 Vector Spaces

n−1 1. The set of polynomials {Π i−1 j=0 (z − λ j )}i=0 forms a basis for Fn [z]. Here we use the convention that Π −1 j=0 (z − λ j ) = 1. 2. For every p(z) ∈ Fn [z] there exist unique numbers ci such that

p(z) =

n−1

∑ ci Π i−1 j=0 (z − λ j ).

(2.14)

i=0

We will refer to (2.14) as the Newton expansion of p(z). This expansion is important for solving polynomial interpolation problems, and we will return to it in Section 5.5.3.

2.13 Exercises 1. Let K , L , M be subspaces of a vector space. Show that K ∩ (K ∩ L + M ) = K ∩ L + K ∩ M , (K + L ) ∩ (K + M ) = K + (K + L ) ∩ M . 2. Let Mi be subspaces of a finite-dimensional vector space X . Show that if dim(∑ki=1 Mi ) = ∑ki=1 dim(Mi ), then M1 + · · · + Mk is a direct sum. 3. Let V be a finite-dimensional vector space over F. Let f be a bijective map on V . Define operations by v1 ∗ v2 = f −1 ( f (v1 ) + f (v2 )), α ∇v = f −1 (α f (v)). Show that with these operations, V is a vector space over F. 4. Let V = {pn−1 zn−1 + · · · + p1 z + p0 ∈ F[z]|pn−1 + · · · + p1 + p0 = 0}. Show that V is a finite-dimensional subspace of F[z] and find a basis for it. 5. Let q(z) be a monic polynomial with distinct zeros λ1 , . . . , λn . Let p(z) be a g(λ ) polynomial of degree n − 1. Show ∑ni=1 f  (λj ) = 1. j 6. Let f (z), g(z) ∈ F[z] with g(z) nonzero. Then f (z) has a unique representation of the form f (z) = ∑i ai (z)g(z)i with deg ai < deg g. 7. Let CR (X) be the space of all real-valued, continuous functions on a topological space X. Let V± = { f ∈ CR (X)| f (x) = ± f (−x)}. Show that CR (X) = V+ ⊕ V− .

2.14 Notes and Remarks Linear algebra is geometric in origin. It traces its development to the work of Fermat and Descartes on analytic geometry. In that context points in the plane are identified with ordered pairs of real numbers. This was extended in the work of Hamilton, who

2.14 Notes and Remarks

53

discovered quaternions. In its early history linear algebra focused on the solution of systems of linear equations, and the use of determinants for that purpose was predominant. Matrices came rather late in the game, formalized by Cayley and Sylvester, both of whom made important contributions. This ushered in the era of matrix theory, which also served as the title of the classic book Gantmacher (1959). For a short discussion of the contributions of Cayley and Sylvester to matrix theory, see Higham (2008). The major step in the development of linear algebra as a field of mathematics is due to Grassmann (1844). In this remarkable book, Grassmann gives a fairly abstract definition of a linear vector space and develops the theory of linear independence very much in the spirit of modern linear algebra expositions. He defines the notions of subspace, independence, span, dimension, sum and intersection of subspaces, as well as projections onto subspaces. He also proves Theorem 2.13, usually attributed to Steinitz. He shows that any finite set has an independent subset with the same span and that any independent set extends to a basis, and he proves the important identity dim(U + V ) = dim U + dim V − dim(U ∩ V ). He obtains the formula for change of coordinates under change of basis, defines elementary transformations of bases, and shows that every change of basis is a product of elementary matrices. Since Grassmann was ahead of his time and did not excel at exposition, his work had little immediate impact. The first formal, modern definition of a vector space appears in Peano (1888), where it is called a linear system. Peano treats also the set of all linear transformations between two vector spaces. However, his most innovative example of a vector space was that of the polynomial functions of a real variable. He noted that if the polynomial functions were restricted to those of degree at most n, then they would form a vector space of dimension n + 1. But if one considered all such polynomial functions, then the vector space would have an infinite dimension. This is a precursor of functional analysis. Thirty years after the publication of Peano’s book, an independent axiomatic approach to vector spaces was given by Weyl. Extensions of the vector space axioms to include topological considerations are due to Schmidt, Banach, Wiener, and Von Neumann. The classic book van der Waerden (1931) seems to be the first algebra book having a chapter devoted to linear algebra. Adopting Noether’s point of view, modules are defined first and vector spaces appear as a special case.

Chapter 3

Determinants

3.1 Introduction Determinants used to be an important tool for the study of systems of linear equations. With the development of the abstract approach, based on the axioms of vector spaces, determinants lost their central role. Still, we feel that taking them out altogether from a text on linear algebra would be a mistake. Throughout the book we shall return to the use of special determinants for showing linear independence of vectors, invertibility of matrices, and coprimeness of polynomials. Since we shall need to compute determinants with polynomial entries, it is natural to develop the theory of determinants over a commutative ring with an identity.

3.2 Basic Properties Let R be a commutative ring with identity. Let X ∈ Rn×n and denote by x1 , . . . , xn its columns. Definition 3.1. A determinant is a function D : Rn×n −→ R that as a function of the columns of a matrix in Rn×n satisfies 1. D(x1 , . . . , xn ) is multilinear, that is, it is a linear function in each of its columns. 2. D(x1 , . . . , xn ) is alternating, by which we mean that it is zero whenever two adjacent columns coincide. 3. It is normalized so that if ei , . . . , en are the columns of the identity matrix, then D(e1 , . . . , en ) = 1. Proposition 3.2. If two adjacent columns in a matrix are interchanged, then the determinant changes sign.

P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 3, © Springer Science+Business Media, LLC 2012

55

56

3 Determinants

Proof. Let x and y be the the ith and (i + 1)th columns respectively. Then 0 = D(. . . , x + y, x + y, . . .) = D(. . . , x, x + y, . . . ) + D(. . . , y, x + y, . . . ) = D(. . . , x, x, . . . ) + D(. . . , x, y, . . . ) + D(. . . , y, x, . . . ) + D(. . ., y, y, . . . ) = D(. . . , x, y, . . . ) + D(. . . , y, x, . . . ).



This property of determinants explains the usage of alternating. Corollary 3.3. If any two columns in a matrix are interchanged then the determinant changes sign. Proof. Suppose the ith and jth columns of the matrix A are interchanged. Without loss of generality, assume i < j. By j − i transpositions of adjacent columns, the jth column can be brought to the ith place. Now the ith column is brought to the jth place by j − i − 1 transpositions of adjacent columns. This changes the value of the determinant by only a factor (−1)2 j−2i−1 = −1.  Corollary 3.4. If in a matrix any two columns are equal, then the determinant vanishes. Corollary 3.5. If the jth column of a matrix, multiplied by c, is added to the ith columnn, the value of the determinant does not change. Proof. We use the linearity in the ith variable to compute D(x1 , . . . , xi + cx j , . . . , x j , . . . , xn ) = D(x1 , . . . , xi , . . . , x j , . . . , xn ) + D(x1 , . . . , x j , . . . , x j , . . . , xn ) = D(x1 , . . . , xi , . . . , x j , . . . , xn ).



So far, we have proved elementary properties of the determinant function, but we do not know yet whether such a function exists. Our aim now is to prove the existence of a determinant function, which we will do by a direct, inductive, construction. Definition 3.6. Let A be an n × n matrix over R. We define the (i, j)th minor of A, which we denote by Mi j , as the determinant of the matrix of order n − 1 obtained from A by eliminating the ith row and jth column. We define the (i, j)-cofactor, which we denote by Ai j , by Ai j = (−1)i+ j Mi j . The matrix whose i, j entry is A ji will be called the classical adjoint of A and denoted by adj (A).

3.2 Basic Properties

57

Theorem 3.7. For each integer n ≥ 1, a determinant function exists, and is given by the following formula to which we refer as the expansion by the ith row: D(A) =

n

∑ a i j Ai j .

(3.1)

j=1

Proof. The construction is by an inductive process. For n = 1 we define D(a) = a, and this clearly satisfies the required three properties of the determinant function. Assume that we have constructed determinant functions for integers ≤ n − 1. Now we define a determinant of A via the expansion by rows formula (3.1). We show now that the properties of a determinant are satisfied. 1. Multilinearity: Fix an index k. We will show that D(A) is linear in the kth column. For j = k, ai j is not a function of the entries of the kth column, but Ai j is linear in the entries of all columns appearing in it, including the kth. For j = k, the cofactor Aik is not a function of the elements of the kth column, but the coefficient aik is a linear function. So D(A) is linear as a function of the kth column. 2. Alternacy: Assume that the kth and (k + 1)th columns are equal. Then, for each index j = k, k + 1 we clearly have Ai j = (−1)i+ j Mi j , and Mi j contains two adjacent and equal columns, hence is zero by the induction hypothesis. Therefore D(A) = aik Aik + ai(k+1)Ai(k+1) = aik (−1)i+k Mik + ai(k+1)(−1)i+k+1 Mi(k+1) = aik (−1)i+k Mik + aik (−1)i+k+1 Mi(k+1) = 0, for aik = ai(k+1) and Mik = Mi(k+1) . 3. Normalization: We compute D(I) =

n

∑ δi j (−1)i+ j Ii j = (−1)2iIii = 1,

j=1

for Iii is the determinant of the identity matrix of order n − 1, hence equal to 1 by the induction hypothesis.  We use now the availability of the determinant function to discuss permutations. By a permutation σ of n elements we will mean a bijective map of the set {1, . . . , n} onto itself. The set of all such permutations Sn is a group and is called the symmetric group. We use alternatively the notation (σ1 , . . . , σn ) for the permutation. Assume now that (σ1 , . . . , σn ) is a permutation. Let e1 , . . . , en be the standard unit vectors in Rn . Since a determinant function exists, D(eσ1 , . . . , eσn ) = ±1. We can bring the permutation matrix, whose columns are the eσi , to the identity matrix in

58

3 Determinants

a finite number of column exchanges. Clearly, the sign depends on the permutation only. Thus we define (−1)σ = sign (σ ) = D(eσ1 , . . . , eσn ).

(3.2)

We will call (−1)σ the sign of the permutation. We say that the permutation is even if (−1)σ = 1 and odd if (−1)σ = −1. Given a matrix A, we denote by A(i) its ith column. Thus we have n

A( j) = ∑ ai j ei . i=1

Therefore, we can compute D(A) = D(A(1) , . . . , A(n) ) = D(∑ni1 =1 ai1 1 ei1 , . . . , ∑nin =1 ain 1 ein ) = ∑ni1 =1 · · · ∑nin =1 D(ai1 1 ei1 , . . . , ain n ein ) = ∑σ ∈Sn aσ1 1 · · · aσn n D(eσ1 , . . . , eσn ) = ∑σ ∈Sn (−1)σ aσ1 1 · · · aσn n D(e1 , . . . , en ) = ∑σ ∈Sn (−1)σ aσ1 1 · · · aσn n .

(3.3)

This gives us another explicit expression for the determinant function. In fact, this expression for the determinant function could be an alternative starting point for the development of the theory, but we chose to do it differently. From now on we will use the notation det(A) for the determinant of A, i.e., we have (3.4) det(A) = ∑ (−1)σ aσ1 1 · · · aσn n . σ ∈Sn

The existence of this expression leads immediately to the following. Theorem 3.8. Let R be a commutative ring with an identity. Then, for any integer n, there exists a unique determinant function, and it is given by (3.4). Proof. Let D be any function defined on Rn×n and satisfying the determinantal axioms. Then clearly, using the computation (3.3), we get D(A) = det(A).  There was a basic asymmetry in our development of determinants, since we focused on the columns. The next result is a step toward putting the theory of determinants into a more symmetric state. Theorem 3.9. Given an n × n matrix A over the commutative ring R, we have for ˜ its transpose A, ˜ = det(A). det(A) (3.5) Proof. We use the fact that a˜i j = a ji , where a˜i j denotes the i j entry of the transposed ˜ Thus we have matrix A.

3.2 Basic Properties

59

˜ = ∑σ ∈S (−1)σ a˜σ (1)1 · · · a˜σ (n)n det(A) n = ∑σ ∈Sn (−1)σ a1σ (1) · · · anσ (n) −1 = ∑σ −1 ∈Sn (−1)σ aσ −1 (1)1 · · · aσ −1 (n)n = det(A).



Corollary 3.10. For each integer n ≥ 1, a determinant function exists, and is given by the following formula, to which we refer as the expansion by the jth column: n

D(A) = ∑ ai j Ai j .

(3.6)

i=1

Proof. We use the expansion by the ith column for the classical adjoint matrix. We use the fact that a˜ ji = ai j and A˜ ji = Ai j to obtain ˜ = ∑nj=1 a˜ ji A˜ ji = ∑nj=1 ai j Ai j . det(A) = D(A)



Corollary 3.11. The determinant, as a function of the rows of a matrix, is multilinear, alternating, and satisfies det(I) = 1. We can prove now the important multiplicative rule of determinants. Theorem 3.12. Let R be a commutative ring and let A and B be matrices in Rn×n . Then det(AB) = det(A) det(B). Proof. Let C = AB. Let A( j) and C( j) be the jth columns of A and C respectively. Clearly C( j) = ∑ni=1 bi j A(i) . Hence det(C) = det(C(1) , . . . ,C(n) ) = det(∑ni1 =1 bi1 1 A(i1 ) , . . . , ∑nin =1 bin n A(in ) ) = ∑ni1 =1 · · · ∑nin =1 bi1 1 · · · bin n det(A(i1 ) , . . . , A(in ) ) = ∑σ ∈Sn bσ (1)1 · · · bσ (n)n (−1)σ det(A(1) , . . . , A(n) ) = det(A) ∑σ ∈Sn (−1)σ bσ (1)1 · · · bσ (n)n = det(A) det(B).



We can use the expansion by rows and columns to obtain the following result. Corollary 3.13. Let A be an n × n matrix over R. Then ∑nk=1 aik A jk = δi j det(A), ∑nk=1 aki Ak j = δi j det(A).

(3.7)

Proof. The first equation is the expansion of the determinant by the jth row if i = j, whereas if i = j it is the expansion by the jth row of a matrix derived from A by

60

3 Determinants

replacing the jth row with the ith. Thus, it is the determinant of a matrix with two equal rows; hence its value is zero. The other formula is proved analogously.  The determinant expansion rules can be written in a matrix form by defining a matrix adj A, called the classical adjoint of A, via (adj A)i j = A ji . Clearly, the expansion formulas, given in Corollary 3.13, can now be written concisely as (adjA)A = A(adjA) = det A · I. (3.8) This leads to the important determinantal criterion for the nonsingularity of matrices. Corollary 3.14. Let A be a square matrix over the field F. Then A is invertible if and only if detA = 0. Proof. Assume that A is invertible. Then there exists a matrix B such that AB = I. By the multiplication rule of determinants, we have det(AB) = det(A) · det(B) = det(I) = 1, and hence det(A) = 0. Conversely, assume det(A) = 0. Then A−1 =

1 adjA. det A

(3.9) 

3.3 Cramer’s Rule We can use determinants to give a closed-form representation of the solution of a system of linear equations, provided that the coefficient matrix of the system is nonsingular. Theorem 3.15. Given the system of linear equations Ax = b, with A a⎛nonsingular, ⎞ x1 ⎜ ⎟ n × n matrix. Denote by A(i) the columns of the matrix A and let x = ⎝ ... ⎠ be the xn solution vector, that is, b = ∑ni=1 xi A(i) . Then

3.3 Cramer’s Rule

61

xi = or equivalently,

det(A(1) , . . . , b, . . . , A(n) ) , det(A)

  a11   .  1  . xi =  det A  .   .   an1

. . . . . .

b1 . . . . bn

.. .. .. .. .. ..

 a1n  .   .  . .   .  ann 

(3.10)

(3.11)

Proof. We compute det(A(1) , . . . , b, . . . , A(n) ) = det(A(1) , . . . , b, ∑nj=1 x j A( j) , A(n) ) = ∑nj=1 x j det(A(1) , . . . , b, A( j) , A(n) ) = xi det(A(1) , . . . , b, A( j) , A(n) ) = xi det(A). Another way to see this is to note that in this case, the unique solution is given by x = A−1 b. We will compute the ith component, xi : xi = (A−1 b)i = ∑nj=1 (A−1 )i j b j 1 n 1 = ∑ (adj A)i j b j = det A det A j=1

n

∑ b j A ji.

j=1

However, this is just the expansion by the ith column of the matrix A, where the ith column has been replaced by the elements of the vector b. The representation (3.11) of the solution to the system of equations Ax = b goes by the name Cramer’s rule.  We prove now an elementary, but useful, result about the computation of determinants. Proposition 3.16. Let A and B be square matrices, not necessarily of the same size. Then

AC = det A · detB. (3.12) det 0B

AC Proof. We note that det is multilinear and alternating in the columns of A 0 B

and in the rows of B. Therefore



AC IC det = det A · detB · det . 0B 0 I

62

3 Determinants

But a consideration of elementary operations in rows and columns leads to det

IC 0 I



= det

I 0 0I

= 1.



Lemma 3.17. Let A, B,C, and D be matrices of appropriate size, with A, D square, such that C and D commute. Then

AB det = det(AD − BC). (3.13) CD Proof. Without loss of generality, let detD = 0. From the equality

AB CD



=

A − BD−1C B 0 D



0 , D−1C I I

we conclude that

AB det = det(A − BD−1C) · detD CD = det(A − BCD−1) · detD = det(AD − BC).

(3.14) 

3.4 The Sylvester Resultant We give now a determinantal condition for the coprimeness of two polynomials. Our approach is geometric in nature. Let p(z), q(z) ∈ F[z]. We know, from Corollary 1.38, that p(z) and q(z) are coprime if and only if the Bezout equation a(z)p(z) + b(z)q(z) = 1 has a polynomial solution. The following gives a matrix condition for coprimeness. Theorem 3.18. Let p(z), q(z) ∈ F[z] with p(z) = p0 + · · · + pm zm and q(z) = q0 + q1 z + · · · + qnzn . Then the following conditions are equivalent: 1. The polynomials p(z) and q(z) are coprime. 2. We have the direct sum representation Fm+n [z] = pFn [z] ⊕ qFm [z]. 3. The resultant matrix

(3.15)

3.4 The Sylvester Resultant

63

⎛

p0 ⎜. ⎜ ⎜. ⎜ ⎜. ⎜ ⎜ ⎜p Res (p, q) = ⎜ m ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝

. .. ... .... .... ... .. .

q0 . . . . p0 q n . . . pm

⎞ ⎟ . ⎟ ⎟ .. ⎟ ... ⎟ ⎟ ⎟ . . . q0 ⎟ ⎟ .... ⎟ ⎟ .... ⎟ ⎟ ... ⎟ ⎟ .. ⎠

(3.16)

qn

is nonsingular. 4. The resultant, defined as the determinant of the resultant matrix, is nonzero. Proof. (1) ⇒ (2) Follows from the Bezout equation directly or from Proposition 2.21 as a special case. (2) ⇒ (1). Since 1 ∈ Fm+n [z], it follows that a solution exists to the Bezout equation a(z)p(z) + b(z)q(z) = 1. Moreover, with the conditions deg a < n, deg b < m satisfied, it is unique. (2) ⇒ (3) By Proposition 2.21, the equality Fm+n [z] = pFn [z] + qFm [z] is equivalent to this sum being a direct sum. In turn, this implies that a union of bases for pFn [z] and qFm [z] is a basis for Fm+n [z]. Now {zi p(z)|i = 0, . . . , n − 1} is a basis for pFn [z], and {zi q(z)|i = 0, . . . , m − 1} is a basis for qFm [z]. Thus Bres = {p(z), zp(z), . . . , zn−1 p(z), q(z), zq(z), . . . , zm−1 q(z)} forms a basis for Fm+n [z]. The last space has also the standard basis Bst = {1, z, . . . , zm+n−1 }. It is easy to check that Res (p, q) = [I]stres , i.e., it is a change of basis transformation, hence necessarily invertible. (3) ⇒ (2) If Res (p, q) is nonsingular, this shows that Bres is a basis for Fm+n [z]. This implies the equality (3.15). (3) ⇔ (4) Follows from Corollary 3.14.



We conclude this short chapter by computing the determinant of the Vandermonde matrix, derived in (2.12). Proposition 3.19. Let ⎛

1 λ1 ⎜. . ⎜ ⎜ V (λ1 , . . . , λn ) = ⎜ . . ⎜ ⎝. . 1 λn be the Vandermonde matrix. Then we have

⎞ . . λ1n−1 .. . ⎟ ⎟ ⎟ .. . ⎟ ⎟ .. . ⎠ . . λnn−1

64

3 Determinants

 1 λ 1  . .   detV (λ1 , . . . , λn ) =  . .  . .   1 λn

 . . λ1n−1  . . .   . . .  = Π1≤ j
(3.17)

Proof. By induction on n. For n = 1 the equality is trivially satisfied. Assume that this holds for all integers < n. We consider the following determinant  1 λ 1  . .   f (z) =  . .   1 λn−1  1 z

 . . λ1n−1  . . .   . . . . n−1  . . λn−1  . . zn−1 

Obviously, f (z) is a polynomial of degree n − 1 that vanishes at the points λ1 , . . . , λn−1 . Thus f (z) = Cn−1 (z − λ1 ) · · · (z − λn−1 ). Expanding by the last row, and using the induction hypothesis, we get    1 λ . . λ n−2  1   1 . . .. .      Cn−1 =  . . . . .  = Π1≤ j


We note that for (3.17) to hold, we do not have to assume the λi to be distinct. However, clearly, the determinant of a Vandermonde matrix is nonzero if and only if the λi are distinct.

3.5 Exercises 1. Let A, B be n × n matrices. For the classical adjoint show the following: a. b. c. d.

adj (AB) = adj A · adjB. det adjA = (det A)n−1 . adj (adjA) = (det A)n−2 · A. 2 det adj(adj A) = (det A)(n−1) .

2. Let a square n × n matrix A be defined by ai j = 1 − δi j , where δi j is the Kronecker delta. Show that detA = (n − 1)(−1)n−1.

3.5 Exercises

65

3. Let A, B be square matrices. Show that det

AB BA

= det(A + B) · det(A − B).

4. Given the block matrix



AB CD

,

with A, D square and D nonsingular. Show that

AB det CD



= det D · det(A − BD−1C).

5. Let A be a square n × n matrix, x, y ∈ Fn , and 0 = α ∈ F. Show that det

A x y˜ α

˜ = det(α A − xy).

Show that adj(I − xy) ˜ = xy˜ − (1 − y˜x)I. 6. Let p(z) = be a polynomial of degree ≤ n − 1. Show that p(z) and zn − 1 are coprime if and only if i ∑n−1 i=0 pi z

⎛

p0 pn−1 ⎜ p ⎜ 1 p0 ⎜ det ⎜ . . ⎜ ⎝ . . pn−1 .

. . . . .

⎞ . p1 . . ⎟ ⎟ ⎟ . . ⎟ = 0. ⎟ . pn−1 ⎠ p1 p 0

7. Let V (λ1 , . . . , λn ) be the Vandermonde determinant. Show that  1  .   .  .  .  1

λ1 . . . . λn

 . . λ1n−2 λ2 · · · λn   .. . .   .. . .   = (−1)n−1V (λ1 , . . . , λn ).  .. . .   .. . .  n−2 . . λn λ1 · · · λn−1 

8. Let V (λ1 , . . . , λn ) be the Vandermonde determinant. Show that

66

3 Determinants

  1 λ1  . .   . .  . .  . .   1 λn

 . . λ1n−2 (λ2 + · · · + λn )n−1   .. . .   .. . .   = (−1)n−1V (λ1 , . . . , λn ).  .. . .   .. . .  n−2 n−1  . . λn (λ1 + · · · + λn−1)

9. Given λ1 , . . . , λn , define sk = p1 λ1k + · · ·+ pn λnk . Set, for i, j = 0, . . . , n − 1, ai j = si+ j . Show that   s  0  .   det(ai j ) =  .   .   sn−1

 . . . sn−1  . . . .   . . . .  = p1 · · · pn ∏(λi − λ j )2 .  i> j ... .   . . . s2n−2 

10. Show that   1   x 1 + y1   .   .   .   1   x +y n 1

... ... ... ... ...

1 x 1 + yn . . . 1 x n + yn

       ∏i> j (xi − x j )(yi − y j ) = .  ∏i, j (xi + y j )     

This determinant is generally known as the Cauchy determinant.

3.6 Notes and Remarks The theory of determinants predates the development of linear algebra, and it reached its high point in the nineteenth century. Thomas Muir, in his monumental The Theory of Determinants in the Historical Order of Development, Macmillan, 4 volumes, 1890-1923, (reprinted by Dover Books 1960), begins with Leibniz in 1693. However, a decade earlier, the Japanese mathematician Seki Kowa developed a theory of determinants; see Joseph (2000). The first systematic study is due to Cauchy, who apparently coined the term determinant in the sense we use today, and made important contributions, including the multiplication rule and the expansion rules that are named after him. The axiomatic treatment presented here is due to Kronecker and Weierstrass, probably in the 1860s. Their work became known in 1903, when Weierstrass’ On Determinant Theory and Kronecker’s Lectures on Determinant Theory were published posthumously.

Chapter 4

Linear Transformations

4.1 Introduction The study of linear transformations, and their structure, provides the core of linear algebra. We shall study matrix representations of linear transformations, linear functionals, and duality and the adjoint transformation. To conclude, we show how a linear transformation in a vector space induces a module structure over the corresponding ring of polynomials. This simple construction opens the possibility of using general algebraic results as a tool for the study of linear transformations.

4.2 Linear Transformations Definition 4.1. Let V and W be two vector spaces over the field F. A mapping T from V to W , denoted by T : V −→ W , is a linear transformation, or a linear operator, if T (α x + β y) = α (T x) + β (Ty) (4.1) holds for all x, y ∈ V and all α , β ∈ F. For an arbitrary vector space V , we define the zero and identity transformations by 0x = 0 and IV x = x, for all x ∈ V , respectively. Clearly, both are linear transformations. If there is no confusion about the space, we may drop the subscript and write I. The property of linearity of a transformation T is equivalent to the following two properties: • Additivity: T (x + y) = T x + Ty. • Homogeneity: T (α x) = α (T x). The structure of a linear transformation is very rigid. It suffices to know its action on basis vectors in order to determine the transformation uniquely. P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 4, © Springer Science+Business Media, LLC 2012

67

68

4 Linear Transformations

Theorem 4.2. Let V , W be two vector spaces over the same field F. Let {e1 , . . . , en } be a basis for V and let { f1 , . . . , fn } be arbitrary vectors in W . Then there exists a unique linear transformation T : V −→ W for which Tei = fi , for all i = 1, . . . , n. Proof. Every x ∈ V has a unique representation x = ∑ni=1 αi ei . We define T : V −→ W by n

T x = ∑ αi fi . i=1

Clearly Tei = fi , for all i = 1, . . . , n. Next we show that T is linear. Let y ∈ V have the representation y = ∑ni=1 βi ei . Then T (α x + β y) = T ∑ni=1 (ααi + β βi )ei = ∑ni=1 (ααi + β βi ) fi = ∑ni=1 (ααi ) fi + ∑ni=1 (β βi ) fi = α ∑ni=1 αi fi + β ∑ni=1 βi fi = α (T x) + β (Ty). To prove uniqueness, let S : V −→ W be a linear transformation satisfying Sei = fi for all i. Then, for every x ∈ V , n

n

n

i=1

i=1

i=1

Sx = S ∑ αi ei = ∑ αi Sei = ∑ αi fi = T x. This implies T = S.



Given a linear transformation T : V −→ W , there are two important subspaces that are determined by it, namely 1. Ker T ⊂ V , the kernel of T , defined by Ker T = {x ∈ V |T x = 0}. 2. Im T ⊂ W , the image of T , defined by Im T = {T x|x ∈ V }. Theorem 4.3. Let T : V −→ W be a linear transformation. Then Ker T and Im T are subspaces of V and W respectively. Proof. 1. It is clear that Ker T ⊂ V and 0 ∈ Ker T . Let x, y ∈ Ker T and α , β ∈ F. Since T (α x + β y) = α (T x) + β (Ty) = 0, it follows that α x + β y ∈ Ker T . 2. By definition Im T ⊂ W . Let y1 , y2 ∈ Im T . Thus there exist vectors x1 , x2 ∈ V for which yi = T xi . Let α1 , α2 ∈ F. Then

4.2 Linear Transformations

69

α1 y1 + α2 y2 = α1 T x1 + α2 T x2 = T (α1 x1 + α2 x2 ), and α1 y1 + α2 y2 ∈ Im T .



We define the rank of a linear transformation T , denoted by rank(T ), by rank (T ) = dim Im T,

(4.2)

and the nullity of T , denoted null(T ), by null (T ) = dim Ker T.

(4.3)

The rank and nullity of a linear transformation are connected via the following theorem. Theorem 4.4. Let V and W be finite-dimensional vector spaces over F and let T : V −→ W be a linear transformation. Then rank(T ) + null(T ) = dim V ,

(4.4)

dim Im T + dim Ker T = dim V .

(4.5)

or equivalently, Proof. Let {e1 , . . . , en } be a basis of V for which {e1 , . . . , e p } is a basis of KerT . Let x ∈ V , then x has a unique representation of the form x = ∑ni=1 αi ei . Since T is linear, we have Tx = T

n

n

i=1

i=1

∑ αi ei = ∑ αi Tei =

n

∑

αi Tei ,

i=p+1

since Tei = 0 for i = 1, . . . , p. Therefore it follows that {Te p+1 , . . . , Ten } is a spanning set of Im T . We will show that they are also linearly independent. Assume that there exist {α p+1 , . . . , αn } such that ∑ni=p+1 αi (Tei ) = 0. Now ∑ni=p+1 αi (Tei ) = T ∑ni=p+1 αi ei , which implies that ∑ni=p+1 αi ei ∈ Ker T . Hence, it can be represented p αi ei . So we get as a linear combination of the form ∑i=1 p

n

i=1

i=p+1

∑ αi ei − ∑

αi ei = 0.

Using the fact that {e1 , . . . , en } is a basis of V , we conclude that αi = 0 for all i = 1, . . . , n. Thus we have proved that {Te p+1, . . . , Ten } is a basis of Im T . Therefore null(T ) = p, rank (T ) = n − p, and null(T ) + rank (T ) = p + (n − p) = n = dim V .  Let U and V be vector spaces over the field F. We denote by L(U , V ), or Hom F (U , V ), the space of all linear transformations from U to V . Given two transformations T, S ∈ L(U , V ), we define their sum by

70

4 Linear Transformations

(T + S)x = T x + Sx, and the product with any scalar α ∈ F by (α T )x = α (T x). Theorem 4.5. With the previous definitions, L(U , V ) is a vector space. Proof. We show that T + S and α T are indeed linear transformations. Let x, y ∈ V , α ∈ F. We have (T + S)(x + y) = T (x + y) + S(x + y) = (T x + Ty) + (Sx + Sy) = (T x + Sx) + (Ty + Sy) = (T + S)x + (T + S)y. Similarly, (T + S)(α x) = T (α x) + S(α x) = α (T x) + α (Sx) = α (T x + Sx) = α ((T + S)x). This shows that T +S is a linear transformation. The proof for α T goes along similar lines. We leave it to the reader to verify that all the axioms of a linear space hold for L(U , V ).  It is clear that the dimension of L(U , V ) is determined by the spaces U and V . The next theorem characterizes it. Theorem 4.6. Let U and V be vector spaces over the field F. Then dim L(U , V ) = dim U · dimV .

(4.6)

Proof. Let {e1 , . . . , en } be a basis for U and { f1 , . . . , fm } a basis for V . We proceed to construct a basis for L(U , V ). By Theorem 4.2 a linear transformation is determined by its values on basis elements. We define a set of mn linear transformations Ei j , i = 1, . . . , m, j = 1, . . . , n, by letting Ei j ek = δ jk fi ,

k = 1, . . . , n.

(4.7)

Here δi j is the Kronecker delta function, defined by (2.4). On the vector x = ∑nk=1 αk ek the transformation Ei j acts as Ei j x = Ei j or

n

n

n

k=1

k=1

k=1

∑ αk ek = ∑ αk Ei j ek = ∑ αk δ jk fi = α j fi , Ei j x = α j f i .

4.2 Linear Transformations

71

We now show that the set of transformations Ei j forms a basis for L(U , V ). To prove this we have to show that this set is linearly independent and spans L(U , V ). We begin by showing linear independence. Assume there exist αi j ∈ F for which m

n

∑ ∑ αi j Ei j = 0.

i=1 j=1

We operate with the zero transformation, in both its representations, on an arbitrary basis element ek in V : m

0 = 0 · ek = ∑

n

m

n

m

∑ αi j Ei j ek = ∑ ∑ αi j δ jk fi = ∑ αik fi .

i=1 j=1

i=1 j=1

i=1

Since the fi are linearly independent, we get αik = 0 for all i = 1, . . . , m. The index k was chosen arbitrarily; hence we conclude that all the αi j are zero. This completes the proof of linear independence. Let T be an arbitrary linear transformation in L(U , V ). Since Tek ∈ V , it has a unique representation of the form Tek = ∑m i=1 bik f i , k = 1, . . . , n. We show now that n T = ∑m b E . Of course, by Theorem 4.2, it suffices to prove the equality ∑ i=1 j=1 i j i j on the basis elements in U . We compute n m n m n (∑m i=1 ∑ j=1 bi j Ei j )ek = ∑i=1 ∑ j=1 bi j (Ei j ek ) = ∑i=1 ∑ j=1 bi j δ jk f i = ∑m i=1 bik f i = Tek .



Since k can be chosen arbitrarily, the proof is complete.

In some cases linear transformations can be composed, or multiplied. Let U , V , W be vector spaces over F. Let S ∈ L(V , W ) and T ∈ L(U , V ), i.e., T

S

U −→ V −→ W . We define the transformation ST : U −→ W by (ST )x = S(T x),

∀x ∈ U .

(4.8)

We call the transformation ST the composition, or product, of S and T . Theorem 4.7. The transformation ST : U −→ W is linear, i.e., ST ∈ L(U , W ). Proof. Let x, y ∈ U and α , β ∈ F. By the linearity of S and T we get (ST )(α x + β y) = S(T (α x + β y)) = S(α (T x) + β (Ty)) = α S(T x) + β S(Ty) = α (ST )x + β (ST)y. 

72

4 Linear Transformations

Composition of transformations is associative; that is, if T

S

R

U −→ V −→ W −→ Y, then it is easily checked that with the composition of linear transformations defined by (4.8), the following associative rule holds: (RS)T = R(ST ).

(4.9)

We saw already the existence of the identity map I in each linear space U . Clearly, we have IT = T I = T for every linear transformation T . Definition 4.8. Let T ∈ L(V , W ). We say that T is right invertible if there exists a linear transformation S ∈ L(W , V ) for which T S = IW . Similarly, we say that T is left invertible if there exists a linear transformation S ∈ L(W , V ) for which ST = IV . We say that T is invertible if it is both right and left invertible. Proposition 4.9. If T is both right and left invertible, then the right inverse and the left inverse are equal and will be denoted by T −1 . Proof. Let T S = IW and RT = IV . Then R(T S) = (RT )S implies R = RIW = IV S = S.



Left and right invertibility are intrinsic properties of the transformation T , as follows from the next theorem. Theorem 4.10. Let T ∈ L(V , W ). Then 1. T is right invertible if and only if T is surjective. 2. T is left invertible if and only if T is injective. Proof. 1. Assume T is right invertible. Thus, there exists S ∈ L(W , V ) for which T S = IW . Therefore, for all x ∈ W , we have T (Sx) = (T S)x = IW x = x, or x ∈ Im T , and so Im T = W and T is surjective. Conversely, assume T is surjective. Let { f 1 , . . . , fm } be a basis in W and let {e1 , . . . , em } be vectors in V satisfying Tei = fi , for i = 1, . . . , m. Clearly, the

4.2 Linear Transformations

73

vectors {e1 , . . . , em } are linearly independent. We define a linear transformation S ∈ L(W , V ) by  ei , i = 1, . . . , m, S fi = 0, i = m + 1, . . ., n. Then, for all i, (T S) fi = T (S fi ) = Tei = fi , i.e., T S = IW . 2. Assume T is left invertible. Let S ∈ L(W , V ) be a left inverse, that is, ST = IV . Let x ∈ Ker T . Then x = IV x = (ST )x = S(T x) = S(0) = 0, that is, KerT = {0}. Conversely, assume Ker T = {0}. Let {e1 , . . . , en } be a basis in V . It follows from our assumption that {Te1 , . . . , Ten } are linearly independent vectors in W . We complete this set of vectors to a basis {Te1 , . . . , Tem , fm+1 , . . . , fn }. We now define a linear transformation S ∈ L(W , V ) by  S

Tei = ei , T fi = 0,

i = 1, . . . , m, i = m + 1, . . ., n.

Obviously, STei = ei for all i, and hence ST = IV .



It should be noted that the left inverse is generally not unique. In our construction, we had the freedom to define S differently on the vectors { fm+1 , . . . , fn }, but we opted for the simplest definition. The previous characterizations are special cases of the more general result. Theorem 4.11. Let U , V , W be vector spaces over F. 1. Assume A ∈ L(U , W ) and B ∈ L(U , V ). Then there exists C ∈ L(V , W ) such that A = CB if and only if Ker A ⊃ Ker B.

(4.10)

2. Assume A ∈ L(U , W ) and B ∈ L(V , W ). Then there exists C ∈ L(U , V ) such that A = BC if and only if Im A ⊂ Im B. Proof. 1. If A = CB, then for x ∈ Ker B we have Ax = C(Bx) = C0 = 0, i.e., x ∈ Ker A and (4.10) follows.

(4.11)

74

4 Linear Transformations

Conversely, assume Ker A ⊃ Ker B. Let {e1 , . . . , er , er+1 , . . . , e p , e p+1 , . . . , en } be a basis in U such that {e1 , . . . , e p } is a basis for Ker A and {e1 , . . . , er } is a basis for Ker B. The vectors {Ber+1 , . . . , Ben } are linearly independent in V . We complete them to a basis of V and define a linear transformation C : V −→ W by  Aei , i = p + 1, . . ., n, CBei = 0, i = r + 1, . . . , p, and arbitrarily on the other basis elements of V . Then, for every x = ∑ni=1 αi ei ∈ U we have Ax = A ∑ni=1 αi ei = ∑ni=1 αi Aei = ∑ni=p+1 αi (Aei ) = ∑ni=p+1 αi (CB)ei = CB ∑ni=p+1 αi ei = CB ∑ni=1 αi ei = CBx, and so A = CB. 2. If A = BC, then for every x ∈ U we have Ax = (BC)x = B(Cx), or Im A ⊂ Im B. Conversely, assume Im A ⊂ Im B. Then we let {e1 , . . . , er , er+1 , . . . , en } be a basis for U such that {er+1 , . . . , en } is a basis for Ker A. Then {Ae1 , . . . , Aer } are linearly independent vectors in W . By our assumption, Im A ⊂ Im B, there exist vectors {g1 , . . . , gr } in V , necessarily linearly independent, such that Aei = Bgi ,

i = 1, . . . , r.

We complete them to a basis {g1 , . . . , gr , gr+1 , . . . , gm } of W . We define now C ∈ L(U , V ) by  gi , i = 1, . . . , r, Cei = 0, i = r + 1, . . . , n. For x = ∑ni=1 αi ei ∈ U , we get A ∑ni=1 αi ei = ∑ri=1 αi Aei = ∑ri=1 αi Bgi = B ∑ri=1 αi gi = B ∑ri=1 αiCei = B ∑ni=1 αiCei = BC ∑ni=1 αi ei = BCx, and A = BC.



Theorem 4.12. Let T ∈ L(U , V ) be invertible. Then T −1 ∈ L(V , U ). Proof. Let y1 , y2 ∈ V and α1 , α2 ∈ F. There exist unique vectors x1 , x2 ∈ U such that T xi = yi , or equivalently, xi = T −1 yi . Since

4.3 Matrix Representations

75

T (α1 x1 + α2 x2 ) = α1 T x1 + α2 T x2 = α1 y1 + α2 y2 , therefore T −1 (α1 y1 + α2 y2 ) = α1 x1 + α2 x2 = α1 T −1 y1 + α2 T −1 y2 .



A linear transformation T : U −→ V is called an isomorphism if it is invertible. In this case we will say that U and V are isomorphic spaces. Clearly, isomorphism of vector spaces is an equivalence relation. Theorem 4.13. Every n-dimensional vector space U over F is isomorphic to Fn . Proof. Let B = {e1 , . . . , en } be a basis for U . The map x → [x]B is an isomorphism.  Theorem 4.14. Two finite-dimensional vector spaces V and W over F are isomorphic if and only if they have the same dimension. Proof. If U and V have the same dimension, then we can construct a map that maps the basis elements in U onto the basis elements in V . This map is necessarily an isomorphism. Conversely, if U and V are isomorphic, the image of a basis in U is necessarily a basis in V . Thus dimensions are equal. 

4.3 Matrix Representations Let U , V be vector spaces over a field F. Let BU = {e1 , . . . , en } and BV = { f1 , . . . , fm } be bases of U and V respectively. We saw before that the set of linear transformations {Ei j }, i = 1, . . . , m, j = 1, . . . , n, defined by Ei j ek = δ jk fi ,

(4.12)

is a basis for L(U , V ). We denote this basis by BU × BV . A natural problem presents itself, namely finding the coordinate vector of a linear transformation T ∈ L(U , V ) with respect to this basis. We observed before, in Theorem 4.6, that T can be written as m

T=∑

n

∑ t i j Ei j .

(4.13)

i=1 j=1

Hence m

n

m

n

m

Tek = ∑ ∑ ti j Ei j ek = ∑ ∑ ti j δ jk fi = ∑ tik fi , i=1 j=1

i=1 j=1

i=1

(4.14)

76

4 Linear Transformations

i.e.,

m

Tek = ∑ tik fi .

(4.15)

i=1

Therefore, the coordinate vector of T with respect to the basis BU × BV will have the entries tik arranged in the same order in which the basis elements are ordered. In contrast to the case of an abstract vector space, we will arrange the coordinates in this case in an m × n matrix (ti j ) and call it the matrix representation of T with respect to the bases BU in U and BV in V . We will use the following notation: B

[T ]BV = (tik ). U

(4.16)

The importance of the matrix representation stems froms the following theorem, which reduces the application of an arbitrary linear transformation to matrix multiplication. Theorem 4.15. Let U and V be n-dimensional and m-dimensional vector spaces with respective bases BU and BV , and let T : U −→ V be a linear transformation. Then the following diagram is commutative: T U V

[ · ]BU

[ · ]BV ? F

B

[T ]BV

U

n

-

? Fm

In other words, we have BU V [T x]BV = [T ]B . BU [x]

Proof. Assume x = ∑nj=1 α j e j and Te j = ∑m i=1 ti j f i . Then m n T x = T ∑nj=1 α j e j = ∑nj=1 α j ∑m i=1 ti j f i = ∑i=1 (∑ j=1 ti j α j ) f i .

Thus

⎞ ∑nj=1 t1 j α j ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ B =⎜ . ⎟ = [T ]BVU [x]BU . ⎟ ⎜ ⎠ ⎝ . n ∑ j=1 tm j α j ⎛

[T x]BV



4.3 Matrix Representations

77

The previous theorem shows that by choosing bases, we can pass from an abstract representation of spaces and transformations to a concrete one in the form of column vectors and matrices. In the latter form, computations are easily mechanized. The next theorem deals with the matrix representation of the product of two linear transformations. Theorem 4.16. Let U , V , W be vector spaces over a field F of dimensions n, m, and p respectively. Let T ∈ L(U , V ) and S ∈ L(V , W ). Let BU = {e1 , . . . , en }, BV = { f1 , . . . , fm }, and BW = {g1 , . . . , g p } be bases in U , V , and W respectively. Then B

B

B

[ST ]BW = [S]BW [T ]BV . U

V

(4.17)

U

p p Proof. Let Te j = ∑m k=1 tk j f k , S f k = ∑i=1 sik gi and (ST )e j = ∑i=1 ri j gi . Therefore

(ST )e j = S(Te j ) = S ∑m k=1 tk j f k p

m = ∑m k=1 tk j S f k = ∑k=1 tk j ∑i=1 sik gi p = ∑i=1 [∑m k=1 sik tk j ]gi .

Since {g1 , . . . , g p } is a basis for W , we get ri j =

m

∑ siktk j ,

k=1



and (4.17) follows. Clearly, we have proved the commutativity of the following diagram: U

T

S

-V

[ · ] BU

-W

[ · ]BV ? Fn

B

[T ]BV

U

-

? Fm

[ · ]BW B

[S]BW V

? - p F

Corollary 4.17. Let the vector space V have the two bases B1 and B2 . Then we have B2 B1 1 [I]B (4.18) B2 [I]B1 = [I]B1 = I, or alternatively,

−1

B2 1 = [I] . [I]B B2 B1

(4.19)

78

4 Linear Transformations

An important special case of the previous result is that of the relation between two matrix representations of a given linear transformation, taken with respect to two different bases. We begin by noting that the following diagram commutes. I

V

I

-V

[ · ]B1

-V

[ · ] B2 2 [I]B B1

? Fn

-

[ · ]B1 1 [I]B B2

? Fn

? - n F

Theorem 4.18. Let T be a linear transformation in a vector space V . Let B1 and B2 be two bases of V . Then we have B2 B1 B1 2 [T ]B B2 = [I]B1 [T ]B1 [I]B2 ,

or

(4.20)

−1 B1 B1 2 1 [T ]B [T ]B B2 = [I]B2 B1 [I]B2 .

(4.21)

Proof. We observe the following commutative diagram, from which the result immediately follows: I V

-

[ · ]B2 ? Fn

T V

-

I V

[ · ]B1 1 [I]B B2

-

? Fn

-

[ · ]B1 1 [T ]B B1

-

? Fn

V

[ · ]B2 2 [I]B B1

-

? Fn 

Equation (4.21) leads us to the following definition. Definition 4.19. Given matrices A, B ∈ Fn×n , we say that A and B are similar, and write A  B, if there exists an invertible matrix R ∈ Fn×n for which A = R−1 BR. Clearly, similarity is an equivalence relation.

(4.22)

4.4 Linear Functionals and Duality

79

Theorem 4.18 plays a key role in the study of linear transformations. The structure of a linear transformation is determined through the search for simple matrix representations, which is equivalent to finding appropriate bases. The previous theorem’s content is that through a change of basis the matrix representation of a given linear transformation undergoes a similarity transformation.

4.4 Linear Functionals and Duality We note that a field F is at the same time a one-dimensional vector space over itself. Given a vector space V over a field F, a linear transformation from V to F is called a linear functional. The set of all linear functionals on V , i.e., L(V , F), is called the dual space of V and will be denoted by V ∗ . Thus V ∗ = L(V , F).

(4.23)

Given elements x ∈ V and x∗ ∈ V ∗ , we will write also x∗ (x) = x, x∗ 

(4.24)

and call x, x∗  a duality pairing of V and V ∗ . Note that x, x∗  is linear in both variables. More generally, given F-vector spaces V , W , we say that a map v, w : V × W −→ F is a bilinear pairing or a bilinear form if v, w is linear in both variables, i.e.,

α1 v1 + α2 v2 , w = α1 v1 , w + α2 v2 , w, (4.25)

v, α1 w1 + α2 w2  = α1 v, w1  + α2 v, w2 , for all αi ∈ F, vi ∈ V , wi ∈ W . Examples: • Let V = Fn . Then, for α1 , . . . , αn ∈ F, the map f : Fn −→ F defined by ⎛ ⎞ x1 ⎜ . ⎟ ⎜ ⎟ n ⎜ ⎟ f ⎜ . ⎟ = ∑ αi xi ⎜ ⎟ i=1 ⎝ . ⎠ xn defines a linear functional. • Let Fn×n be the space of all square matrices of order n. Given a linear transformation T acting in a finite-dimensional vector space U , we define the trace of T , Trace(T ), by n

Trace(T ) = ∑ tii , i=1

(4.26)

80

4 Linear Transformations

where (ti j ) is a matrix representation of T with respect to an arbitrary basis of U . It can be shown directly that the trace is independent of the particular choice of basis. Let X , A ∈ Fn×n . Then f defined by f (X ) = Trace(AX)

(4.27)

is a linear functional. Let V be an n-dimensional vector space, and let B = {e1 , . . . , en } be a basis for V . Thus, each vector x ∈ V has a unique representation of the form x = ∑nj=1 α j e j . We define functionals fi by fi (x) = αi ,

i = 1, . . . , n.

It is easily checked that the fi so defined are linear functionals. To see this, let y = ∑nj=1 β j e j . Then fi (x + y) = fi (∑nj=1 α j e j + ∑nj=1 β j e j ) = fi (∑nj=1 (α j + β j )e j ) = αi + βi = fi (x) + fi (y). In the same way, fi (α x) = f i (α ∑nj=1 α j e j ) = fi (∑nj=1 (αα j )e j ) = ααi = α fi (x). Since clearly, ei = ∑nj=1 δi j e j , we have fi (e j ) = δi j for 1 ≤ i, j ≤ n. Theorem 4.20. Let V be an n-dimensional vector space, and let B = {e1 , . . . , en } be a basis for V . Then 1. We have

dim V ∗ = dim V .

(4.28)

2. The set of linear functionals { f1 , . . . , fn } defined, through linear extensions, by fi (e j ) = δi j is a basis for V ∗ . Proof. 1. We apply Theorem 4.6, i.e., the fact that dim L(U , V ) = dim U dim V and the fact that dim F = 1. 2. The functionals { f 1 , . . . , fn } are linearly independent. For let ∑ni=1 αi fi = 0. Then 0 = 0 · ej =

n

∑ αi fi

i=1

n

n

i=1

i=1

e j = ∑ αi fi (e j ) = ∑ αi δi j = α j .

Therefore α j = 0 for all j, and linear independence is proved. Since dim V ∗ = n, it follows that { f1 , . . . , fn } is a basis for V ∗ . 

4.4 Linear Functionals and Duality

81

Definition 4.21. Let V be an n-dimensional vector space, and let B = {e1 , . . . , en } be a basis for V . Then the basis { f1 , . . . , fn } defined by fi (e j ) = δi j is called the dual basis to B and will be denoted by B ∗ . Definition 4.22. Let S be a subset of a vector space V . We denote by S ⊥ the subset of V ∗ defined by S ⊥ = { f ∈ V ∗ | f (s) = 0, for all s ∈ S }. The set S ⊥ is called the annihilator of S . Proposition 4.23. Let S be a subset of the vector space V . Then 1. The set S ⊥ is a subspace of V ∗ . 2. If Q ⊂ S then S ⊥ ⊂ Q⊥ . 3. We have S ⊥ = (span (S ))⊥ . Proof. 1. Let f , g ∈ S ⊥ and α , β ∈ F. Then for an arbitrary x ∈ S , (α f + β g)x = α f (x) + β g(x) = α · 0 + β · 0 = 0, that is, α f + β g ∈ S ⊥ . 2. Let f ∈ S ⊥ . Then f (x) = 0 for all x ∈ S and particularly, by the inclusion Q ⊂ S , for all x ∈ Q. So f ∈ Q ⊥ . 3. It is clear that S ⊂ span (S ), and therefore span(S )⊥ ⊂ S ⊥ . On the other hand, if f ∈ S ⊥ , xi ∈ S and αi ∈ F, then f

n

∑ αi xi

i=1

n

n

i=1

i=1

= ∑ αi f (xi ) = ∑ αi · 0 = 0,

which means that f annihilates all linear combinations of elements of S . So f ∈ span (S )⊥ or S ⊥ ⊂ span (S )⊥ . From these two inclusions the equality S ⊥ = span (S )⊥ follows.  We will find the following proposition useful in the study of duality. Proposition 4.24. Let V be a vector space and M its subspace. Then the dual space to the quotient space V /M is isomorphic to M ⊥ . Proof. Let φ ∈ M ⊥ . Then φ induces a linear functional Φ on V /M by Φ ([x]) = φ (x). The functional Φ is well defined, since [x1 ] = [x2 ] if and only if x1 − x2 ∈ M , and this implies φ (x1 ) − φ (x2 ) = φ (x1 − x2 ) = 0. The linearity of Φ follows from the linearity of φ . Conversely, given Φ ∈ (V /M )∗ , we define φ by φ (x) = Φ ([x]). Clearly φ (x) = 0 for x ∈ M , i.e., φ ∈ M ⊥ . Finally, we note that the map φ → Φ is linear, which proves the required isomorphsm. 

82

4 Linear Transformations

Theorem 4.25. Let V be an n-dimensional vector space. Let M be a subspace of V and M ⊥ its annihilator. Then dim V = dim M + dim M ⊥ . Proof. Let {e1 , . . . , en } be a basis for V , with {e1 , . . . , em } a basis for M . Let { f1 , . . . , fn } be the dual basis. We will show that { fm+1 , . . . , fn } is a basis for M ⊥ . The linear independence of the vectors { fm+1 , . . . , fn } is clear, since they are a subset of a basis. Moreover, they are clearly in M ⊥ . It remains to show that they actually span M ⊥ . So let f ∈ M ⊥ . It can be written as f = ∑nj=1 α j f j . But since f ∈ M ⊥ , we have f (e1 ) = · · · = f (em ) = 0. Since f (ei ) = ∑nj=1 α j f j (ei ) = ∑nj=1 α j δ ji = αi , we conclude that α1 = · · · = αm = 0 and therefore f = ∑nj=m+1 α j f j . The proof is  complete, since dim M = m and dim M ⊥ = n − m. Let U be a vector space over F. We define a hyperspace M to be a maximal nontrivial subspace of U . Proposition 4.26. Let dim U = n. The following statements are equivalent. 1. M is a hyperspace. 2. dim M = n − 1. 3. M is the kernel of a nonzero functional φ . Proof. (1) ⇒ (2). Assume M is a hyperspace. Fix a nonzero vector x ∈ / M . Then span (x, M ) = {α x + m | α ∈ F, m ∈ M } is a subspace that properly contains M . Thus necessarily span (x, M ) = U . Let {e1 , . . . , ek } be a basis for M . Then {e1 , . . . , ek , x} are linearly independent and span U , so they are a basis for U . Necessarily k = dim M = n − 1. (2) ⇒ (3). Let M be an (n − 1) dimensional subspace of U . Let x ∈ / M . Define a linear functional φ by setting φ (x) = 1, φ |M = 0. For each u ∈ U there exist unique γ ∈ F and m ∈ M such that u = γ f + m. It follows that φ (u) = φ (γ f + m) = γφ ( f ) + φ (m) = γ . Therefore, u ∈ Ker φ if and only if u ∈ M . (3) ⇒ (1). Assume M = Ker φ with φ a nontrivial functional. Fix f ∈ / M , which exists by the nontriviality of φ . We show now that every u ∈ U has a unique representation of the form u = γ f + m with m ∈ M . In fact, it suffices to take γ = φφ ((u)  f ) . Thus M is a hyperspace. Recalling Definition 2.25, where the codimension of a space was introduced, we can extend Proposition 4.26 in the following way. Proposition 4.27. Let U be an n-dimensional vector space over F, and let M be a subspace. Then the following statements are equivalent:

4.4 Linear Functionals and Duality

83

1. M has codimension k. 2. We have dim M = n − k. 3. M is the intersection of the kernels of k linearly independent functionals. Proof. (1) ⇒ (2). Let f1 , . . . , fk be linearly independent vectors over M and let L(M , f1 , . . . , fk ) = U . Let e1 , . . . , e p be a basis for M . We claim that {e1 , . . . , e p , f1 , . . . , fk } is a basis for U . The spanning property is obvious. To see linear p independence, assume there exist αi , βi ∈ F such that ∑i=1 αi ei + ∑ki=1 βi fi = 0. p This implies ∑ki=1 βi fi = − ∑i=1 αi ei ∈ M . This implies in turn that all βi are zero. From this we conclude that all αi are zero. Thus we must have that p + k = n, or dim M = n − k. (2) ⇒ (3). Assume dim M = n − k. Let {e1 , . . . , en−k } be a basis for M . We extend it to a basis {e1 , . . . , en }. Define, using linear extensions, k linear functionals φ1 , . . . , φk such that

φi (e j ) = δi j ,

j = n − k + 1, . . ., n.

Obviously, φ1 , . . . , φk are linearly independent and ∩i Ker φi = M . (3) ⇒ (1). Assume φ1 , . . . , φk are linearly independent and ∩i Ker φi = M . We choose a sequence of vectors xi inductively. We choose a vector x1 such that φ1 (x1 ) = 1. Suppose we have chosen already x1 , . . . , xi−1 . Then we choose xi such that xi ∈ ∩i−1 j=1 Ker φ j and φi (xi ) = 1. To see that this is possible, we note that if this is not the case, then ⎞ ⎛ φ1 ⎟ ⎜. ⎟ ⎜ ⎟ ⎜ i−1 Ker φi ⊃ ∩ j=1 Ker φ j = Ker ⎜ . ⎟. ⎟ ⎜ ⎠ ⎝. φi−1 Using Theorem 4.11, it follows that there exist α j such that φi = ∑i−1 j=1 α j φ j , contrary to the assumption of linear independence. Clearly, the vectors x1 , . . . , xk are linearly independent. Moreover, it is easy to check that x − ∑kj=1 α j x j ∈ ∩kj=1 Ker φ j . This implies dim M = n − k.  Let V be a vector space over F and let V ∗ be its dual space. Thus, given x ∈ V and f ∈ V ∗ we have f (x) ∈ F. But, fixing a vector x ∈ F, we can view the expression f (x) as a F-valued function defined on the dual space V ∗ . We denote this function by x, ˆ and it is defined by x( ˆ f ) = f (x). (4.29) Theorem 4.28. The function xˆ : V ∗ −→ F, defined by (4.29), is a linear map. Proof. Let f , g ∈ V ∗ and α , β ∈ F. Then x( ˆ α f + β g) = (α f + β g)(x) = (α f )(x) + (β g)(x) ˆ f ) + β x(g). ˆ = α f (x) + β g(x) = α x(



84

4 Linear Transformations

Together with the function xˆ ∈ L(V ∗ , F) = V ∗∗ , we have the function φ : V −→ V defined by φ (x) = x. ˆ (4.30) ∗∗

The function φ so defined is referred to as the canonical embedding of V in V ∗∗ . Theorem 4.29. Let V be a finite-dimensional vector space over F. The function φ : V −→ V ∗∗ defined by (4.30) is injective and surjective, i.e., it is an invertible linear map. Proof. We begin by showing the linearity of φ . Let x, y ∈ V , α , β ∈ F, and f ∈ V ∗ . Then (φ (α x + β y)) f = f (α x + β y) = α f (x) + β f (y) = α x( ˆ f ) + β y( ˆ f ) = (α xˆ + β y)( ˆ f) = (αφ (x) + β φ (y)) f . Since this is true for all f ∈ V ∗ , we get

φ (α x + β y) = αφ (x) + β φ (y), i.e., the canonical map φ is linear. It remains to show that φ is invertible. Since dim V ∗ = dim V , we get also dim V ∗∗ = dim V . So, to show that φ is invertible, it suffices to show injectivity. To this end, let x ∈ Ker φ . Then for each f ∈ V ∗ we have ˆ f ) = f (x). (φ (x))( f ) = 0 = x( This implies x = 0. For if x = 0, there exists a functional f for which f (x) = 0. The easiest way to see this is to complete x to a basis and take the dual basis.  Corollary 4.30. Let L ∈ V ∗∗ . Then there exists a x ∈ V such that L = x. ˆ Corollary 4.31. Let V be an n-dimensional vector space and let V ∗ be its dual space. Let { f1 , . . . , fn } be a basis for V ∗ . Then there exists a basis {e1 , . . . , en } for V for which fi (e j ) = δi j , i.e., every basis in V ∗ is the dual of a basis in V . Proof. Let {E1 , . . . , En } be the basis in V ∗∗ that is dual to the basis { f1 , . . . , fn } of V ∗ . Now, there exist unique vectors {e1 , . . . , en } in V for which Ei = eˆi . So, for each f ∈ V ∗ , we have Ei ( f ) = eˆi ( f ) = f (ei ), and in particular,

f j (ei ) = Ei ( f j ) = δi j .



4.5 The Adjoint Transformation

85

Let now M ⊂ V ∗ be a subspace. We define ⊥

M = ∩ f ∈M Ker f = {x ∈ V | f (x) = 0, ∀ f ∈ M }.

Theorem 4.32. We have

φ (⊥ M ) = M ⊥ .

ˆ f) = Proof. We have x ∈ ∩ f ∈M Ker f if and only if for each f ∈ M we have x( ⊥ f (x) = 0. However, the last condition is equivalent to xˆ ∈ M .  Corollary 4.33. Let V be an n-dimensional vector space and M an (n − k)dimensional subspace of V ∗ . Let φ (⊥ M ) = ∩ f ∈M Ker f . Then dim φ (⊥ M ) = k. Proof. We have dim V ∗ = n = dim M + dim M ⊥ = n − k + dimM ⊥ , that is, dimM ⊥ = k. We conclude by observing that dim φ (⊥ M ) = dim M ⊥ .



p Theorem 4.34. Let f , g1 , . . . , g p ∈ V ∗ . Then Ker f ⊃ ∩i=1 Ker gi if and only if f ∈ span (g1 , . . . , g p ).

Proof. Define the map g : V −→ F p by ⎞ g1 (x) ⎜ . ⎟ ⎟ ⎜ ⎟ ⎜ g(x) = ⎜ . ⎟ . ⎟ ⎜ ⎝ . ⎠ g p (x) ⎛

(4.31)

p Obviously, we have the equality Ker g = ∩i=1 Ker gi . We apply Theorem 4.11 to   conclude that there exists a map α = α1 . . . α p : F p −→ F such that f = α g = p  ∑i=1 αi gi .

4.5 The Adjoint Transformation Let U and V be two vector spaces over the field F. Assume T ∈ L(U , V ) and f ∈ V ∗ . Let us consider the composition of maps T

f

U −→ V −→ F. It is clear that the product, or composition, of f and T , i.e., f T , is a linear transformation from U to F. This means that f T ∈ U ∗ and we denote this

86

4 Linear Transformations

functional by T ∗ f . Therefore T ∗ : V ∗ −→ U ∗ is defined by T ∗ f = f T,

(4.32)

or (T ∗ f )u = f (Tu),

u∈U.

(4.33)

∗

The transformation T is called the adjoint transformation of T . In terms of the duality pairing f (v) = v, f , we can write (4.33) as

Tu, f  = u, T ∗ f .

(4.34)

Theorem 4.35. The transformation T ∗ is linear, i.e., T ∗ ∈ L(V ∗ , U ∗ ). Proof. Let f , g ∈ V ∗ and α , β ∈ F. Then, for every x ∈ U , (T ∗ (α f + β g))x = (α f + β g))(T x) = α f (T x) + β g(T x) = α (T ∗ f )x + β (T ∗ g)x = (α T ∗ f + β T ∗ g)x. Therefore T ∗ (α f + β g) = α T ∗ f + β T ∗ g, which proves linearity. Theorem 4.36. Let T ∈ L(U , V ). Then (Im T )⊥ = Ker T ∗ .

(4.35)

Proof. For every x ∈ U and f ∈ V ∗ we have (T ∗ f )x = f (T x), so f ∈ (Im T )⊥ if and only if for all x ∈ U , we have 0 = f (T x) = (T ∗ f )x, which is equivalent to f ∈ Ker T ∗ .  Corollary 4.37. Let T ∈ L(U , V ). Then rank (T ) = rank (T ∗ ).

(4.36)

Proof. Let dim U = n and dim V = m. Assume rank T = dim Im T = p. Now m = dim V = dim V ∗ = dim Im T ∗ + dim Ker T ∗ = rank T ∗ + dimKer T ∗. Since we have dim Ker T ∗ = dim(Im T )⊥ = m − dim(Im T ) = m − rankT = m − p, it follows that rank T ∗ = p.



4.5 The Adjoint Transformation

87

Theorem 4.38. Let T ∈ L(U , V ) and let BU = {e1 , . . . , en } be a basis in U ∗ = {φ , . . . , φ } and B ∗ = and let BV = { f1 , . . . , fm } be a basis in V . Let BU n 1 V ∗ ∗ {ψ1 , . . . , ψm } be the dual bases in U and V respectively. Then B∗  B [T ∗ ]BU∗ = [T ]BV . V

Proof. We recall that

U

B

([T ]BVU )i j = ψi (Te j ), ∗ . Now we know that so in order to compute, we have to find the dual basis to BU ∗∗ BU = {eˆ1 , . . . , eˆn }, so B∗

B

([T ∗ ]BU∗ )i j = eˆi (T ∗ ψ j ) = (T ∗ ψ j )ei = ψ j (Tei ) = ([T ]BVU ) ji . V

 Corollary 4.39. Let A be an m × n matrix. Then the row and column ranks of A are equal. 

Proof. Follows from (4.36). T∗

= T. We consider now a special class of linear transformations T that satisfy Assuming T ∈ L(U , V ), it follows that T ∗ ∈ L(V ∗ , U ∗ ). For the equality T ∗ = T to hold, it is therefore necessary that V = U ∗ and V ∗ = U . Of course V = U ∗ implies V ∗ = U ∗∗ , and therefore V ∗ = U will hold only if we identify U ∗∗ with U , which we can do using the canonical embedding. Let now x, y ∈ U . Then we have (T x)(y) = y(T ˆ x) = (T ∗ y)(x) ˆ = x(T ˆ ∗ y). ˆ

(4.37)

If we rewrite now the action of a functional x∗ on a vector x by

we can rewrite (4.37) as

x∗ (x) = x, x∗ ,

(4.38)

T x, y = x, T ∗ y.

(4.39)

We say that a linear transformation T ∈ L(U , U ∗ ) is self-dual if T ∗ = T , or equivalently, for all x, y ∈ U we have T x, y = x, Ty. Proposition 4.40. Let U be a vector space and U ∗ its dual, B a basis in U and B ∗ its dual basis. Let T ∈ L(U , U ∗ ) be a self-dual linear transformation. Then ∗ [T ]B B is a symmetric matrix. Proof. We use the fact that U ∗∗ = U and B ∗∗ = B. Then

88

4 Linear Transformations ∗ B  B [T ]B B = [T ]B ∗∗ = [T ]B . ∗

∗

∗

 ∑ni=1 ξi ei .

If B = {e1 , . . . , en }, then every vector x ∈ U has an expansion x = ∗ This leads to T x, x = ∑ni=1 ∑nj=1 Ti j ξi ξ j , where (Ti j ) = [T ]B B . The expression n n ∑i=1 ∑ j=1 Ti j ξi ξ j is called a quadratic form. We will return to this topic in much greater detail in Chapter 8.

4.6 Polynomial Module Structure on Vector Spaces Let U be a vector space over the field F. Given a linear transformation T in U , there is in U a naturally induced module structure over the ring F[z]. This module structure, which we proceed to define, will be central to our study of linear transformations. Given a polynomial p(z) ∈ F[z], p(z) = ∑kj=0 p j z j , and a linear transformation T in a vector space U over F, we define p(T ) =

k

∑ p jT j,

(4.40)

j=0

with T 0 = I. The action of a polynomial p(z) on a vector x ∈ U is defined by p · x = p(T )x.

(4.41)

Proposition 4.41. Given a linear transformation T in a vector space U over F, the map p(z) → p(T ) is an algebra homomorphism of F[z] into L(U ). Proof. Clearly (α p + β q)(T ) = α p(T ) + β q(T ) and (pq)(T ) = p(T )q(T ).



Given two spaces U1 , U2 and linear transformations Ti ∈ L(Ui ), a linear transformation X : U1 −→ U2 is said to intertwine T1 and T2 if XT1 = T2 X .

(4.42)

Obviously (4.42) implies, for an arbitrary polynomial p(z), that X p(T1 ) = p(T2 )X. Thus, for x ∈ U1 , we have X (p · x) = p · Xx. This shows that intertwining maps are F[z] module homomorphisms from U1 to U2 . Two operators Ti are said to be similar, and we write T1  T2 , if there exists an invertible map intertwining them. Equivalently, T2 = RT1 R−1 , where R : U1 −→ U2 is invertible. A natural strategy for studying the similarity of two transformations is to characterize intertwining maps and find conditions guaranteeing their invertibility.

4.6 Polynomial Module Structure on Vector Spaces

89

Definition 4.42. Let U be an n-dimensional vector space over the field F, and T : U −→ U a linear transformation. 1. A subspace M ⊂ U is called an invariant subspace of T if for all x ∈ M we have also T x ∈ M . 2. A subspace M ⊂ U is called a reducing subspace of T if it is an invariant subspace of T and there exists another invariant subspace N ⊂ U such that U = M ⊕ N. We note that given a linear transformation T in U , T -invariant subspaces are just F[z]-submodules of U relative to the F[z]-module structure defined in U by (4.41). Similarly, reducing subspaces of T are equivalent to module direct summands of the module U . Proposition 4.43. 1. Let M be a subspace of U invariant under T . Let B = {e1 , . . . , en } be a basis for U such that B1 = {e1 , . . . , em } is a basis for M . Then, with respect to this basis, T has the block triangular matrix representation [T ]B B

 =

T11 T12 0 T22

 .

1 Moreover, T11 = [T |M ]B B1 . 2. Let M be a reducing subspace for T and let N be a complementary invariant subspace. Let B = {e1 , . . . , en } be a basis for U such that B1 = {e1 , . . . , em } is a basis for M and B2 = {em+1 , . . . , en } is a basis for N . Then the matrix representation of T with respect to this basis is block diagonal. Specifically,

[T ]B B

 =

T1 0 0 T2

 ,

B2 1 where T1 = [T |M ]B B1 and T2 = [T |N ]B2 .

Corollary 4.44. 1. Let T be a linear transformation in U . Let {0} ⊂ M1 ⊂ · · · ⊂ Mk ⊂ U be invariant subspaces. Let B = {e1 , . . . , en } be a basis for U such that Bi = {en1 +···+ni−1 +1 , . . . , en1 +···+ni } is a basis for Mi . Then ⎛

T11 ⎜ 0 ⎜ ⎜ [T ]B B =⎜ . ⎜ ⎝ . .

⎞ T12 . . T1n T22 . ⎟ ⎟ ⎟ . . . ⎟. ⎟ . . . ⎠ . . 0 Tkk

90

4 Linear Transformations

2. Let U = M1 ⊕ · · ·⊕ Mk , with Mi being T -invariant subspaces. Let Bi be a basis for Mi and let B be the union of the bases Bi . Then ⎛ [T ]B B

⎜ ⎜ ⎜ =⎜ ⎜ ⎝

⎞

T11

⎟ ⎟ ⎟ ⎟. ⎟ ⎠

T22 . . Tkk

We leave the proof of the two previous results to the reader. An invariant subspace of a linear transformation T in U induces two other transformations, namely the restriction T |M of T to the invariant subspace M and the induced transformation T |U /M in the quotient space U /M , which is defined by T |U /M [x]M = [T x]M . (4.43) Proposition 4.45. Let T ∈ L(U ), let M ⊂ U be a T -invariant subspace, and let π : U −→ U /M be the canonical projection, i.e., the map defined by

π (x) = [x]M .

(4.44)

Then 1. There exists a unique linear transformation T : U /M −→ U /M that makes the following diagram commutative: π - U /M U T

T ? U

i.e., we have

? - U /M

π

T π x = π T x.

(4.45)

2. Let B = {e1 , . . . , en } be a basis for U such that B1 = {e1 , . . . , em } is a basis for M . Then B = {π em+1, . . . , π en } is a basis for U /M . If [T ]B B = then



T11 T12 0 T22

[T ]B = T22 . B

 ,

(4.46)

(4.47)

4.6 Polynomial Module Structure on Vector Spaces

91

Proof. 1. In terms of equivalence classes modulo M , we have T [x] = [T x]. To show that T is well defined, we have to show that [x] = [y] implies [T x] = [Ty]. This is the direct consequence of the invariance of M under T . 2. Follows from the invariance of M under T .  Definition 4.46. 1. Let M ⊂ U be a T -invariant subspace. The restriction of T to M is the unique linear transformation T |M : M −→ M defined by T |M x = T x,

x ∈ M.

2. The induced map, namely the map induced by T on the quotient space U /M , is the unique map defined by (4.45). We will use the notation T |U /M for the induced map. Invariance and reducibility of linear transformations are conveniently described in terms of projection operators. We turn to that. Assume the vector space U admits a direct sum decomposition U = M ⊕ N . Thus every vector x ∈ U can be written, in a unique way, as x = m + n with m ∈ M and n ∈ N . We define a transformation PN : U −→ U by PN x = m. We call PN the projection on M in the direction of N . Clearly PN is a linear transformation and satisfies Ker PN = N and Im PN = M . Proposition 4.47. A linear transformation P is a projection if and only if P2 = P. Proof. Assume PN is the projection on M in the direction of N . If x = m + n, it is clear that PN m = m, so PN2 x = PN m = m = PN x. Conversely, assume P2 = P. Let M = Im P and N = Ker P. Since for every x ∈ U , we have x = Px + (I − P)x, with Px ∈ Im P and (I − P)x ∈ Ker P, it follows that U = M +N . To show that this is a direct sum decomposition, assume x ∈ M ∩N . This implies x = Px = (I − P)y. This implies in turn that x = Px = P(I − P)y = 0.  Proposition 4.48. A linear transformation P in U is a projection if and only if I − P is a projection. Proof. It suffices to show that (I − P)2 = I − P, and this is immediate.



If U = M ⊕ N , and PN is the projection on M in the direction of N , then PM = I − PN is the projection on N in the direction of M . The next proposition is central. It expresses the geometric conditions of invariance and reducibility in terms of arithmetic conditions involving projection operators. Proposition 4.49. Let P be a projection operator in a vector space X and let T be a linear transformation in X . Then 1. A subspace M ⊂ X is invariant under T if and only if for any projection P on M we have T P = PT P. (4.48)

92

4 Linear Transformations

2. Let X = M ⊕ N be a direct sum decomposition of X and let P be the projection on M in the direction of N . Then the direct sum decomposition reduces T if and only if T P = PT. (4.49) Proof. 1. Assume (4.48) holds, and Im P = M . Then, for every m ∈ M , we have T m = T Pm = PT Pm = PT m ∈ M . Thus M is invariant. The converse is immediate. 2. The direct sum reduces T if and only if both M and N are invariant subspaces. By part 1 this is equivalent to the two conditions T P = PT P and T (I − P) = (I − P)T (I − P). These two conditions are equivalent to (4.49).  We clearly have, for a projection P, that I = P+ (I − P), which corresponds to the direct sum decomposition X = Im P ⊕ Im (I − P). This generalizes in the following way. Theorem 4.50. Given the direct sum decomposition X = M1 ⊕ · · · ⊕ Mk , there exist k projections Pi on X such that 1. Pi Pj = δi j Pj . 2. I = P1 + · · · + Pk . 3. Im Pi = Mi , i = 1, . . . , k. Conversely, given k operators Pi satisfying (1)-(3), then with Mi = Im Pi , we have X = M 1 ⊕ · · · ⊕ Mk . Proof. Assume we are given the direct sum decomposition X = M1 ⊕ · · · ⊕ Mk . Thus, each x ∈ X has a unique representation in the form x = m1 + · · · + mk with mi ∈ Mi . We define Pi x = mi . The operators Pi are clearly linear, projections with Im Pi = Mi , and satisfy (1)-(3). Moreover, we have KerPi = ∑ j=i M j . Conversely, assume Pi satisfy conditions (1)-(3), and let Mi = Im Pi . From I = P1 + · · · + Pk it follows that x = P1 x + · · · + Pk x and hence X = M1 + · · · + Mk . This representation of x is unique, for if x = m1 + · · · + mk , with mi ∈ Mi another such representation, then since m j ∈ M j , we have m j = Pj y j for some y j . Therefore, k

Pi x = Pi ∑ m j = j=1

k

k

k

j=1

j=1

j=1

∑ Pi m j = ∑ PiPj y j = ∑ δi j Pj y j = Pi yi = mi ,

which shows that the sum is a direct sum.



In search for nontrivial invariant subspaces we begin by looking for those that are 1-dimensional. If M is a 1-dimensional invariant subspace and x a nonzero vector in M , then every other vector in M is of the form α x. Thus the invariance condition is T x = α x, and this leads us to the following definition.

4.6 Polynomial Module Structure on Vector Spaces

93

Definition 4.51. 1. Let T be a linear transformation in a vector space U over the field F. A nonzero vector x ∈ U will be called an eigenvector, or characteristic vector, of T if there exists α ∈ F such that T x = α x. Such an α will be called an eigenvalue, or characteristic value, of T . 2. The characteristic polynomial of T , dT (z), is defined by dT (z) = det(zI − T ). Clearly, deg dT = dim U . Proposition 4.52. Let T be a linear transformation in a finite-dimensional vector space U over the field F. An element α ∈ F is an eigenvalue of T if and only if it is a zero of the characteristic polynomial of T . Proof. The homogeneous system (α I − T )x = 0 has a nontrivial solution if and only if α I − T is singular, i.e., if and only if dT (α ) = det(α I − T ) = 0.  Let T be a linear transformation in an n-dimensional vector space U . We say that a polynomial p(z) ∈ F[z] annihilates T if p(T ) = 0, where p(T ) is defined by (4.40). Clearly, every linear transformation is annihilated by the zero polynomial. A priori, it is not clear that an arbitrary linear transformation has a nontrivial annihilator. This is proved next. Proposition 4.53. For every linear transformation T in an n-dimensional vector space U there exists a nontrivial annihilating polynomial. Proof. L(U ), the space of all linear transformations in U , is n2 -dimensional. 2 Therefore the set of linear transformations {I, T, . . . , T n } is linearly dependent. Hence, there exist coefficients pi ∈ F, i = 0, . . . , n2 , not all zero, for which 2 2  ∑ni=0 pi T i = 0, or p(T ) = 0, where p(z) = ∑ni=0 pi zi . Theorem 4.54. Let T be a linear transformation in an n-dimensional vector space U . There exists a unique monic polynomial of minimal degree that annihilates T . Proof. Let J = {p(z) ∈ F[z]| p(T ) = 0}. Clearly, J is a nontrivial ideal in F[z]. Since F[z] is a principal ideal domain, there exists a unique monic polynomial mT (z) for which J = mT F[z]. Obviously if 0 = p(z) ∈ J, then deg p ≥ deg m.  The polynomial m(z) whose existence is established in the previous theorem is called the minimal polynomial of T . Both the characteristic and the minimal polynomial are similarity invariants.

94

4 Linear Transformations

Proposition 4.55. Let T1 and T2 be similar linear transformations. Let dT1 (z), dT2 (z) be their characteristic polynomials and mT1 (z), mT2 (z) their minimal polynomials. Then dT1 (z) = dT2 (z) and mT1 (z) = mT2 (z). Proof. If T2  T1 , then for some invertible R, zI2 − T2 = R(zI1 − T1 )R−1 . Using the multiplication rule of determinants, it follows that dT2 (z) = det(zI2 − T2 ) = det(zI1 − T1 ) = dT1 (z). Note that T2 = RT1 R−1 implies that for any p(z) ∈ F[z], we have p(T2 ) = Rp(T1 )R−1 . In particular, we have mT1 (T2 ) = RmT1 (T1 )R−1 = 0, which shows that mT2 (z) | mT1 (z). By symmetry, we have mT1 (z) | mT2 (z), and the equality of the minimal polynomials follows.  Since the characteristic polynomial of a linear transformation T is a similarity invariant, all coefficients of dT (z) are similarity invariants too. We single out two. If dT (z) = zn +tn−1 zn−1 + · · ·+t0 , then it is easily checked that with Trace T defined by (4.26), we have tn−1 = −TraceT = − ∑ni=1 tii , for any matrix representation of T . In the same way, we check that t0 = (−1)n det T , so det T is also a similarity invariant.

4.7 Exercises 1. Let T be an m × n matrix over the field F. Define the determinant rank of T to be the order of the largest nonvanishing minor of T . Show that the rank of T is equal to its determinantal rank. 2. Let M be a subspace of a finite-dimensional vector space V . Show that M ∗  V ∗ /M ⊥ . 3. Let X be an n-dimensional complex vector space. Let T : X −→ X be such that for every x ∈ X , the vectors x, T x, . . . , T m x are linearly dependent. Show that I, T, . . . , T m are linearly dependent. 4. Let A, B,C be linear transformations. Show that there exists a linear transformation Z such that C = AZB if and only if ImC ⊂ Im A, KerC ⊃ Ker B. 5. Let X , Y be finite-dimensional vector spaces, A ∈ L(X ), and W, Z ∈ L(X , Y ). Show that the following statements are equivalent: a. We have AKerW ⊂ Ker Z. b. We have KerW ⊂ Ker ZA. c. There exists a map B ∈ L(Y ) for which ZA = BW . 6. Let V be a finite-dimensional vector space and let Ai ∈ L(V ), i = 1, . . . , s. Show that if M = ∑si=1 Im Ai (M = ∩si=1 Ker Ai ), then there exist Bi ∈ L(V ) such that Im ∑si=1 Ai Bi = M (Ker ∑si=1 Bi Ai = M ).

4.8 Notes and Remarks

95

7. Let V be a finite-dimensional vector space. Show that there is a bijective correspondence between left (right) ideals in L(V ) and subspaces of V . The correspondence is given by J ↔ ∩A∈J Ker A (J ↔ ∑A∈J Im A). 8. Show that Ker A2 ⊃ Ker A. Show also that Ker A2 = Ker A implies KerA p = Ker A for all p > 0. 9. Let T be an injective linear transformation on a (not necessarily finitedimensional) vector space V . Show that if for some integer k, we have T k = T , then T is also surjective. 10. Let A be an n × n complex matrix with eigenvalues λ1 , . . . , λn . Show that det A = ∏ni=1 λi . Show that given a polynomial p(z), the eigenvalues of p(A) are p(λ1 ), . . . , p(λn ) (Spectral mapping theorem). 11. Let A be an n × n complex matrix with eigenvalues λ1 , . . . , λn . Show that the eigenvalues of adj A are ∏ j=1 λ j , . . . , ∏ j=n λ j . 12. Let A be invertible and let dA (z) be its characteristic polynomial. Show that the characteristic polynomial of A−1 is dA−1 (z) = d(0)−1 zn dA (z−1 ). 13. Let the minimal polynomial of alineartransformation A be ∏(z − λ j )ν j . Show that the minimal polynomial of

A I 0 A

is ∏(z − λ j )ν j +1 .

14. Let A be a linear transformation in a finite-dimensional vector space V over the field F. Let mA (z) be its minimal polynomial. Prove that given a polynomial p(z), p(A) is invertible if and only if p(z) and mA (z) are coprime. Show that the minimal polynomial can be replaced by the characteristic polynomial and the result still holds.

4.8 Notes and Remarks An excellent source for linear algebra is Hoffman and Kunze (1961). The classic treatise of Gantmacher (1959), though strongly matrix oriented, is still a rich source for results and ideas and is highly recommended as a general reference. So is Malcev (1963), which is close in spirit to the present book. We already mentioned, in Section 2.14, the contributions of Grassmann and Peano to the study of linear transformations. The conceptual rigor of this part of linear algebra owes much to Weierstrass and his students Kronecker and Frobenius.

Chapter 5

The Shift Operator

5.1 Introduction We now turn our attention to the study of a special class of transformations, namely shift operators. These will turn out later to serve as models for all linear transformations, in the sense that every linear transformation is similar to a shift operator.

5.2 Basic Properties We introduce now an extremely important class of linear transformation that will play a central role in the analysis of the structure of linear transformations. Recall that for a nonzero polynomial q(z), we denote by πq f the remainder of the polynomial f (z) after division by q(z). Clearly, πq is a projection operator in F[z]. We note that given a nonzero polynomial q(z), any f (z) ∈ F[z] has a unique representation of the form f (z) = a(z)q(z) + r(z),

(5.1)

with deg r < deg q. The remainder, r = πq f , can be written in another form based on the direct sum representation (2.7), namely F((z−1 )) = F[z] ⊕ z−1 F[[z−1 ]]. Let π+ , π− be defined by (1.23), i.e., the projections of F((z−1 )) on F[z] and z−1 F[[z−1 ]] respectively, which correspond to the above direct sum decomposition. From the representation (5.1) of f (z), we have q(z)−1 f (z) = a(z) + q(z)−1 r(z). Applying the projection π− , we have π− q−1 f = π− q−1 r = q−1 r, which implies an important alternative representation for the remainder, namely

πq f = qπ− q−1 f .

P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 5, © Springer Science+Business Media, LLC 2012

(5.2)

97

98

5 The Shift Operator

The importance of equation (5.2) stems from the fact that it easily extends to the case of polynomial vectors. Proposition 5.1. Let q(z) be a monic polynomial in F[z] and let πq : F[z] −→ F[z] be the projection map defined in (5.2). Then we have Ker πq = qF[z]

(5.3)

F[z] = Xq ⊕ qF[z].

(5.4)

Xq = Im πq = {πq f | f (z) ∈ F[z]},

(5.5)

and the direct sum Defining the set Xq by

we have the isomorphism Xq  F[z]/qF[z].

(5.6)

Proof. Clearly, for each nonzero q(z) ∈ F[z], the map πq is a projection map. Equation (5.3) follows by a simple computation. Since πq is a projection, so is I − πq. The identity I = πq + (I − πq ), taken together with (5.3), implies the direct sum (5.4). Finally, the isomorphism (5.6) follows from (5.5) and (5.3).  The isomorphism (5.6) allows us to lift the F[z]-module structure to Xq . This module structure is the one induced by the polynomial z. Definition 5.2. Let q(z) be a monic polynomial in F[z]. We define a linear transformation Sq : Xq −→ Xq by Sq f = z · f = π q z f .

(5.7)

We call Sq the shift operator in Xq . We note that for the shift operator Sq , we have for all k ≥ 0, that Sqk f = πq zk f . This implies that for any p(z) ∈ F[z], we have p · f = p(Sq ) f = πq (p f ),

f (z) ∈ Xq .

(5.8)

We refer to Xq , with the F[z]-module structure induced by Sq , as a polynomial model. The next proposition characterizes elements of a polynomial model and studies some important bases for it. Proposition 5.3. Let q(z) = zn + qn−1 zn−1 + · · · + q0 . Then 1. A polynomial f (z) belongs to Xq if and only if q(z)−1 f (z) is strictly proper. 2. We have dim Xq = deg q = n. 3. The following sets are bases for Xq : a. The standard basis, namely Bst = {1, z, . . . , zn−1 }.

5.2 Basic Properties

99

b. The control basis, namely Bco = {e1 (z), . . . , en (z)}, where ei (z) = zn−i + qn−1zn−i−1 + · · · + qi . c. In case α1 , . . . , αn , the zeros of q(z), are distinct then the polynomials pi (z) = Π j=i (z − α j ), i = 1, . . . , n form a basis for Xq . We refer to this as the spectral basis Bsp of Xq . d. Under the same assumption, the Lagrange interpolation polynomials, given by li (z) = ppii((z) αi ) are a basis for Xq , naturally called the interpolation basis. Proof. 1. Clearly, f (z) ∈ Xq is equivalent to πq f = f . In turn, this can be rewritten as q−1 f = π− q−1 f , and this equality holds if and only if q−1 f is strictly proper. 2. Clearly, the elements of Xq are all polynomials of degree < n = deg q. Obviously, this is an n-dimensional space. 3. Each of the sets has n linearly independent elements, hence is a basis for Xq . The linear independence of the Lagrange interpolation polynomials was proved in Chapter 2, and the polynomials pi (z) are, up to a multiplicative constant, equal to the Lagrange interpolation polynomials.  We proceed by studying the matrix representations of Sq with respect to these bases of Xq . Proposition 5.4. Let Sq : Xq −→ Xq be defined by (5.7). 1. With respect to the standard basis, Sq has the matrix representation ⎛

0 ⎜1 ⎜ ⎜ Cq = [Sq ]stst = ⎜ . ⎜ ⎝ .

⎞ −q0 · ⎟ ⎟ ⎟ · ⎟. ⎟ · ⎠ 1 −qn−1

(5.9)

2. With respect to the control basis, Sq has the diagonal matrix representation ⎛ Cq

=

[Sq ]co co

⎜ ⎜ ⎜ =⎜ ⎜ ⎝

⎞

0 1 .

· .

1 −q0 . . . −qn−1

⎟ ⎟ ⎟ ⎟. ⎟ ⎠

3. With respect to the spectral basis, Sq has the matrix representation

(5.10)

100

5 The Shift Operator

⎛ [Sq ]sp sp

⎜ ⎜ ⎜ =⎜ ⎜ ⎝

⎞

α1 . . .

αn

⎟ ⎟ ⎟ ⎟. ⎟ ⎠

(5.11)

Proof. 1. Clearly, we have Sq zi =

⎧ ⎨ zi+1 ,

i = 0, . . . , n − 2,

⎩−

i = n − 1.

i ∑n−1 i=0 qi z ,

2. We compute, defining e0 (z) = 0, Sq ei = πq zei (z) = πq z(zn−i + qn−1zn−i−1 + · · · + qi ) = πq (zn−i+1 + qn−1zn−i + · · · + qi z) = πq (zn−i+1 + qn−1zn−i + · · · + qi z + qi−1) − qi−1en (z). So we get Sq ei = ei−1 − qi−1 en .

(5.12)

3. Noting that q(z) = (z − αi )pi (z), we compute Sq pi (z) = πq (z − αi + αi )pi = πq (q + αi pi ) = αi pi .



matrices Cq ,Cq

The are called the companion matrices of the polynomial q(z). We note that the change of basis transformation from the control to the standard basis has a particularly nice form. In fact, we have ⎛

q1 . . qn−1 ⎜ . . ⎜ ⎜ [I]stco = ⎜ . . ⎜ ⎝ qn−1 . 1

1

⎞ ⎟ ⎟ ⎟ ⎟. ⎟ ⎠

We relate now the invariant subspaces of the shift operators Sq , or equivalently the submodules of Xq , to factorizations of the polynomial q(z). This is an example of the interplay between algebra and geometry that is one of the salient characteristics of the use of functional models. Theorem 5.5. Given a monic polynomial q(z), M ⊂ Xq is an Sq -invariant subspace if and only if M = q1 Xq2 , for some factorization q(z) = q1 (z)q2 (z).

5.2 Basic Properties

101

Proof. Assume q(z) = q1 (z)q2 (z) and M = q1 Xq2 . Thus f (z) ∈ M implies f (z) = q1 (z) f1 (z) with deg f1 < deg q2 . Using Lemma 1.22, we compute Sq f = πq z f = πq1 q2 zq1 f1 = q1 πq2 z f1 = q1 Sq2 f1 ∈ M. Conversely, let M be an Sq -invariant subspace. Now, for each f (z) ∈ Xq there exists a scalar α that depends on f (z) for which Sq f = z f − α q.

(5.13)

Consider now the set N = M + qF[z]. Obviously N is closed under addition, and using (5.13), z{M + qF[z]} ⊂ {M + qF[z]}. Thus N is an ideal in F[z], hence of the form q1 F[z]. Since, obviously, qF[z] ⊂ q1 F[z], it follows from Proposition 1.46 that q1 (z) is a divisor of q(z), that is, we have a factorization q(z) = q1 (z)q2 (z). It is clear that M = πq {M + qF[z]} = πq q1 F[z] = πq1 q2 q1 F[z] = q1 Xq2 .



Proper subspaces of a vector space may have many complementary subspaces and invariant subspaces of polynomial models are no exception. The following proposition exhibits a particular complementary subspace to such a proper invariant subspace. Proposition 5.6. Let q(z) ∈ F[z] be nonzero and let q(z) = q1 (z)q2 (z) be a factorization. Then, as vector spaces, we have the following direct sum representation: Xq = Xq1 ⊕ q1 Xq2 .

(5.14)

Proof. That Xq1 ⊂ Xq follows from the fact that deg q1 ≤ deg q. Next, we note that Xq1 ∩ q1 Xq2 = {0} and every nonzero polynomial in Xq1 has degree < degq1 , whereas a nonzero polynomial in q1 Xq2 has degree ≥ deg q1 . Finally, the equality in (5.14) follows from dim Xq = degq = deg q1 + degq2 = dim Xq1 + dim Xq2 = dim Xq1 ⊕ q1 Xq2 .  Note that Xq1 ⊂ Xq is generally not an invariant subspace for Sq . The following proposition sums up the basic arithmetic properties of invariant subspaces of the shift operator. This can be viewed as the counterpart of Proposition 1.46. Proposition 5.7. Given a monic polynomial q(z) ∈ F[z], the following hold: 1. Let q(z) = q1 (z)q2 (z) = p1 (z)p2 (z) be two factorizations. Then we have the inclusion q1 Xq2 ⊂ p1 X p2 (5.15) if and only if p1 (z) | q1 (z), or equivalently q2 (z) | p2 (z).

102

5 The Shift Operator

2. Given factorizations q(z) = pi (z)qi (z), i = 1, . . . , s, then ∩si=1 pi Xqi = pXq with p(z) the l.c.m of the pi (z) and q(z) the g.c.d. of the qi (z). 3. Given factorizations q(z) = pi (z)qi (z), i = 1, . . . , s, then ∑si=1 pi Xqi = pXq with q(z) the l.c.m of the qi (z) and p(z) the g.c.d. of the pi (z). Proof. 1. Assume p1 (z)|q1 (z), i.e., q1 (z) = p1 (z)r(z) for some polynomial r(z). Then q(z) = q1 (z)q2 (z) = (p1 (z)r(z))q2 (z) = p1 (z)(r(z)q2 (z)) = p1 (z)p2 (z), and in particular, p2 (z) = r(z)q2 (z). This implies q1 Xq2 = p1 rXq2 ⊂ p1 Xrq2 = p1 X p2 . Conversely, assume the inclusion (5.15) holds. From this we have q1 Xq2 + qF[z] = q1 Xq2 + q1 q2 F[z] = q1 [Xq2 + q2 F[z]] = q1 F[z]. So (5.15) implies the inclusion q1 F[z] ⊂ p1 F[z]. By Proposition 1.46 it follows that p1 (z)|q1 (z). 2. Let t(z) = pi (z)qi (z), i = 1, . . . , s. Since ∩si=1 pi Xqi is a submodule of Xq , it is of the form pXq with t(z) = p(z)q(z). Now the inclusion pXq ⊂ pi Xqi implies pi (z)|p(z) and q(z)|qi (z), so p(z) is a common multiple of the pi (z), and q(z) a common divisor of the qi (z). Let now q (z) be any common divisor of the qi (z). Since necessarily q (z) | t(z), we can write t(z) = p (z)q (z). Now applying Proposition 1.46, we have p Xq ⊂ pi Xqi and hence p Xq ⊂ ∩si=1 pi Xqi = pXq . This implies q (z) | q(z), and hence q(z) is a g.c.d. of the qi (z). By the same token, we conclude that p(z) is the l.c.m. of the pi (z). 3. Since p1 Xq1 + · · · + ps Xqs is an invariant subspace of Xt it is of the form pXq with t(z) = p(z)q(z). Now the inclusions pi Xqi ⊂ pXq imply the division relations qi (z) | q(z) and p(z) | pi (z). So q(z) is a common multiple of the qi (z), and p(z) a common divisor of the pi (z). Let p (z) be any other common divisor of the pi (z). Then pi (z) = p (z)ei (z) for some polynomials ei (z). Now t(z) = pi (z)qi (z) = p (z)ei (z)qi (z) = p (z)q (z), so ei (z)qi (z) = q (z), and q (z) is a common multiple of the qi (z). Now pi (z) = p (z)ei (z) implies pi Xqi = p ei Xqi ⊂ p Xei qi = p Xq and hence pXq = p1 Xq1 + · · ·+ ps Xqs = p Xq . This shows that q(z) | q (z), and so q(z) is the l.c.m. of the qi (z).  Corollary 5.8. Given the factorizations q(z) = pi (z)qi (z), i = 1, . . . , s, then 1. Xq = p1 Xq1 + · · · + ps Xqs if and only if the pi (z) are coprime. 2. The sum p1 Xq1 + · · · + ps Xqs is a direct sum if and only if q1 (z), . . . , qs (z) are mutually coprime. 3. We have the direct sum decomposition Xq = p1 Xq1 ⊕ · · · ⊕ ps Xqs if and only if the pi (z) are coprime and the qi (z) are mutually coprime.

5.2 Basic Properties

103

4. We have the direct sum decomposition Xq = p1 Xq1 ⊕ · · · ⊕ ps Xqs if and only if the qi (z) are mutually coprime and q(z) = q1 (z) · · · qs (z). In this case pi (z) = Π j=i q j (z). Proof. 1. Let the invariant subspace p1 Xq1 + · · · + ps Xqs have the representation pν Xqν with pν (z) the g.c.d. of the pi (z) and qν (z) the l.c.m. of the qi (z). Therefore pν Xqν = Xq if and only if pν (z) = 1 or equivalently qν (z) = q(z). 2. The sum p1 Xq1 + · · · + ps Xqs is a direct sum if and only if for each index i, we have pi Xqi ∩ ∑ p j Xq j = {0}. j=i

Now ∑ j=i p j Xq j is an invariant subspace and hence of the form πi Xσi for some factorization q(z) = πi (z)σi (z). Here πi (z) is the g.c.d. of the p j (z), j = i and σi (z) is the l.c.m. of the q j (z), j = i. Now pi Xqi ∩ πi Xσi = {0} if and only if σi (z) and qi (z) are coprime. This, however, is equivalent to qi (z) being coprime with each of the q j (z), j = i, i.e., to the mutual coprimeness of the qi (z), i = 1, . . . , s. 3. Follows from the previous two parts. 4. Clearly, the pi (z) are coprime.  Corollary 5.9. Let p(z) = p1 (z)ν1 · · · pk (z)νk be the primary decomposition of the polynomial p(z). Define πi (z) = Π j=i p j (z)ν j , for i = 1, . . . , k. Then X p = π1 X pν1 ⊕ · · · ⊕ πk X pνk . 1

(5.16)

k

Proof. Clearly, the g.c.d. of the πi (z) is 1, whereas the l.c.m. of the pνi i (z) is p(z).  The structure of the shift operator restricted to an invariant subspace can be easily deduced from the corresponding factorization. Proposition 5.10. Let q(z) = q1 (z)q2 (z). Then we have the similarity Sq |q1 Xq2  Sq2 .

(5.17)

Proof. Let φ : Xq2 −→ q1 Xq2 be the map defined by

φ ( f ) = q1 f , which is clearly an isomorphism of the two spaces. Next we compute, for f (z) ∈ Xq2 ,

φ Sq2 f = q1 Sq2 f = qπq q1 z f = πq zq1 f = Sq φ f .

104

5 The Shift Operator

Therefore, the following diagram is commutative: Xq2

φ

-

q1 Xq2 Sq |q1 Xq2

S q2 ? Xq2

φ

-

? q1 Xq2



This is equivalent to (5.17).

Since eigenvectors span 1-dimensional invariant subspaces, we expect a characterization of eigenvectors of the shift Sq in terms of the polynomial q(z). Proposition 5.11. Let q(z) be a nonzero polynomial. 1. The eigenvalues of Sq coincide with the zeros of q(z). 2. f (z) ∈ Xq is an eigenvector of Sq corresponding to the eigenvalue α if and only if it has the representation cq(z) f (z) = . (5.18) z−α Proof. Let f (z) be an eigenvector of Sq corresponding to the eigenvalue α , i.e., Sq f = α f . By (5.13), there exists a scalar c for which (Sq f )(z) = z f (z) − cq(z). Thus z f (z) − cq(z) = α f (z), which implies (5.18). Since f (z) is a polynomial, we must have q(α ) = 0. Conversely, if q(α ) = 0, q(z) is divisible by z − α and hence f (z), defined by (5.18), is in Xq . We compute (Sq − α I) f = πq (z − α )

cq(z) = πq cq(z) = 0, z−α

which shows that α is an eigenvalue of Sq , and a corresponding eigenvector is given by (5.18).  The previous proposition suggests that the characteristic polynomial of Sq is q(z) itself. Indeed, this is true and is proved next. Proposition 5.12. Let q(z) = zn + qn−1 zn−1 + · · · + q0 be a monic polynomial of degree n, and let Sq be the shift operator defined by (5.7). Then the characteristic polynomial of Sq is q(z). Proof. It suffices to compute det(zI − C) for an arbitrary matrix representation of Sq . We find it convenient to do the computation in the standard basis. In this case the matrix representation is given by the companion matrix Cq of (5.9). We prove the result by induction on n. For n = 1, we have C = (−q0 ) and det(zI −C) = z + q0 .

5.2 Basic Properties

105

Assume the statement holds up to n − 1. We proceed to compute, using the induction hypothesis and expanding the determinant by the first row,    z q0     −1 ·      det(zI − Cq ) =  . ·      . ·    −1 z + qn−1        z  −1 z q1       −1   .. ·         n+1 = z .. . ·   + (−1) q0          . z . ·        −1  −1 z + qn−1 = z(zn−1 + qn−1zn−2 + · · · + q1 ) + (−1)n+1 q0 (−1)n−1 = q(z).



We introduce now the class of cyclic transformations. Their importance lies in that they turn out to be the building blocks of general linear transformations. Definition 5.13. Let U be an n-dimensional vector space over the field F. A map A : U −→ U is called a cyclic transformation if there exists a vector b ∈ U for which {b, Ab, . . ., An−1 b} is a basis for U . Such a vector b will be called a cyclic vector for A. Lemma 5.14. 1. Given a nonzero polynomial q(z) and f (z) ∈ Xq , the smallest Sq invariant invariant subspace of Xq containing f (z) is q1 Xq2 , where q1 (z) = q(z)∧ f (z) and q(z) = q1 (z)q2 (z). 2. Sq is a cyclic transformation in Xq . 3. A polynomial f (z) ∈ Xq is a cyclic vector of Sq if and only if f (z) and q(z) are coprime. Proof. 1. Let M be the subspace of Xq spanned by the vectors {Sqi f |i ≥ 0}. This is the smallest Sq invariant subspace containing f (z). Therefore it has the representation M = q1 Xq2 for a factorization q(z) = q1 (z)q2 (z). Since f (z) ∈ M, there exists a polynomial f 1 (z) ∈ Xq2 for which f (z) = q1 (z) f1 (z). This shows that q1 (z) is a common divisor of q(z) and f (z). To show that it is the greatest common divisor, let us assume that q (z) is an arbitrary common divisor of q(z) and f (z). Thus we have q(z) = q (z)q (z) and f (z) = q (z) f  (z). Using Lemma 1.24, we compute Sqk f = πq xk f = πq xk q f  = q πq xk f  = q Sqk  f  .

106

5 The Shift Operator

Thus we have M ⊂ q Xq , or equivalently q1 Xq2 ⊂ q Xq . This implies q (z)|q1 (z); hence q1 (z) is the g.c.d. of q(z) and f (z). 2. Obviously 1 ∈ Xq and 1 ∧ q(z) = 1. So 1 is a cyclic vector for Sq . 3. Obviously, since dim q1 Xq2 = dim Xq2 = deg q2 , X = q1 Xq2 if and only if deg q1 = 0, that is, f (z) and q(z) are coprime.  The availability of eigenvectors of the shift allows us to study under what conditions the shift Sq is diagonalizable, i.e., has a diagonal matrix representation. Proposition 5.15. Let q(z) be a monic polynomial of degree n. Then Sq is diagonalizable if and only if q(z) splits into the product of n distinct linear factors, or equivalently, it has n distinct zeros. n (z − α ). Proof. Assume α1 , . . . , αn are the distinct zeros of q(z), i.e., q(z) = Πi=1 i q(z) Let pi (z) = z−αi = Π j=i (z − α j ). Then Bsp = {p1 , . . . , pn } is the spectral basis for Xq , differing from the Lagrange interpolation basis by constant factors only. It is easily checked that (Sq − αi )pi = 0. So

⎛ [Sq ]sp sp

⎜ ⎜ ⎜ =⎜ ⎜ ⎝

⎞

α1 . . .

αn

⎟ ⎟ ⎟ ⎟, ⎟ ⎠

(5.19)

and Sq is diagonalizable. Conversely, assume Sq is diagonalizable. Then with respect to some basis it has the representation (5.19). Since Sq is cyclic, its minimal and characteristic polynomials coincide. Necessarily all the αi are distinct.  Proposition 5.16. Let q(z) be a monic polynomial and Sq the shift operator in Xq defined by (5.7). Then p(Sq ) f = πq (p f ),

f (z) ∈ Xq .

(5.20)

Proof. Using linearity, it suffices to show that Sqk f = πq zk f ,

f (z) ∈ Xq .

We prove this by induction. For k = 1 this is the definition. Assume we proved it up to an integer k. Now, using the fact that zKer πq ⊂ Ker πq , we compute Sqk+1 f = Sq Sqk f = πq zπq zk f = πq zk+1 f .



Clearly, the operators p(Sq ) all commute with the shift Sq . We proceed to state the simplest version of the commutant lifting theorem. It characterizes operators

5.2 Basic Properties

107

commuting with the shift Sq in Xq via operators commuting with the shift S+ in F[z]. The last class of operators consists of multiplication operators by polynomials. Theorem 5.17. 1. Let q(z) be a monic polynomial and Sq the shift operator in Xq defined by (5.7). Let Z be any operator in Xq that commutes with Sq . Then there exists an operator Z that commutes with S+ and such that Z = πq Z|Xq .

(5.21)

Equivalently, the following diagram is commutative:

F[z]

Z

-

πq

πq ?

Xq

F[z]

Z

-

? Xq

2. An operator Z in Xq commutes with Sq if and only if there exists a polynomial p(z) ∈ F[z] for which Z f = πq p f = p(Sq ) f ,

f (z) ∈ Xq .

(5.22)

Proof. 1. By Proposition 6.2, there exists a polynomial p(z) for which Z = p(Sq ). We define Z : F[z] −→ F[z] by Z f = p f . It is easily checked that (5.21) holds. 2. If Z is represented by (5.22), then Sq Z f = πq zπq p f = πq zp f = πq pz f = πq pπq z f = ZSq f . Conversely, if Z commutes with the shift, it has a representation (5.21). Now, any map Z commuting with S+ is a multiplication by a polynomial p(z), which proves (5.22).  The proof of the previous theorem is slightly misleading, inasmuch as it uses the fact that Sq is cyclic. In Chapter 8, we will return to this subject using tensor products and the Bezout map. Since any operator of the form p(Sq ) is completely determined by the two polynomials p(z) and q(z), all its properties should be derivable from these data. The next proposition focuses on the invertibility properties of p(Sq ). Theorem 5.18. Given the polynomials p(z), q(z), with q(z) monic, let r(z) and s(z) be the g.c.d and the l.c.m. of p(z) and q(z) respectively. Then

108

5 The Shift Operator

1. We have the factorizations q(z) = r(z)q1 (z), p(z) = r(z)p1 (z),

(5.23)

with p1 (z), q1 (z) coprime, as well as the factorizations s(z) = r(z)p1 (z)q1 (z) = p(z)q1 (z) = q(z)p1 (z).

(5.24)

Moreover, we have Ker p(Sq ) = q1 Xr , Im p(Sq ) = rXq1 .

(5.25)

2. Let p(z), q(z) ∈ F[z], with q(z) nonzero. Then the linear transformation p(Sq ) is invertible if and only if p(z) and q(z) are coprime. Moreover, we have p(Sq )−1 = a(Sq ),

(5.26)

where the polynomial a(z) arises out of any solution of the Bezout equation a(z)p(z) + b(z)q(z) = 1.

(5.27)

Proof. 1. With r(z) and s(z) defined as above, (5.24) is an immediate consequence of (5.23). Applying Theorem 5.5, it follows that q1 Xr and rXq1 are invariant subspaces of Xq . If f (z) ∈ q1 Xr ,, then f = q1 g, with g ∈ Xr . We compute, using (5.23), p(Sq ) f = πq p f = q1 rπ− r−1 q−1 1 rp1 q1 g = q1 r π− g = 0, which shows that q1 Xr ⊂ Ker p(Sq ). Conversely, assume f (z) ∈ Ker p(Sq ). Then πq p f = 0 or there exists g(z) such that p(z) f (z) = q(z)g(z), which implies p1 (z) f (z) = q1 (z)g(z). Since p1 (z) and q1 (z) are coprime, we have f (z) = q1 (z) f1 (z) for some polynomial f1 (z). As f (z) ∈ Xq , we must have f1 (z) ∈ Xr . So Ker p(Sq ) ⊂ q1 Xr , and the first equality in (5.25) follows. Next, assume g(z) ∈ Im p(Sq ), i.e., there exists an f (z) ∈ Xq such that g = πq p f . We compute, using (5.23) and (5.24), g = πq p f = q1 rπ− r−1 q−1 1 rp1 f = rπq1 p1 f ∈ rXq1 , i.e., we have the inclusion Im p(Sq ) ⊂ rXq1 . Conversely, assume g(z) ∈ rXq1 , i.e., g(z) = r(z)g1 (z) with g1 (z) ∈ Xq1 . By the coprimeness of p1 (z) and q1 (z), the map f1 → πq1 p1 f1 , acting in Xq1 , is an invertible map. Hence, there exists f1 (z) ∈ Xq1 for which g1 = πq1 p1 f1 . This implies

5.3 Circulant Matrices

109

rg1 = rq1 π− r−1 q−1 1 p1 r f1 = πq pr f 1 . This shows that rXq1 ⊂ Im p(Sq ), and the second equality in (5.25) follows. 2. From the characterization (5.25) of Ker p(Sq ) and Im p(Sq ), it follows that the injectivity of p(Sq ) is equivalent to the coprimeness of p(z) and q(z), and the same holds for surjectivity. In order to actually invert p(Sq ), we use the coprimeness of p(z) and q(z). This implies the existence of polynomials a(z), b(z) that solve the Bezout equation a(z)p(z) + b(z)q(z) = 1. Applying the functional calculus to the shift Sq , and noting that q(Sq ) = 0, we get a(Sq )p(Sq ) + b(Sq )q(Sq ) = a(Sq )p(Sq ) = I, 

and (5.26) follows.

5.3 Circulant Matrices In this section we give a short account of a special class of structured matrices, called circulant matrices. We do this for its own sake, and as an illustration of the power of polynomial algebra. Definition 5.19. An n × n matrix over a field F is called a circulant matrix, or simply circulant for short, if it has the form ⎛

c0 cn−1 ⎜ c ⎜ 1 c0 ⎜ C = circ (c0 , . . . , cn−1 ) = ⎜ . . ⎜ ⎝ . . cn−1 .

. . . . .

⎞ . c1 . . ⎟ ⎟ ⎟ . . ⎟, ⎟ . cn−1 ⎠ c1 c0

i.e., ci j = c(i− jmod n) . We define a polynomial c(z) by c(z) = c0 + c1 z + · · · + cn−1 zn−1 . The polynomial c is called the representer of circ (c0 , . . . , cn−1 ). Theorem 5.20. For circulant matrices the following properties hold: 1. The circulant matrix circ (c0 , . . . , cn−1 ) is the matrix representation of c(Szn −1 ) with respect to the standard basis of Xzn −1 . 2. The sum of circulant matrices is a circulant matrix. Specifically, circ(a1 , . . . , an ) + circ(b1 , . . . , bn ) = circ(a1 + b1 , . . . , an + bn ). 3. We have

α circ (a1 , . . . , an ) = circ (α a1 , . . . , α an ).

110

5 The Shift Operator

4. Define the special circulant matrix Π by ⎛

0 ⎜1 . . ⎜ ⎜ Π = circ (0, 1, 0, . . . , 0) = ⎜ . . ⎜ ⎝ . 1

5. 6. 7.

8.

⎞ 1 .⎟ ⎟ ⎟ . ⎟. ⎟ .⎠ 0

Then C ∈ Fn×n is a circulant if and only if CΠ = Π C. The product of circulants is commutative. The product of circulants is a circulant. The inverse of a circulant is a circulant. Moreover, the inverse of a circulant with representer c(z) is a circulant with representer a(z), where a(z) comes from a solution of the Bezout equation a(z)c(z) + b(z)(zn − 1) = 1. Over an algebraically closed field, circulants are diagonalizable.

Proof. 1. We compute

⎛

⎞ 0.. . 1 ⎜1 ·⎟ ⎜ ⎟ ⎜ ⎟ st [Szn −1 ]st = ⎜ . · ⎟. ⎜ ⎟ ⎝ . ·⎠ 10

This is a special case of the companion matrix in (5.9). Now Szi n −1 z j

=

zi+ j , zi+ j−n ,

So c(Szn −1 )z j =

i + j ≤ n − 1, i + j ≥ n.

n−1

n−1− j

i=0

i=0

∑ ci Szi n−1 z j =

∑

ci zi+ j +

n−1

∑

ci zi+ j−n ,

i=n− j

and this implies the equality [c(Szn −1 )]stst = circ (c0 , . . . , cn−1 ). 2. Given polynomials a(z), b(z) ∈ F[z], we have (a + b)(Szn−1 ) = a(Szn −1 ) + b(Szn−1 ), and hence circ (a0 + b0 , . . . , an−1 + bn−1 ) = [(a + b)(Szn−1 )]stst = [a(Szn −1 )]stst + [b(Szn−1 )]stst = circ (a0 , . . . , an−1 ) + circ(b0 , . . . , bn−1 ).

5.4 Rational Models

111

3. We compute circ(α a0 , . . . , α an−1 ) = [α a(Szn −1 )]stst = α [a(Szn −1 )]stst = α circ (a0 , . . . , an−1 ). 4. Clearly, Π = circ (0, 1, 0, . . . , 0) = [Szn −1 ]stst . Obviously, Szn −1 is cyclic. Hence, a linear transformation K commutes with Szn −1 if and only if K = c(Szn −1 ) for some polynomial c(z). Thus, assume CΠ = Π C. Then there exists a linear transformation K in Xzn −1 satisfying [K]stst = C, and K commutes with Szn −1 . Therefore K = c(Szn −1 ) and C = circ (c0 , . . . , cn−1 ). Conversely, if C = circ(c0 , . . . , cn−1 ), we have CΠ = [c(Szn −1 )]stst [Szn −1 ]stst = [Szn −1 ]stst [c(Szn −1 )]stst = Π C. 5. Follows from c(Szn −1 )d(Szn −1 ) = d(Szn −1 )c(Szn −1 ). 6. Follows from c(Szn −1 )d(Szn −1 ) = (cd)(Szn −1 ). 7. Let C = circ (c0 , . . . , cn−1 ) = c(Szn −1 ), where c(z) = c0 + c1 z + · · · + cn−1zn−1 . By Theorem 5.18, c(Szn −1 ) is invertible if and only if c(z) and zn − 1 are coprime. In this case there exist polynomials a(z), b(z) satisfying the Bezout identity a(z)c(z) + b(z)(zn − 1) = 1. We may assume without loss of generality that deg a < n. From the Bezout identity we conclude that c(Szn −1 )c(Szn −1 ) = I and hence circ (c0 , . . . , cn−1 )−1 = [a(Szn −1 )]stst = circ (a0 , . . . , an−1 ). 8. The polynomial zn − 1 has a multiple zero if and only if zn − 1 and nzn−1 have a common zero. Clearly, this cannot occur. Since all the roots of zn − 1 are distinct, it follows from Proposition 5.15 that Szn −1 is diagonalizable. This implies the diagonalizability of c(Szn −1 ). 

5.4 Rational Models Given a field F, we saw that the ring of polynomials F[z] is an entire ring. Hence, by Theorem 1.49, it is embeddable in its field of quotients. We call the field of quotients of F[z] the field of rational functions and denote it by F(z). Strictly speaking, the elements of F(z) are equivalence classes of pairs of polynomials (p(z), q(z)), with q(z) nonzero. However, in each nonzero equivalence class, there is a unique pair with p(z), q(z) coprime and q(z) monic. The corresponding equivalence class will p(z) be denoted by q(z) . Given such a pair of polynomials p(z), q(z), there is a unique representation of p(z) in the form p(z) = a(z)q(z) + r(z), with deg r < deg q. This allows us to write r(z) p(z) = a(z) + . (5.28) q(z) q(z)

112

5 The Shift Operator

A rational function r(z)/q(z) with deg r ≤ degq will be called proper, and if p(z) deg r < deg q is satisfied, strictly proper. Thus, any rational function g(z) = q(z) has a unique representation as a sum of a polynomial and a strictly proper rational function. We denote by F− (z) the space of strictly proper rational functions and observe that it is an infinite-dimensional linear space. Equation (5.28) means that for F(z) we have the following direct sum decomposition: F(z) = F[z] ⊕ F−(z).

(5.29)

With this direct sum decomposition we associate two projection operators in F(z), π+ and π− , with images F[z] and F− (z) respectively. To be precise, given the representation (5.28), we have

 p =a q

 p r = . π− q q

π+

(5.30)

Proper rational functions have an expansion as formal power series in the variable gi n n k i z−1 , i.e., in the form g(z) = ∑∞ i=0 zi . Assume p(z) = ∑k=0 pk z , q(z) = ∑i=0 qi z , with qn = 0. We compute n

n

∞

n gj ∑ pk z = ∑ qi z ∑ z j = ∑ i=0 j=0 k=0 k=−∞ k

i



n

∑ qigi−k

 zk .

i=k

By comparing coefficients we get the infinite system of linear equations n

pk = ∑ qi gi−k ,

−∞ < k ≤ n.

i=k

This system has a unique solution, which can be found by solving it recursively, starting with k = n. An alternative way of finding this expansion is via the process of long division of p(z) by q(z). This generalizes easily to the case of the field of rational functions. We can consider the field F(z) of rational functions as a subfield of F((z−1 )), the field of truncated Laurent series. The space F− (z) of strictly proper rational functions can be viewed as a subspace of z−1 F[[z−1 ]]. In the same way that we have the isomorphism z−1 F[[z−1 ]]  F((z−1 ))/F[z], we have also F− (z)  F(z)/F[z]. In fact, the projection π− : F(z) −→ F− (z) is a surjective linear map with kernel equal to F[z]. However, the spaces F((z−1 )), F(z), and F[z] all carry also a natural F[z]-module structure, with polynomials acting by multiplication. This structure induces an F[z]-module structure in the quotient spaces. This module structure is transferred to F− (z) by defining, for p(z) ∈ F[z],

5.4 Rational Models

113

p · g = π− (pg),

g(z) ∈ F− (z).

(5.31)

In particular, we define the backward shift operator S− , acting in F− (z), by S− g = π− (zg),

g ∈ F− (z).

(5.32)

We shall use the same notation when z−1 F[[z−1 ]] replaces F− (z). We point out that in the behavioral literature, S− is denoted by σ . Using this, equation (5.31) can be rewritten, for p(z) ∈ F[z], as a Toeplitz-like operator p(σ )g = π− (pg),

g(z) ∈ z−1 F[[z−1 ]].

(5.33)

In terms of the expansion of g(z) ∈ F− (z) around infinity, i.e., in terms of the gi representation g(z) = ∑∞ i=1 zi we have ∞

∞ gi gi+1 = . ∑ i i i=1 z i=1 z

S− ∑

This explains the use of the term backward shift for this operator. We will show now that for the backward shift S− in z−1 F[[z−1 ]], any α ∈ F is an eigenvalue and it has associated eigenfunctions of any order. Indeed, let α ∈ F. Then α i−1 it is easy to check that (z − α )−1 = ∑∞ i=1 zi is an eigenvector of S− correspomding to the eigenvalue α . Analogously, (z − α )−k is a generalized eigenvector of S− of order k. Moreover, any other such eigenvector is necessarily of the form c(z − α )−k . Therefore, we have, for k ≥ 1, dim Ker (α I − S−)k = k. In contrast to this richness of eigenfunctions of the backward shift S− , the forward shift S+ does not have any eigenfunctions. This asymmetry will be further explored in the study of duality. The previous discussion indicates that the spectral structure of the shift S− , with finite multiplicity, might be rich enough to model all finite-dimensional linear transformations, up to similarity transformations, as the backward shift S− restricted to a finite-dimensional backward-shift-invariant subspace. This turns out to be the case, and this result lies at the heart of the matter, explaining the effectiveness of the use of shifts in linear algebra and system theory. In fact, the whole book can be considered, to a certain extent, to be an elaboration of this idea. We will study a special case of this in Chapter 6. It is easy to construct many finite-dimensional S− -invariant subspaces of F− (z). In fact, given any nonzero polynomial d(z), we let Xd =

r d

 | deg r < deg d .

It is easily checked that X d is indeed an S− -invariant subspace, and its dimension equals the degree of d(z).

114

5 The Shift Operator

It is natural to consider the restriction of the operator S− to X d . Thus we define a linear transformation Sd : X d −→ X d by Sd = S− |X d , or equivalently, for h ∈ X d , Sd h = S− h = π− zh.

(5.34)

The modules Xd and X d have the same dimension and are defined by the same polynomial. Just as the polynomial model Xd has been defined in terms of the projection πd , so can the rational model X d be characterized as the image of a projection. Definition 5.21. Let the map π d : z−1 F[[z−1 ]] −→ z−1 F[[z−1 ]] be defined by

π d h = π− d −1 π+ dh,

h(z) ∈ F[[z−1 ]].

(5.35)

Proposition 5.22. Let π d be defined by (5.35). Then 1. π d is a projection in z−1 F[[z−1 ]]. 2. We have

X d = Im π d .

(5.36)

3. With d(σ ) defined by (5.33), we have X d = Ker d(σ ).

(5.37)

Proof. 1. For h(z) ∈ z−1 F[[z−1 ]], we compute (π d )2 h = π d (π d h) = π− d −1 π+ d π− d −1 π+ dh = π− d −1 π+ dd −1 π+ dh = π− d −1 π+ π+ dh = π− d −1 π+ dh = π d h, i.e., π d is a projection. 2. Assume h(z) = d −1 r, with deg r < deg d. Then

π d (d −1 r) = π− d −1 π+ dd −1 r = π− d −1 r = d −1 r, which shows that X d ⊂ Im π d . Conversely, assume h(z) ∈ Im π d . Thus there exists g(z) ∈ z−1 F[[z−1 ]] for which h(z) = π d g. Hence

π d h = (π d )2 g = π d g = h. From this it follows that

π− dh = π− d π d h = π− d π− d −1 π+ dh = π− dd −1 π+ dh = π− π+ dh = 0.

5.4 Rational Models

115

This implies that there exists r(z) ∈ F[z] for which d(z)h(z) = r(z) and deg r < deg d. In turn, we conclude that h(z) = d(z)−1 r(z) ∈ X d , which implies the inclusion Im π d ⊂ X d . 3. Assume h ∈ Ker d(σ ), which implies π+ dh = dh. We compute

π d h = π− d −1 π+ dh = π− d −1 dh = π− h = h, i.e., Ker d(σ ) ⊂ X d . Conversely, assume h ∈ X d . This implies h = π− d −1 π+ dh. We compute d(σ )h = π− dh = π− d π− d −1 π+ dh = π− dd −1 π+ dh = π− π+ dh = 0, i.e., X d ⊂ Ker d(σ ). The two inclusions imply (5.37).



It is natural to conjecture that the polynomial model Xd and the rational model X d must be isomorphic, and this is indeed the case, as the next theorem shows. Theorem 5.23. Let d(z) be a nonzero polynomial. Then the operators Sd and Sd are isomorphic. The isomorphism is given by the map ρd : X d −→ Xd defined by

ρd h = dh.

(5.38)

Proof. We compute

ρd Sd h = ρd π− zh = d π− zh = d π− d −1 dzh = πd z(dh) = Sd (ρd h), i.e.,

ρd S d = S d ρd ,

which, by the invertibility of ρd , proves the isomorphism.

(5.39) 

The polynomial and rational models that have been introduced are isomorphic, yet they represent two fundamentally different points of view. In the case of polynomial models, all spaces of the form Xq , with deg q = n, contain the same elements, but the associated shifts Sq act differently. On the other hand, the operators Sq in the spaces X q act in the same way, since they are all restrictions of the backward shift S− . However, the spaces are different. Thus the polynomial models represent an arithmetic perspective, whereas rational models represent a geometric one. Our next result is the characterization of all finite-dimensional S− -invariant subspaces. Proposition 5.24. A subset M of F− (z) is a finite-dimensional S− -invariant subspace if and only if we have M = X d .

116

5 The Shift Operator

Proof. Assume that for some polynomial d(z), M = X d . Then S−

r r = π− z = d −1 d π− d −1 zr = d −1 πd zr ∈ M. d d

Conversely, let M be a finite-dimensional S− -invariant subspace. By Theorem 4.54, there exists a nonzero polynomial p(z) of minimal degree such that π− ph = 0, for all h ∈ M. Thus, we get M ⊂ X p . It follows that pM is a submodule of X p , hence of the form p1 X p2 for some factorization p(z) = p1 (z)p2 (z). We conclude that M = −1 p2 p−1 p1 X p2 = p−1 2 p1 p1 X p2 = X . The minimality of p(z) implies p(z) = p2 (z), up to a constant nonzero factor.  The spaces of rational functions of the form X d will be referred to as rational models. Note that in the theory of differential equations, these spaces appear as the Laplace transforms of the spaces of solutions of a homogeneous linear differential equation with constant coefficients. The following sums up the basic arithmetic properties of rational models. It is the counterpart of Proposition 5.7. Proposition 5.25. 1. Given polynomials p(z), q(z) ∈ F[z], we have the inclusion X p ⊂ X q if and only if p(z) | q(z). 2. Given polynomials pi (z) ∈ F[z], i = 1, . . . , s, then ∩si=1 X pi = X p with p(z) the g.c.d. of the pi (z). 3. Given polynomials pi ∈ F[z], then ∑si=1 X pi = X q with q(z) the l.c.m. of the pi (z). Proof. Follows from Proposition 5.7, using the isomorphism of polynomial and rational models given by Theorem 5.23.  The primary decomposition theorem and the direct sum representation (5.16) have a direct implication toward the partial fraction decomposition of rational functions. s p (z)νi be the primary decomposition of the nonzero Theorem 5.26. Let p(z) = Πi=1 i polynomial p(z). Then

1. We have

ν1

νs

X p = X p1 ⊕ · · · ⊕ X ps . 2. Each rational function g(z) ∈

Xp

(5.40)

has a unique representation of the form

g(z) =

s νi ri j r(z) = ∑ ∑ j, p(z) i=1 j=1 pi

with deg ri j < deg pi , j = 1, . . . , νi . Proof. 1. Given the primary decomposition of p(z), we define πi (z) = Π j=i pνi i . Clearly, by Corollary 5.9, we have X p = π1 X pν1 ⊕ · · · ⊕ πs X psνs . 1

(5.41)

5.4 Rational Models

117

We use the isomorphism of the modules X p and X p and the fact that p(z) = νi

νi pi , to get the direct πi (z)pi (z)νi , which implies p−1 πi X pνi = p− i πi −1πi X piνi = X i sum decomposition (5.40). i −1 2. For any ri (z) ∈ X pνi we have ri = ∑νj=0 ri(νi − j) pij . With rp = ∑si=1 rνi i , using (5.41) pi

i

and taking ri ∈ X pνi , we have i

ri = pνi i

νi

∑

j=1

ri j pij

.

(5.42) 

The isomorphism (5.39) between the shifts Sq and Sq can be used to transform Theorems 5.17 and 5.18 into the context of rational models. Thus we have the following Theorem. Theorem 5.27. 1. Let q(z) be a monic polynomial and Sq the shift operator in X q defined by (5.34). Let W be any operator in X q that commutes with Sq . Then there exists an operator Z that commutes with Sq and such that Z = ρqW ρq−1 .

(5.43)

Equivalently, the following diagram is commutative: W Xq

-

ρq ? Xq

Xq

ρq Z

-

? Xq

2. An operator W in X q commutes with Sq if and only if there exists a polynomial p(z) ∈ F[z] for which W h = π− ph = p(Sq )h,

h(z) ∈ X q .

(5.44)

Proof. 1. By the isomorphism (5.39), we have ρq Sq = Sq ρq . Since W Sq = SqW , it follows that W ρq−1 Sq ρq = ρq−1 Sq ρqW , or (ρqW ρq−1 )Sq = Sq (ρqW ρq−1 ). Defining Z = ρqW ρq−1 , we have ZSq = Sq Z. 2. If W is represented by (5.44), then SqW h = π q zπ− ph = π− q−1 π+ qzπ− ph = π− pq−1 q−1 zπ− ph = π− zπ− ph = W Sq h.

118

5 The Shift Operator

Conversely, assume W Sq = SqW , and define Z by (5.43). Then we have ZSq = Sq Z. Applying Theorem 5.17, there exists a polynomial p(z) ∈ F[z] for which Z = p(Sq ). Since W = ρq−1 Z ρq , this implies W h = ρq−1 Z ρq h = q−1 qπ− q−1 pqh = π− ph.



Theorem 5.28. Given the polynomials p(z), q(z), with q(z) monic. Let r(z) and s(z) be the g.c.d. and the l.c.m. of p(z) and q(z) respectively. Then 1. We have the factorizations q(z) = r(z)q1 (z), p(z) = r(z)p1 (z),

(5.45)

with p1 (z), q1 (z) coprime, as well as the factorizations s(z) = r(z)p1 (z)q1 (z) = p(z)q1 (z) = q(z)p1 (z).

(5.46)

We have Ker p(Sq ) = X r , Im p(Sq ) = X q1 .

(5.47)

2. Let p(z), q(z) ∈ F[z], with q(z) nonzero. Then the linear transformation p(Sq ) is invertible if and only if p(z) and q(z) are coprime. Moreover, we have p(S q )−1 = a(Sq ),

(5.48)

where the polynomial a(z) arises out of any solution of the Bezout equation a(z)p(z) + b(z)q(z) = 1.

(5.49) 

5.5 The Chinese Remainder Theorem and Interpolation The roots of the Chinese remainder theorem are in number theory. However, we interpret it, the underlying ring taken to be F[z], as an interpolation result.

5.5 The Chinese Remainder Theorem and Interpolation

119

Theorem 5.29 (Chinese remainder theorem). Let qi (z) ∈ F[z] be mutually coprime polynomials and let q(z) = q1 (z) · · · qs (z). Then, given polynomials ai (z) such that deg ai < deg qi , there exists a unique polynomial f (z) ∈ Xq , i.e., such that deg f < deg q, and for which πqi f = ai . Proof. The interesting thing about the proof of the Chinese remainder theorem is its use of coprimeness in two distinct ways. One is geometric, the other one is spectral. Let us define d j (z) = ∏i= j qi (z). The mutual coprimeness of the qi (z) implies the direct sum decomposition Xq = d1 Xq1 ⊕ · · · ⊕ ds Xqs .

(5.50)

This is the geometric use of the coprimeness assumption. The condition deg f < deg q is equivalent to f (z) ∈ Xq . Let f (z) = ∑sj=1 d j (z) f j (z) with f j (z) ∈ Xq j . Since for i = j, qi (z) | d j (z), it follows that in this case πqi d j f j = 0. Hence

π q i f = π qi

s

∑ d j f j = πqi di fi = di (Sqi ) fi .

j=1

For the spectral use of coprimeness, we observe that the pairwise coprimeness of the qi (z) implies the coprimeness of di (z) and qi (z). In turn, this shows, applying Theorem 5.18, that the module homomorphism di (Sqi ) in Xqi is actually an isomorphism. Hence, there exists a unique fi (z) in Xqi such that ai = di (Sqi ) fi and fi = di (Sqi )−1 ai . So f = ∑sj=1 d j di (Sqi )−1 ai is the required polynomial. Note that the inversion of di (Sqi ) can be done easily, as in Theorem 5.18, using the Euclidean algorithm. The uniqueness of f (z), under the condition f (z) ∈ Xq , follows from the fact that (5.50) is a direct sum representation. This completes the proof. 

5.5.1 Lagrange Interpolation Revisited Another solution to the Lagrange interpolation problem, introduced in Chapter 2, can be easily derived from the Chinese remainder theorem. Indeed, given distinct numbers αi ∈ F, i = 1, . . . , n, the polynomials qi (z) = (z − αi ) are mutually coprime. We define q(z) = Π nj=1 q j (z) and qi (z) = Π j=i q j (z), which leads to the factorizations q(z) = di (z)qi (z). The coprimeness assumption implies the direct sum representation Xq = d1 Xq1 ⊕ · · · ⊕ dn Xqn . Thus, any polynomial f (z) ∈ Xq has a unique representation of the form f (z) =

n

∑ c j d j (z).

j=1

(5.51)

120

5 The Shift Operator

We want to find a polynomial f (z) that satisfies the interpolation conditions f (λi ) = ai , i = 1, . . . , n. Applying the projection πqi to the expansion (5.51), and noting that πqi d j = 0 for j = i, we obtain ai = f ( λ i ) = π q i f = π q i

n

∑ c j d j (z) = ci di (λi ).

j=1

Defining the Lagrange interpolation polynomials by li (z) = di (z)/di (λi ), we get f (z) = ∑nj=1 a j l j (z) for the unique solution in Xq of the Lagrange interpolation problem. Any other solution differs by a polynomial that has a zero at all the points λi , hence is divisible by q(z).

5.5.2 Hermite Interpolation We apply now the Chinese remainder theorem to the problem of higher-order interpolation, or Hermite interpolation. In Hermite interpolation, which is a generalization of the Lagrange interpolation problem, we prescribe not only the value of the interpolating polynomial at given points, but also the value of a certain number of derivatives, the number of which may differ from point to point. Specifying the first ν derivatives, counting from zero, of a polynomial p(z) at a point α means that we are given a representation f (z) =

ν −1

∑

fi,α (z − αi )i + (z − α )ν g(z).

i=0

ν −1 fi,α (z − αi )i < ν , this means that Of course, since deg ∑i=0 ν −1

deg ∑ fi,α (z − α )i = deg π(z−α )ν f . i=0

Hence we can formulate the Hermite interpolation problem: Given distinct i −1 α1 , . . . , αk ∈ F, positive integers ν1 , . . . , νk , and polynomials fi (z) = ∑νj=0 f j,αi (z − j αi ) , find a polynomial f (z) such that

π(z−αi )νi f = fi ,

i = 1, . . . , k.

(5.52)

Proposition 5.30. There exists a unique solution f (z), of degree < n = ∑ki=1 νi , to the Hermite interpolation problem. Any other solution of the Hermite interpolation problem is of the form f (z) + p(z)g(z), where g(z) is an arbitrary polynomial and p(z) is given by k p(z) = Πi=1 (z − αi )νi . (5.53)

5.5 The Chinese Remainder Theorem and Interpolation

121

Proof. We apply the Chinese remainder theorem. Obviously, the polynomials (z − αi )νi , i = 1, . . . , k are mutually coprime. Then, with p(z) defined by (5.53), there exists a unique f (z) with deg f < n for which (5.52) holds. If fˆ(z) is any other solution, then h(z) = ( f (z) − fˆ(z)) satisfies π(z−αi )νi h = 0, that is h(z) is divisible by (z − αi )νi . Since these polynomials are mutually coprime, it follows that p(z) | h(z), or h(z) = p(z)g(z) for some polynomial g(z). 

5.5.3 Newton Interpolation There are situations in which the interpolation data are given to us sequentially. At each stage we solve the corresponding interpolation problem. Newton interpolation is recursive in the sense that the solution at time k is the basis for the solution at time k + 1. Definition 5.31. Given λi , ai ∈ F, i = 0, 1, . . . , we define the Newton interpolation problem NIP(i): Find polynomials fi (z) ∈ Xdi , i ≥ 1, satisfying the interpolation conditions NIP(i) : fi (λ j ) = a j ,

0 ≤ j ≤ i − 1.

(5.54)

Theorem 5.32. Given λi , ai ∈ F, i = 0, 1, . . . , with the λi distinct, we define polynomials by qi (z) = z − λi , di (z) = Π i−1 j=0 (z − λ j ).

(5.55)

di+1 (z) = di (z)qi (z),

(5.56)

Then 1. We have the factorizations

with di (z), qi (z) coprime. 2. We have the following direct sum decomposition: Xdi+1 = Xdi ⊕ diXqi .

(5.57)

3. Every f (z) ∈ Xdi+1 has a unique representation of the form f (z) = g(z) + cdi(z), for some g(z) ∈ Xdi and c ∈ F.

(5.58)

122

5 The Shift Operator

4. Let fi (z) be the solution to the Newton interpolation problem NIP(i), then there exists a constant for which f i+1 (z) = fi (z) + di (z)ci

(5.59)

is the solution to the Newton interpolation problem NIP(i + 1) and it has the representation fi+1 (z) =

i

∑ c j d j (z),

(5.60)

j=0

where cj =

f j+1 (λ j ) − f j (λ j ) . d j (λ j )

(5.61)

Proof. 1. Follows from (5.55). The coprimeness of di (z) and qi (z) is a consequence of our assumption that the λ j are distinct. 2. Follows from the factorization (5.56) by applying Proposition 5.6. 3. Follows from the direct sum representation (5.57). 4. In equation (5.59), we substitute the corresponding expression for fi (z) and proceed inductively to get (5.60). Alternatively, we note that degdi = i, for i ≥ 0; hence {d0 (z), . . . , di (z)} is a basis for Xdi+1 and an expansion (5.60) exists. Let fi+1 (z) be defined by (5.59). Since di (λ j ) = 0 for j = 0, . . . , i−1, it follows that fi+1 (λ j ) = a j for j = 0, . . . , i − 1. All we need for fi+1 (z) to be a solution of NIP(i + 1) is to choose ci such that ai+1 = fi+1 (λi ) = fi (λi ) + di (λi )ci or, since di (λi ) = 0, we get (5.61). 

5.6 Duality The availability of both polynomial and rational models allows us to proceed with a deeper study of duality. Our aim is to obtain an identification of the dual space to a polynomial model in terms of a polynomial model. On F(z), and more generally on F((z−1 )), we introduce a bilinear form as nf ng follows. Given f (z) = ∑ j=−∞ f j z j and g(z) = ∑ j=−∞ g j z j , let [ f , g] =

∞

∑

f j g− j−1 .

(5.62)

j=−∞

Clearly, the sum in (5.62) is well defined, since only a finite number of summands are nonzero. Given a subspace M ⊂ F(z), we let M ⊥ = { f ∈ F(z)|[m, f ] = 0, ∀m ∈ M}. It is easy to check that F[z]⊥ = F[z]. We will need the following simple computational rule.

5.6 Duality

123

Proposition 5.33. Let φ (z), f (z), g(z) be rational functions. Then [φ f , g] = [ f , φ g].

(5.63)

Proof. With the obvious notation we compute [φ f , g] = ∑∞j=−∞ (φ f ) j h− j−1 = ∑∞j=−∞ (∑∞ i=−∞ φi f j−i )h− j−1 ∞ ∞ ∞ = ∑∞ i=−∞ f j−i ∑ j=−∞ φi h− j−1 = ∑k=−∞ f k ∑ j=−∞ φ j−k h− j−1 ∞ ∞ = ∑∞ k=−∞ f k ∑i=−∞ φi h−i−k−1 = ∑k=−∞ f k (φ h)−k−1 = [ f , φ h].

 Multiplication operators in F(z) of the form Lφ h = φ h are called Laurent operators. The function φ is called the symbol of the Laurent operator. Before getting the representation of the dual space to a polynomial model, we derive a representation of the dual space to F[z]. Theorem 5.34. The dual space of F[z] can be identified with z−1 F[[z−1 ]]. Proof. Clearly, every element h(z) ∈ z−1 F[[z−1 ]] defines, by way of the pairing (5.62), a linear functional on F[z]. Conversely, given a linear functional Φ on F[z], it induces linear functionals φi on F by defining, for ξ ∈ F,

φi (ξ ) = [zi ξ , φ ] = Φ (zi ξ ), and an element h ∈ z−1 F[[z−1 ]] is defined by letting h(z) = ∑∞j=0 φ j z− j−1 . It follows that Φ ( f ) = [ f , h].  Point evaluations are clearly linear functionals in F[z]. It is easy to identify the representing functions. In fact, this is an algebraic version of Cauchy’s theorem. Proposition 5.35. Let α ∈ F and f (z) ∈ F[z]. Then f (α ) = [ f , (z − α )−1 ]. i−1 z−i . So this follows from (5.62), since Proof. We have (z − α )−1 = ∑∞ i=1 α

 f,

 n 1 = ∑ fi α i = f (α ). z−α i=0 

Theorem 5.36. Let M = dF[z] with d(z) ∈ F[z]. Then M ⊥ = X d .

124

5 The Shift Operator

Proof. Let f (z) ∈ F[z] and h(z) ∈ M. Then 0 = [d f , h] = [ f , dh] = [ f , π− dh]. But this implies d(z)h(z) ∈ Xd , or h(z) ∈ X d .



Next, we compute the adjoint of the projection πd : F[z] −→ F[z]. Clearly πd∗ is a transformation acting in z−1 F[[z−1 ]]. Theorem 5.37. The adjoint of πd is π d . Proof. Let f (z) ∈ F[z] and h(z) ∈ z−1 F[[z−1 ]]. Then ˜ ˜ = [d −1 f , π+ dh] [πd f , h] = [d π− d −1 f , h] = [π− d −1 f , dh] −1 −1 −1 ˜ ˜ ˜ ˜ ˜ ˜ = [ f , d π+ dh] = [π+ f , d π+ dh] = [ f , π− d π+ dh] ˜ d = [ f , π h].



Not only are we interested in the study of duality on the level of F[z] and its dual space z−1 F[[z−1 ]], but also we would like to study it on the level of the modules Xd and X d . The key to this study is the fact that if X is a vector space and M a subspace, then (X/M)∗  M ⊥ . Theorem 5.38. Let d(z) ∈ F[z] be nonsingular. Then Xd∗ is isomorphic to X d and Sd∗ = Sd . Proof. Since Xd is isomorphic to F[z]/dF[z], then Xd∗ is isomorphic to (F[z]/dF[z])∗ , which, in turn, is isomorphic to (dF[z])⊥ . However, this last module is X d . It is clear that under the duality pairing we introduced, we actually have Xd∗ = X d . Finally, let f (z) ∈ Xd and let h(z) ∈ X d . Then   [Sd f , h] = [πd z f , h] = z f , π d h , [z f , h] = [ f , zh] = [π+ f , zh], [ f , π− zh] = [ f , S− h] = f , Sd h . Hence, we can identify Xd˜ with Xd∗ by defining a new pairing  f , g = [d −1 f , g] = [ f , d −1 g] for all f (z), g(z) ∈ Xd .

(5.64) 

As a direct corollary of Theorem 5.38 we have the following. Theorem 5.39. The dual space of Xd under the pairing  ,  introduced in (5.64) is Xd , and moreover, Sd∗ = Sd .  With the identification of the polynomial model Xd with its dual space, we can identify some pairs of dual bases.

5.6 Duality

125

Proposition 5.40. 1. Let d(z) = zn + dn−1 zn−1 + · · · + d0 , and let Bst = {1, z, . . . , zn−1 } be the standard and Bco = {e1 (z), . . . , en (z)} the control bases respectively of Xd . Then Bco = Bst∗ , that is, the control and standard bases are dual to each other. n (z − λ ), with the λ distinct. Let B = {π (z), . . . , π (z)}, with 2. Let d(z) = Πi=1 i i in n 1 πi (z) the Lagrange interpolation polynomials, be the interpolation basis in Xd . Let Bsp = {p1 (z), . . . , pn (z)} be the spectral basis, with pi (z) = Π j=i (z − λ j ). ∗ =B . Then Bsp in Proof. 1. We use (5.64) to compute zi−1 , e j  = [d −1 zi−1 , π+ z− j d] = [d −1 zi−1 , z− j d] = [zi− j−1 , 1] = δi j . 2. We use Proposition 5.35 and note that for every f (z) ∈ Xd we have  f , pi  = [d

−1

  1 pi  = f, f , pi ] = f , = f (λi ). d z − λi 

In particular, πi , p j  = πi (λ j ) = δi j .



This result explains the connection between the two companion matrices given in Proposition 5.4. Indeed,  ∗ co co   Cq = [Sq ]stst = [S q ]co = [Sq ]co = Cq .

Next we compute the change of basis transformations. Proposition 5.41. 1. Let q(z) = zn + qn−1zn−1 + · · · + q0 ,. Then ⎛

q1 . . qn−1 ⎜ . . ⎜ ⎜ [I]stco = ⎜ . . ⎜ ⎝ qn−1 . 1 and

1

⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠

(5.65)

⎞ 1 ⎜ . ψ1 ⎟ ⎜ ⎟ ⎜ ⎟ co [I]st = ⎜ ... ⎟, ⎜ ⎟ ⎝ . ... ⎠ 1 ψ1 . . ψn−1

(5.66)

⎛

126

5 The Shift Operator

where for q (z) = zn q(z−1 ), ψ (z) = ψ0 + · · · + ψn−1 zn−1 is the unique solution, of degree < n, of the Bezout equation q (z)ψ (z) + zn σ (z) = 1.

(5.67)

2. The matrices in (5.65) and (5.66) are inverses of each other. 3. Let d(z) = Π nj=1 (z − α j ) with α1 , . . . , αn distinct and let Bsp , Bin be the corresponding spectral and interpolation bases of Xd . Then we have the following change of basis transformations: ⎛

1 α1 ⎜. . ⎜ ⎜ [I]in st = ⎜ . . ⎜ ⎝. . 1 αn

⎞ . . α1n−1 .. . ⎟ ⎟ ⎟ . . . ⎟, ⎟ .. . ⎠ . . αnn−1

(5.68)

⎛

⎞ 1 ... 1 ⎜ α ... α ⎟ ⎜ 1 n ⎟ ⎜ ⎟ [I]co . . . . ⎟, sp = ⎜ . ⎜ ⎟ ⎝ . ... . ⎠ α1n−1 . . . αnn−1 ⎞ ⎛ p1 (α1 ) ⎟ ⎜ . ⎟ ⎜ ⎟ ⎜ in [I]sp = ⎜ . ⎟. ⎟ ⎜ ⎠ ⎝ . pn (αn )

(5.69)

(5.70)

Proof. 1. Note that if ψ (z) is a solution of (5.67), then necessarily ψ0 = 1. With J the transposition matrix defined in (8.39), we compute st −1 J[I]co = (q (Szn ))−1 . st = ([I]co J)

However, if we consider the map Szn , then ⎞ ⎛ 1 qn−1 . . . q1 ⎟ ⎜ . .... ⎟ ⎜ ⎟ ⎜ .... ⎟ ⎜ ⎟ = q (Szn ), ⎜ ⎟ ⎜ ... ⎟ ⎜ ⎝ . qn−1 ⎠ 1 and its inverse is given by ψ (Szn ), where ψ (z) solves the Bezout equation (5.67). This completes the proof.

5.7 Universality of Shifts

127

2. Follows from the fact that [I]stco [I]co st = I. 3. The matrix representation for [I]in st has been derived in Corollary 2.36. The matrix representation for [I]co sp follows from (5.68) by applying duality theory, in particular Theorem 4.38. We can also derive this matrix representation directly, which we proceed to do. For this, we define polynomials s1 (z), . . . , sn (z) by si (z) = e1 (z) + αi e2 (z) + · · · + αin−1en (z). We claim that si (z) are eigenfunctions of Sd corresponding to the eigenvalues αi . Indeed, using equation (5.12) and the fact that 0 = q(αi ) = q0 + q1αi + · · · + αin , we have Sq si = −q0 en + αi (e1 − q1 en ) + · · · + αi (en−1 − qn−1 en ) = αi e1 + · · · + αin−1 en−1 − (q0 + · · · + qn−1αin−1 )en = αi e1 + · · · + αin en = αi si (z). This implies that there exist constants γi such that si (z) = γi pi (z). Since both si (z) and ei (z) are obviously monic, it follows that necessarily γi = 1 and si (z) = pi (z). The equations pi (z) = e1 (z) + αi e2 (z) + · · · + αin−1en (z) imply the matrix representation (5.69). Finally, the matrix representation in (5.70) follows from the trivial identities pi (z) = pi (αi )πi (z).



5.7 Universality of Shifts We end this chapter by explaining the reason that the polynomial and rational models we introduced can be used so effectively in linear algebra and its applications. The main reason for this is the universality property of shifts. For our purposes, we shall show that any linear transformation in a finite-dimensional vector space V over the field F is isomorphic to the compression of the forward shift S+ : F[z]n −→ F[z]n , defined in (1.31), to a quotient space, or alternatively, to the restriction of the backward shift S− : z−1 F[[z−1 ]]n −→ z−1 F[[z−1 ]]n , defined in (1.32), to an invariant subspace. For greater generality, we will have to stray away from scalar-valued polynomial and rational functions. Assume we are given a linear transformation A in a finite-dimensional vector space V , over the field F. Without loss of generality, by choosing a matrix representation, we may as well assume that V = Fn and that A is a square matrix. By Fn [z] we denote the space of vector polynomials with coefficients in Fn , whereas F[z]n will denote the space of vectors with polynomial entries. We will use freely the isomorphism between Fn [z] and F[z]n and we will identify the two spaces.

128

5 The Shift Operator

With the linear polynomial matrix zI − A we associate a map πzI−A : F[z]n −→ Fn , given by k

πzI−A ∑ ξ j z j = j=0

k

∑ A jξ j.

(5.71)

j=0

The operation defined above can be considered as taking the remainder of a polynomial vector after left division by the polynomial matrix zI − A. Proposition 5.42. For the map πzI−A defined by (5.71) 1. We have πzI−A is surjective. 2. We have Ker πzI−A = (zI − A)F[z]n . 3. For the map S+

: F[z]n

−→

F[z]n

(5.72)

defined by

(S+ f )(z) = z f (z), the following diagram is commutative: πzI−A F[z]n

-

(5.73)

Fn

S+

A

πzI−A

? F[z]n This implies that

-

? Fn

Ax = πzI−A z · x.

(5.74)

4. Given a polynomial p(z) ∈ F[z], we have p(A)x = πzI−A p(z)x.

(5.75)

Proof. 1. For each constant polynomial x ∈ F[z]n , we have πzI−A x = x. The surjectivity follows. 2. Assume f (z) = (zI − A)g(z) with g(z) = ∑ki=0 gi zi . Then k

k

k

i=0

i=0

i=0

f (z) = (zI − A) ∑ gi zi = ∑ gi zi+1 − ∑ Agi zi . Therefore

k

k

i=0

i=0

πzI−A f = ∑ Ai+1 gi − ∑ Ai Agi = 0, i.e.,

(zI − A)F[z]n

⊂ Ker πzI−A .

5.7 Universality of Shifts

129

Conversely, assume f (z) = ∑ki=0 fi zi ∈ Ker πzI−A , that is, ∑ki=0 Ai fi = 0. i−1− j A j , we compute Recalling that zi I − Ai = (zI − A) ∑i−1 j=0 z f (z) = ∑ki=0 fi zi = ∑ki=0 fi zi − ∑ki=0 Ai fi i−1− j A j ) f = ∑ki=0 (zi I − Ai ) fi = ∑ki=0 (zI − A)(∑i−1 i j=0 z i−1− j A j ) f = (zI − A)g, = (zI − A) ∑ki=0 (∑i−1 z i j=0 so Ker πzI−A ⊂ (zI − A)F[z]n ; Hence the equality (5.72) follows. 3. To prove the commutativity of the diagram, we compute, with f (z) = ∑ki=0 fi zi ,

πzI−A S+ f = πzI−A z ∑ki=0 fi zi = πzI−A ∑ki=0 fi zi+1 = ∑ki=0 Ai+1 fi = A ∑ki=0 Ai fi = AπzI−A f . 4. By linearity, it suffices to prove this for polynomials of the form zk . We do this by induction. For k = 1 this holds by equation (5.74). Assume it holds up to k − 1. Using the fact that zKer (zI − A) = z(zI − A)F[z]n ⊂ (zI − A)F[z]n = Ker (zI − A), we compute

πzI−A zk x = πzI−A zπzI−A zk−1 x = πzI−A zAk−1 x = AAk−1 x = Ak x.



Clearly, the equality f (z) = (zI − A)g(z) + πzI−A f can be interpreted as πzI−A f being the remainder of f (z) after division by zI − A. As a corollary, we obtain the celebrated Cayley–Hamilton theorem. Theorem 5.43 (Cayley–Hamilton). Let A be a linear transformation in an ndimensional vector space U over F and let dA (z) be its characteristic polynomial. Then dA (A) = 0. Proof. By Cramer’s rule, we have dA (z)I = (zI − A)adj(zI − A); hence we have the inclusion dA (z)F[z]n ⊂ (zI − A)F[z]n . This implies, for each x ∈ U , that dA (A)x = πzI−A d(z)x = 0, so dA (A) = 0.



Corollary 5.44. Let A be a linear transformation in an n-dimensional vector space U over F. Then its minimal polynomial mA (z) divides its characteristic polynomial dA (z). Theorem 5.45. Let A be a linear transformation in Fn . Then A is isomorphic to S− restricted to a finite-dimensional S− -invariant subspace of z−1 F[[z−1 ]]n . Specifically, let Φ : Fn −→ z−1 F[[z−1 ]]n be defined by

130

5 The Shift Operator

Φξ = (zI − A)−1 ξ ,

(5.76)

L = Im Φ .

(5.77)

and let

Then the following diagram is commutative:

Φ

Fn

- Im Φ

S− |Im Φ

A ? Fn

Φ

? - Im Φ

which implies the isomorphism A  S− |Im Φ .

(5.78)

Proof. Note that the map Φ is injective, hence invertible as a map from Fn onto L = Im Φ = {(zI − A)−1 ξ |ξ ∈ Fn }. Since (zI − A)−1ξ =

∞

∑ A j ξ z−( j+1)

j=0

L is a subspace of z−1 F[[z−1 ]]n . Since S− Φξ = S− (zI − A)−1ξ = π− z(zI − A)−1ξ = π− (zI − A + A)(zI − A)−1ξ = (zI − A)−1 Aξ = Φ Aξ , the S− -invariance of L follows as well as the isomorphism (5.78).



Remark 5.46. 1. The underlying idea is that the subspace Im Φ contains all the information about A. The study of L = Im Φ as an F[z]-module is a tool for the study of the transformation A. 2. The subspace L = Im Φ inherits an F[z]-module structure from z−1 F[[z−1 ]]m . With Fn having the F[z]-module structure induced by A, then Φ is clearly an F[z]-module homomorphism.

5.8 Exercises

131

5.8 Exercises 1. Assume q(z) = zn + qn−1zn−1 + · · · + q0 is a real or complex polynomial. Show that the solutions of the linear, homogeneous differential equation y(n) + qn−1 y(n−1) + · · · + q0 y = 0 form an n-dimensional space. Use the Laplace transform L to show that y is a solution if and only if L (y) ∈ X q . 2. Let T be a cyclic transformation with minimal polynomial mT (z) of degree n, and let p(z) ∈ F[z]. Show that the following statements are equivalent: a. b. c. d.

The operator p(T ) is cyclic. There exists a polynomial q(z) ∈ F[z] such that (q ◦ p)(z) = z mod (mT ). The map Λ p : F[z] −→ XmT defined by Λ p (q) = πmT (q ◦ p) is surjective. j We have det(πk j ) = 0, where πk = πmT (pk ) and πk (z) = ∑n−1 j=0 πk j z .

3. Assume the minimal polynomial of a cyclic operator T factors into linear factors, i.e., mT (z) = ∏(z − λi )νi with the λi distinct. Show that p(T ) is cyclic if and only if λi = λ j implies p(λi ) = p(λ j ) and p (λi ) = 0 whenever νi > 1. 4. Show that if π (z) ∈ F[z] is irreducible and deg p is a prime number, then p(Sπ ) is either scalar or a cyclic transformation. 5. Let q(z) = zn + qn−1 zn−1 + · · ·+ q0 and let Cq be its companion matrix. Let f (z) ∈ Xq with f (z) = f0 + · · · + fn−1 zn−1 . We put ⎛ ⎜ ⎜ ⎜ f=⎜ ⎜ ⎝

f0 . . .

⎞ ⎟ ⎟ ⎟ ⎟ ∈ Fn . ⎟ ⎠

fn−1   Show that f (Cq ) = f,Cq f, . . . ,Cqn−1 f . 6. Given a linear transformation A in V , a vector x ∈ V , and a polynomial p(z), define the Jacobson chain matrix by Cm (p, x, A) = (x, Ax, . . . , Ar−1 x, . . . , p(A)m−1 x, p(A)m−1 Ax, . . . , p(A)m−1 Ar−1 x). Prove the following a. Let E = Cm (p, 1,C pm ) then E is a solution of C pm E = EH(pm ). b. Every solution X of C pm X = XH(pm) is of the form X = f (Cpm )E for some polynomial f (z) of degree < m deg p.

132

5 The Shift Operator

m−1 + · · · + p and q(z) = zn + q n−1 + · · · + q . Let 7. Let p(z) = zm + p m−1 z 0 n−1 z 0  Cp 0 H(p, q) = , where N is the n × m matrix whose only nonzero element 

N Cq

is N1m = 1. Show that the general solution to the equation Cpq X = XH(pq) is of the form X = f (Cpq )K, where deg f < deg p + degq and ⎛

1

p0 . . . . . .

⎜ . ⎜ ⎜ . ⎜ ⎜ ⎜ . ⎜ ⎜ . 1 pm−1 ⎜ K=⎜ 1 ⎜ ⎜ . ⎜ ⎜ . ⎜ ⎜ ⎜ . ⎜ ⎝ .

⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ p0 ⎟ ⎟. . ⎟ ⎟ ⎟ . ⎟ ⎟ . ⎟ ⎟ . ⎠ 1

8. Let C be the circulant matrix ⎛

c0 cn−1 ⎜ c ⎜ 1 c0 ⎜ C = circ (c0 , . . . , cn−1 ) = ⎜ . . ⎜ ⎝ . . cn−1 .

. . . . .

⎞ . c1 . . ⎟ ⎟ ⎟ . . ⎟. ⎟ . cn−1 ⎠ c1 c0

Show that det(C) = ∏ni=1 (c0 + c1 ζi + · · · + cn−1 ζin−1 ), where 1 = ζ1 , . . . , ζn are the distinct nth roots of unity, that is, the zeros of zn − 1. 9. Prove the following barycentric representation for the minimal-degree polynomial satisfying the interpolation constraints f (λi ) = ai , i = 0, . . . , n − 1, namely ⎧ wj ⎪ ai ⎪ ∑n−1 i=0 ⎪ ⎪ z− λj ⎪ ⎪ ⎨ n−1 w j , ∑i=0 f (z) = z− λj ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ , ai Here wi =

1 Π j=i (λi −λ j ) .

z = λ j , j = 0, . . . , n − 1,

z = λ j , j = 0, . . . , n − 1.

5.9 Notes and Remarks

133

5.9 Notes and Remarks Shift operators are a cornerstone of modern operator theory. As we explained in Section 5.7, their interest lies in their universality properties, an extremely important fact, first pointed out in Rota (1960). This observation is the key insight to the use of shift operators in modeling linear transformations as well as, more generally, linear systems. The fact that any linear transformation T acting in a finite-dimensional vector space is isomorphic to either a compression of S+ or to a restriction of S− has far-reaching implications. This leads to the use of functional models, in our case polynomial and rational models, in the study of linear transformations. The reason for the great effectiveness of the use of functional models can be traced back to the compactness of the polynomial notation as well as that the polynomial setting provides a richer algebraic language, with terms as zeros, factorizations, ideals, homomorphisms, and modules conveniently at hand. Of the two classes of models we employ, the polynomial models emphasize the arithmetic properties of the transformation, whereas the module structure of rational models is completely determined by the geometry of the model space. In other words, in the case of polynomial models, the space is simple but the tranformation is complicated. On the other hand, for rational models, the space is complicated but the transformation, i.e., the restricted backward shift, is simple. The material in this chapter is mostly standard. The definitions given in Equations (5.71) and (5.75) have far reaching implications. They can be generalized, replacing zI − A by an arbitrary nonsingular polynomial matrix, thus leading to the theory of polynomial models, initiated in Fuhrmann (1976). This is the algebraic counterpart of the functional models used so effectively in operator theory, e.g., Sz.-Nagy and Foias (1970). The results and methods presented in this chapter originate in Fuhrmann (1976) and a long series of follow-up articles. In an analytic, infinite-dimensional setting, Nikolskii (1985) is a very detailed study of shift operators. The central results of this chapter are two theorems. Theorem 5.17 characterizes the maps intertwining the shifts of the form Sq , while Theorem 5.18 studies their invertibility properties in terms of factorizations and polynomial coprimeness. These results generalize to the use of shifts of higher multiplicity, a generalization that requires the use of the algebra of polynomial matrices, which is beyond the scope of the present book. The analytic analogues of these result are the commutant lifting theorem, see Sarason (1967) and Sz.-Nagy and Foias (1970), and a corresponding spectral mapping theorem; see Fuhrmann (1968a,b). We shall return to these topics in Chapter 11. The kernel representation (5.37) of rational models has far-reaching generalizations. With the scalar polynomial replaced by a rectangular polynomial matrix, it is the basic characterizations of behaviors in the behavioral approach to linear systems initiated in Willems (1986, 1989, 1991). In this connection, see also Fuhrmann (2002).

134

5 The Shift Operator

Theorem 5.17 can be extended to the characterization of intertwining maps between two polynomial, or rational, models. We will return to this in Chapter 8, where we give a different proof using tensor products and Bezoutians. Theorem 5.18 highlights the importance of coprimenes and the Bezout equation. Since the Bezout equation can be solved using the Euclidean algorithm, this brings up the possibility of a recursive approach to inversion algorithms for structured matrices. This will be further explored in Chapter 8. The Bezout equation reappears, this time over the ring RH∞ + , in Chapters 11 and 12. Companion matrices of the polynomial q(z) appear in Krull’s thesis. The particularly musical notation for the matrices Cq ,Cq , defined in Proposition 5.4, was introduced by Kalman. Section 5.3 on circulant matrices is based on results taken from Davis (1979). The Chinese remainder theorem has its roots in the problem of solving simultaneous congruences arising in modular arithmetic. It appears in a third-century mathematical treatise by Sun Tzu. For a very interesting account of the history of early non-European mathematics, see Joseph (2000). In working over the ring of polynomials F[z] rather than over the integes Z, it can be applied to interpolation problems. The possibility of applying the Chinese remainder theorem to interpolation problems is not mentioned in many of the classic algebra texts such as van der Waerden, Lang, Mac Lane and Birkhoff or even Gantmacher (1959). Of these monographs, van der Waerden (1931) stands out inasmuch as it discusses both the Lagrange and the Newton interpolation formulas. In connection to interpolation, the following quotation, from Schoenberg (1987), may be of interest: “Sometime in the 1950’s the late Hungarian-Swedish mathematican Marcel Riesz visited the University of Pennsylvania and told us informally that the Chinese remainder theorem (1) can be thought of as an analogue of the interpolation by polynomials.” Cayley, in a letter to Sylvester, actually stated a more general version of the Cayley–Hamilton theorem. That version says that if the square matrices A and B commute and f (x, y) = det(xA − yB), then f (B, A) = 0. This was generalized in Livsic (1983).

Chapter 6

Structure Theory of Linear Transformations

6.1 Introduction In this chapter, we study the structure of linear transformations. Our aim is to understand the structure of a linear transformation in terms of its most elementary components. This we do by representing the transformation by its matrix representation with respect to particularly appropriate bases. The ultimate goal, not always achievable, is to represent a linear transformation in diagonal form.

6.2 Cyclic Transformations We begin our study of linear transformations in a finite-dimensional linear space by studying a special subclass, namely the class of cyclic linear transformations, introduced in Definition 5.13. Later on, we will show that every linear transformation is isomorphic to a direct sum of cyclic transformations. Thus the structure theory will be complete. Proposition 6.1. Let A be a cyclic linear transformation in an n-dimensional vector space U . Then its characteristic and minimal polynomials coincide. Proof. We prove the statement by contradiction. Suppose m(z) = zk + mk−1 zk−1 + · · · + m0 is the minimal polynomial of A, and assume k < n. Then Ak = −m0 I − · · · − +mk−1 Ak−1 . This shows that given an arbitrary vector x ∈ U , we have Ak x ∈ span (x, Ax, . . . , Ak−1 x). By induction, it follows that for any integer p we have span (x, Ax, . . . , A p x) ⊂ span (x, Ax, . . . , Ak−1 x). Since dim span (x, Ax, . . . , Ak−1 x) ≤ k < n, it follows that A is not cyclic.  Later on, we will prove that the coincidence of the characteristic and minimal polynomials implies cyclicity.

P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 6, © Springer Science+Business Media, LLC 2012

135

136

6 Structure Theory of Linear Transformations

Given a linear transformation T in a vector space U , its commutant is defined as the set C (T ) = {X ∈ L(U )|XT = T X}. The next result is a characterization of the commutant of a cyclic transformation. Proposition 6.2. Let T : U −→ U be a cyclic linear transformation. Then X : U −→ U commutes with T if and only if X = p(T ) for some polynomial p(z). Proof. Let p(z) ∈ F[z]. Then clearly, T p(T ) = p(T )T . Conversely, let X commute with T , where T is cyclic. Let x ∈ U be a cyclic vector, i.e., x, T x, . . . , T n−1 x form a basis for U . Thus there exist pi ∈ F for which Xx =

n−1

∑ pi T i x.

i=0

This implies XT x = T Xx = T = ∑n−1 i=0 i ∑n−1 i=0 pi z .

XT k x

and by induction, p(T ), where p(z) =

pi

n−1

n−1

i=0

i=0

∑ pi T i x = ∑ pi T i T x,

T i T k x.

Since the T k x span U , this implies X = 

Next, we show that the class of shift transformations Sm is not that special. In fact, every cyclic linear transformation in a finite-dimensional vector space is isomorphic to a unique transformation of this class. Theorem 6.3. Let V be a finite-dimensional linear space and let A : V −→ V be a cyclic transformation, with b ∈ V a cyclic vector. We consider V an F[z]-module, with the module structure induced by A. Then 1. Define a map Φ : F[z] −→ V , by k

k

i=0

i=0

Φ ∑ fi zi = ∑ fi Ai b.

(6.1)

Then the following diagram commutes: F[z]

Φ

-

S+ ? F[z]

V A

Φ

-

? V

6.2 Cyclic Transformations

137

i.e., we have

Φ S+ = AΦ ,

(6.2)

which shows that Φ is an F[z]-homomorphism. 2. The map Φ is surjective and Ker Φ is a submodule of F[z], i.e., an S+ -invariant subspace. 3. There exists a unique monic polynomial m(z) ∈ F[z], with degm = dim V , for which Ker Φ = mF[z]. (6.3) 4. Let φ : F[z]/mF[z] −→ V be the map induced by Φ on the quotient module, i.e.,

φ [ f ]m = Φ f .

(6.4)

Let π : F[z] −→ F[z]/mF[z] be the canonical projection map defined by

π f = [ f ]m ,

(6.5)

where [ f ]m = f (z) + m(z)F[z]. Then φ π = Φ , and the map φ : F[z]/mF[z] −→ V is an isomorphism. Identifying the quotient module F[z]/mF[z] with the polynomial model Xm , we have the F[z]-module isomorphism φ : Xm −→ V given by

φ f = Φ f, i.e., φ = ΦXm . 5. We have

f (z) ∈ Xm ,

φ S m = Aφ ,

(6.7)

i.e., the following diagram is commutative:

φ

Xm

-

Sm

V A

? Xm

φ

-

? V

Proof. 1. We compute k

k

k

i=0

i=0

i=0

Φ (z f ) = Φ ∑ fi zi+1 = ∑ fi Ai+1 b = A ∑ fi Ai b = AΦ f . This proves (6.2).

(6.6)

138

6 Structure Theory of Linear Transformations

2. Since {b, Ab, . . ., An−1 b} is a basis of V , the map Φ is clearly surjective. We show that Ker Φ is an ideal, or equivalently, a submodule in F[z]. Assume f (z), g(z) ∈ Ker Φ . Then Φ (α f + β g) = αΦ ( f ) + β Φ (g) = 0, so α f + β g ∈ Ker Φ . Similarly, if f ∈ Ker Φ , then

Φ (z f ) = AΦ ( f ) = A0 = 0, which shows that Ker φ is indeed an ideal. In fact, as is easily seen, a non-trivial ideal. That Ker Φ is a submodule of F[z] follows from (6.2). 3. Follows from the fact that F[z] is a principal ideal domain; hence there exists a unique monic polynomial for which Ker Φ = mF[z]. 4. Follows from Proposition 1.55. 5. We compute Aφ f = Aφ πm f = AΦ f = Φ z f = φ πm z f = φ Sm f .



From this theorem it follows that the study of an arbitrary cyclic transformation reduces to the study of the class of model transformations of the form Sm . By Theorem 5.23, the polynomial and rational models that correspond to the same polynomial are isomorphic. This indicates that Theorem 6.3 can be dualized in order to represent linear transformations by S− restricted to finite-dimensional backward-shift-invariant subspaces. This leads to the following direct proof of Rota’s theorem restricted to cyclic transformations. Theorem 6.4. Let A be a cyclic linear transformation in V and let b ∈ V be a cyclic vector for A. Then 1. Let Φ , π and φ be defined by (6.1), (6.5) and (6.4) respectively. Then a. The adjoint map Φ ∗ : V ∗ −→ z−1 F[[z−1 ]] is given by ∞

(b, A∗(i−1)η ) zi i=1

Φ ∗η = ∑

(6.8)

and is injective. b. The adjoint map π ∗ : X m −→ z−1 F[[z−1 ]] is given by

π ∗ h = h,

(6.9)

i.e., it is the injection of X m into F[[z−1 ]]. c. The adjoint map φ ∗ : V ∗ −→ X m is given, for η ∈ V ∗ , by ∞

(b, A∗(i−1) η ) . zi i=1

φ ∗η = ∑

(6.10)

6.2 Cyclic Transformations

139

d. The dual of the diagram in Theorem 6.3 is given by

Φ∗

z−1 F[[z−1 ]] 

V∗

6

6 A∗

S−

Φ∗

z−1 F[[z−1 ]] 

V∗

i.e., we have

Φ ∗ A∗ = S − Φ ∗ .

(6.11)

e. Im Φ ∗ is a finite-dimensional, S− -invariant subspace of z−1 F[[z−1 ]]. f. We have the following isomorphism: S − φ ∗ = φ ∗ A∗ .

(6.12)

2. Let V be a finite-dimensional linear space and let A : V −→ V be a cyclic transformation, with b ∈ V a cyclic vector. Then A is isomorphic to S− restricted to a finite-dimensional S− -invariant subspace of z−1 F[[z−1 ]]. Specifically, let Ψ : V −→ z−1 F[[z−1 ]] be defined by ∞

(b, Ai−1 ξ ) . zi i=1

Ψξ = ∑

(6.13)

Then the following diagram is commutative:

Ψ

V

- ImΨ

S− |Im Ψ

A ? V

Ψ

? - ImΨ

This implies the isomorphism A  S− |Im Ψ .

(6.14)

140

6 Structure Theory of Linear Transformations

Proof. 1. a. We compute (Φ f , η ) = (Φ ∑ki=0 fi zi , η ) = (∑ki=0 fi Ai b, η ) = ∑ki=0 fi (Ai b, η ) = ∑ki=0 fi (b, (A∗ )i η ) = [ f , Φ ∗ η ], which implies (6.8). To show injectivity of Φ ∗ , assume η ∈ Ker Φ ∗ . This means that for all i ≥ 0, we have (b, (A∗ )i η ) = (Ai b, η ) = 0. Since the Ai b span V , necessarily η = 0, which proves injectivity. b. We have made the identification (F[z]/mF[z])∗ = (mF[z])⊥ = X m . For f ∈ F[z] and h ∈ X m , we compute [π f , h] = [[ f ]m , h] = [ f , h] = [ f , π ∗ h], from which (6.9) follows. c. For f ∈ F[z] and η ∈ V ∗ , we compute (φ [ f ]m , η ) = (Φ f , η ) = ( f , Φ ∗ ) = ([ f ]m , φ ∗ ), i.e., φ ∗ = Φ ∗ . Using (6.8), (6.10) follows. d. Follows by dualizing (6.7) and using the representation (6.8). e. Follows from the fact that V ∗ is finite-dimensional. f. Follows from (6.11) and the equality φ ∗ = Φ ∗ . 2. Follows from the representation of the adjoints, computed in the previous part, taken together with F[z]∗  z−1 F[[z−1 ]]. Identity (6.11) is obtained by dualizing (6.2). 

6.2.1 Canonical Forms for Cyclic Transformations In this section we exhibit canonical forms for cyclic transformations. Theorem 6.5. Let p(z) ∈ F[z] and let p(z) = p1 (z)m1 · · · pk (z)mk be the primary decomposition of p(z). We define

πi (z) = ∏ p j (z)m j .

(6.15)

j =i

Then we have the direct sum decomposition X p = π1 (z)X pm1 ⊕ · · · ⊕ πk (z)X pmk 1

k

(6.16)

6.2 Cyclic Transformations

141

and the associated spectral representation S p  S p m1 ⊕ · · · ⊕ S p mk . 1

(6.17)

k

Proof. The direct sum decomposition (6.16) was proved in Theorem 2.23. We define a map Z : X pm1 ⊕ · · · ⊕ X pmk −→ X p = π1 (z)X pm1 ⊕ · · · ⊕ πk (z)X pmk 1

1

k

k

by k

Z( f 1 , . . . , fk ) = ∑ πi fi .

(6.18)

i=1

Clearly, Z is invertible, and it is easily checked that the following diagram is commutative: X p m1 ⊕ · · · ⊕ X p mk 1

Z

k

- Xp

S p m1 ⊕ · · · ⊕ S p mk 1

Sp

k

? X p m1 ⊕ · · · ⊕ X p mk 1

Z

-

k

which shows that Z is an F[z]-isomorphism.

? Xp



Thus, by choosing appropriate bases in the spaces X pmi , we get a block matrix i representation for S p . It suffices to consider the matrix representation of a typical operator S pm , where p(z) is an irreducible polynomial. This is the content of the next proposition. Proposition 6.6. Let p(z) = zr + pr−1 zr−1 + · · · + p0 . Then 1. dim X pm = mr.

(6.19)

2. The set of polynomials B jo = {1, z, . . . , zr−1 , p, zp, . . . , zr−1 p, . . . , pm−1 , zpm−1 , . . . , zr−1 pm−1 } (6.20) forms a basis for X pm . We call this the Jordan basis.

142

6 Structure Theory of Linear Transformations

3. The matrix representation of S pm with respect to this basis is of the form ⎛

⎞ Cp ⎜ N C ⎟ p ⎜ ⎟ ⎜ ⎟ N . ⎜ ⎟ jo [S pm ] jo = ⎜ ⎟, ⎜ ⎟ .. ⎜ ⎟ ⎝ ⎠ . .  N Cp

(6.21)

where Cp is the companion matrix of p(z), defined by (5.9), and ⎞ 0...01 ⎜. 0⎟ ⎟ ⎜ ⎟ ⎜ .⎟ ⎜. N=⎜ ⎟. ⎜. .⎟ ⎟ ⎜ ⎝. .⎠ 0... . 0 ⎛

(6.22)

We refer to (6.21) as the Jordan canonical form of S pm . Proof. 1. We have dim X pm = deg pm = m deg p = mr. 2. In B there is one polynomial of each degree and mr polynomials altogether, so necessarily they form a basis. 3. We prove this by simple computations. Notice that for j < m − 1, S pm zr−1 p j = π pm z · zr−1 p j = π pm zr p j = π pm (zr + pr−1zr−1 + · · · + p0 − pr−1 zr−1 − · · · − p0 )p j i j = π pm p j+1 − ∑r−1 i=0 pi (z p ). On the other hand, for j = m − 1, we have S pm zr−1 pm−1 = π pm zr pm−1 = π pm (zr + pr−1zr−1 + · · · + p0 − pr−1 zr−1 − · · · − p0 )pm−1 i m−1 = π pm pm − ∑r−1 ) i=0 pi (z p r−1 i m−1 ). = − ∑i=0 pi (z p  It is of interest to compute the dual basis to the Jordan basis. This leads us to the Jordan form in the block upper triangular form. Proposition 6.7. Given the monic polynomial p(z) of degree r, let {e1 (z), . . . , er (z)} be the control basis of X p . Then

6.2 Cyclic Transformations

143

1. The basis of X pm dual to the Jordan basis of (6.20) is given by  jo = {pm−1 (z)e1 (z), . . . , pm−1 (z)er (z), . . . , e1 (z), . . . , er (z)}. B

(6.23)

Here e1 (z), . . . , er (z) are the elements of the control basis of X p . We call this basis the dual Jordan basis. 2. The matrix representation of Sqm with respect to the dual Jordan basis is given by ⎞ Cp N˜ ⎟ ⎜ C . ⎟ ⎜ p ⎟ ⎜  . . ⎟ ⎜ [Sqm ] jo = ⎟. ⎜  jo ⎟ ⎜ .. ⎟ ⎜ ˜ ⎝ . N⎠ Cp ⎛

(6.24)

Note that Cp is defined in (5.10). 3. The change of basis transformation from the dual Jordan basis to the Jordan basis is given by ⎛ ⎞ K ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ jo [I]  = ⎜ . ⎟, jo ⎜ ⎟ ⎝ . ⎠ K ⎞ ⎛ p1 . . pr−1 1 ⎟ ⎜ . . ⎟ ⎜ ⎟ ⎜ st where K = [I]co = ⎜ . . ⎟. ⎟ ⎜ ⎠ ⎝ pr−1 . 1 4. The Jordan matrices in (6.21) and (6.24) are similar. A similarity is given by R = [I] jo . jo

Proof. 1. This is an extension of Proposition 5.40. We compute, with 0 ≤ α , β < m and i, j < r, pα ei , pβ z j−1 = [p−m pα π+ z−i p, pβ z j−1 ] = [π+ z−i p, pα +β −m z j−1 ]. The last expression is clearly zero whenever α + β = m−1. When α + β = m−1, we have [π+ z−i p, pα +β −m z j−1 ] = [π+ z−i p, p−1 z j−1 ] = [z−i p, p−1 z j−1 ] = [z j−i−1 , 1] = δi j .

144

6 Structure Theory of Linear Transformations

2. Follows either by direct computation or the fact that Sq∗ = Sq coupled with an application of Theorem 4.38. 3. This results from a simple computation. 

jo jo jo 4. From S pm I = IS pm we get [S pm ] jo jo [I]  = [I]  [S pm ]  . jo



jo

jo

Note that if the polynomial p(z) is monic of degree 1, i.e., p(z) = z − α , then with respect to the basis B = {1, z − α , . . . , (z − α )n−1 } the matrix representation has the following form: ⎞ α ⎟ ⎜1 α ⎟ ⎜ ⎟ ⎜ 1 . ⎟ ⎜ [S(z−α )n ]B ⎟. B =⎜ ⎟ ⎜ .. ⎟ ⎜ ⎝ . . ⎠ 1α ⎛

If our field F is algebraically closed, as is the field of complex numbers, then every irreducible polynomial is of degree 1. In case we work over the real field, the irreducible monic polynomials are either linear, or of the form (z − α )2 + β 2 , with α , β ∈ R. In this case, we make a variation on the canonical form obtained in Proposition 6.6. This leads to the real Jordan form. Proposition 6.8. Let p(z) be the real polynomial p(z) = (z − α )2 + β 2 . Then 1. We have dim X pn = 2n.

(6.25)

2. The set of polynomials B = {β , z − α , β p(z), (z − α )p(z), . . . , β pn−1 (z), (z − α )pn−1 (z)} forms a basis for X pn . 3. The matrix representation of S pn with respect to this basis is given by ⎞ A ⎟ ⎜N A ⎟ ⎜ ⎟ ⎜ N. ⎟ ⎜ jo [S pn ] jo = ⎜ ⎟, ⎟ ⎜ .. ⎟ ⎜ ⎝ . . ⎠ NA ⎛

where

A=

α −β β α



,

N=

0 β −1 0 0

(6.26)

.

Proof. The proof is analogous to that of Proposition 6.6 and is omitted.



6.3 The Invariant Factor Algorithm

145

6.3 The Invariant Factor Algorithm The invariant factor algorithm is the tool that allows us to move from the special case of cyclic operators to the general case. We begin by introducing an equivalence relation in the space F[z]m×n , i.e., the space of all m × n matrices over the ring F[z]. We will use the identification of F[z]m×n with Fm×n [z], the space of matrix polynomials with coefficients in Fm×n . Thus given a matrix A ∈ Fm×m , then zI − A is a matrix polynomial of degree one, but usually we will consider it also as a polynomial matrix. Definition 6.9. A polynomial matrix U(z) ∈ F[z]m×m is called unimodular if it is invertible in F[z]m×m , that is, there exists a matrix V (z) ∈ F[z]m×m for which U(z)V (z) = V (z)U(z) = I.

(6.27)

Lemma 6.10. U(z) ∈ F[z]m×m is unimodular if and only if detU(z) = α ∈ F and α = 0. Proof. Assume U(z) is unimodular. Then there exists V (z) ∈ F[z]m×m for which U(z)V (z) = V (z)U(z) = I. By the multiplicative rule of determinants we have detU(z) detV (z) = 1. This implies that both detU(z) and detV (z) are nonzero scalars. Conversely, if detU(z) is a nonzero scalar, then, using (3.9), we conclude that U(z)−1 is a polynomial matrix.  Definition 6.11. Two matrices A, B ∈ F[z]m×n are called equivalent if there exist unimodular matrices U(z) ∈ F[z]m×m and V (z) ∈ F[z]n×n such that U(z)A(z) = B(z)V (z) = I. It is easily checked that this definition indeed yields an equivalence relation. The next theorem leads to a canonical form for this equivalence. However, we find it convenient to prove first the following lemma. Lemma 6.12. Let A(z) ∈ F[z]m×n . Then A(z) is equivalent to a block matrix of the form ⎛ ⎞ a(z) 0 . . . 0 ⎜ . ⎟ ⎜ ⎟ ⎜ ⎟ B(z) ⎜ . ⎟, ⎜ ⎟ ⎝ . ⎠ 0 with a(z) | bi j (z). Proof. If A(z) is the zero matrix, there is nothing to prove. Otherwise, interchanging rows and columns as necessary, we can bring a nonzero entry of least degree

146

6 Structure Theory of Linear Transformations

to the upper left-hand corner. We use the division rule of polynomials to write each element of the first row in the form a1 j (z) = c1 j (z)a11 (z) + a1 j (z), with deg a1 j < deg a11 . Next, we subtract the first column, multiplied by c1 j (z), from the jth column. We repeat the process with the first column. If all the remainders are zero we check whether a11 (z) divides all other entries in the matrix. If it does, we are through. Otherwise, we bring, using column and row interchanges, a lowest-degree nonzero element to the upper left-hand corner and repeat the process.  Theorem 6.13 (The invariant factor algorithm). Let A(z) ∈ F[z]m×n . Then 1. A(z) is equivalent to a diagonal polynomial matrix with the diagonal elements di (z) monic polynomials satisfying di (z) | di−1 (z). 2. The polynomials di (z) are uniquely determined and are called the invariant factors of A(z). Proof. We use the previous lemma inductively. Thus, by elementary operations, A is reducible to diagonal form with the diagonal entries di (z). If d1 (z) does not divide di (z), then we add the ith row to the first. Then, by an elementary column operation, reduce di (z) modulo d1 (z) and reapply Lemma 6.12. We repeat the process as needed, till d1 (z) | di (z) for i > 1. We proceed by induction to get di (z) | di+1 (z). This is the opposite ordering from that in the statement of the theorem. We can get the right ordering by column and row interchanges.  The polynomial matrix D(z) having nontrivial invariant factors on the diagonal will be called the Smith form of A(z). Proposition 6.14. A polynomial matrix U(z) is unimodular if and only if it is the product of a finite number of elementary unimodular matrices. Proof. The product of unimodular matrices is, by the multiplicative rule for determinants, also unimodular. Conversely, assume U(z) is unimodular. By elementary row and column operations, it can be brought to its Smith form D(z), with the polynomials di (z) on the diagonal. Since D(z) is also unimodular, D(z) = I. Let now Ui (z) and V j (z) be the elementary unimodular matrices representing the elementary operations in the diagonalization process. Thus Uk (z) · · ·U1 (z)U(z)V1 (z) · · ·Vl (z) = I, which implies U(z) = U1 (z)−1 · · ·Uk (z)−1Vl (z)−1 · · ·V1 (z)−1 . This is a representation of U(z) as the product of elementary unimodular matrices.  To prove the uniqueness of the invariant factors, we introduce the determinantal divisors. Definition 6.15. Let A(z) ∈ F[z]m×n . Then we define D0 (A) = 1 and Dk (A) to be the g.c.d., taken to be monic, of all k × k minors of A(z). The Di (A) are called the determinantal divisors of A(z). Proposition 6.16. Let A(z), B(z) ∈ F[z]m×n . If B(z) is obtained from A(z) by a series of elementary row or column operations, then Dk (A) = Dk (B).

6.4 Noncyclic Transformations

147

Proof. We consider the set of all k × k minors of A(z). We consider the effect of applying elementary transformations on A(z) on this set. Multiplication of a row or column by a nonzero scalar α can change a minor by at most a factor α . This does not change the g.c.d. Interchanging two rows or columns leaves the set of minors unchanged, except for a possible sign. The only elementary operation that has a nontrivial effect is adding the jth row, multiplied by a polynomial p(z), to the ith row. In a minor that does not contain the ith row, or contains elements of both the ith and the jth rows, this has no effect. If a minor contains elements of the ith row but not of the jth, we expand by the ith row. We get the original minor added to the product of another minor by the polynomial p(z). Again, this does not change the g.c.d.  Theorem 6.17. Let A(z), B(z) ∈ F[z]m×n . Then A(z) and B(z) are equivalent if and only if Dk (A) = Dk (B). Proof. It is clear from the previous proposition that if a single elementary operation does not change the set of elementary divisors, then also a finite sequence of elementary operation leaves the determinantal divisors unchanged. Since any unimodular matrix is the product of elementary unimodular matrices, we conclude that the equivalence of A(z) and B(z) implies Dk (A) = Dk (B). Assume now that Dk (A) = Dk (B) for all k. Using the invariant factor algorithm, we reduce A(z) and B(z) to their Smith form with the invariant factors d1 (z), . . . , dr (z), ordered by di−1 (z) | di (z), and e1 (z), . . . , es (z), similarly ordered, respectively. Clearly Dk (A) = d1 (z) · · · dk (z) and Dk (B) = e1 (z) · · · ek (z). This implies r = s and, assuming the invariant factors are all monic, ei (z) = di (z). By transitivity of equivalence we get A(z)  B(z).  Corollary 6.18. The invariant factors of A(z) ∈ F[z]m×n are uniquely determined. We can view the invariant factor algorithm as a far-reaching generalization of the Euclidean algorithm, for the Euclidean algorithm is the invariant factor algorithm as  applied to a polynomial matrix of the form p(z) q(z) .

6.4 Noncyclic Transformations We continue now with the study of general, i.e., not necessarily cyclic, transformations in finite-dimensional vector spaces. Our aim is to reduce their study to that of the cyclic ones, and we will do so using the invariant factor algorithm. However, to use it effectively, we will have to extend the modular polynomial arithmetic to the case of vector polynomials. We will not treat the case of remainders with respect to general nonsingular polynomial matrices, since for our purposes, it will suffice to consider two special cases. The first case is the operator πzI−A , defined in (5.71)

148

6 Structure Theory of Linear Transformations

and discussed in Proposition 5.42. The second case is that of nonsingular diagonal polynomial matrices. There is a projection map, similar to the projection πd , defined in (5.2), that we shall need in the following, and which we proceed to introduce. Assume D(z) = ⎛ ⎞ ⎝

d1 (z)

..

⎠, with di (z) nonzero polynomials. Given a vector polynomial

.

⎛

d (z)

n ⎞ f1 (z) ⎜ ⎟ f (z) = ⎝ ... ⎠, we define

fn (z) ⎞ π d1 f 1 ⎟ ⎜ πD f = ⎝ ... ⎠ , ⎛

(6.28)

π dn f n where πdi fi are the remainders of fi (z) after division by di (z). Proposition 6.19. Given a nonsingular diagonal polynomial matrix D(z), then 1. We have 2.

Ker πD = DFn [z]. ⎧⎛ ⎪ ⎨ ⎜ XD = Im πD = ⎝ ⎪ ⎩

(6.29)

⎫ ⎞ f1 ⎪ ⎬ .. ⎟ | f (z) ∈ Im π di . . ⎠ i ⎪ ⎭ fn

(6.30)

In particular, XD  Xd1 ⊕ · · · ⊕ Xdn . 3. Defining the map SD : XD −→ XD by SD f = πD z f for f (z) ∈ XD , we have SD  S d 1 ⊕ · · · ⊕ S dn . Proof. 1. Clearly, f (z) ∈ Ker πD if and only if for all i, fi (z) is divisible by di (z). Thus ⎧⎛ ⎫ ⎞ ⎪ ⎪ ⎨ d1 f 1 ⎬ ⎜ .. ⎟ Ker πD = ⎝ . ⎠ | fi (z) ∈ F[z] = DFn [z]. ⎪ ⎪ ⎩ ⎭ dn f n 2. Follows from the definition of πD . 3. We compute ⎛ ⎞ f1 ⎜ ⎜ ⎟ SD ⎝ ... ⎠ = πD z ⎝ ⎛

fn

⎞ ⎛ ⎞ ⎛ ⎞ π d1 f 1 f1 S d1 f 1 .. ⎟ = ⎜ .. ⎟ = ⎜ .. ⎟ . . ⎠ ⎝ . ⎠ ⎝ . ⎠ fn

π dn f n

S dn f n



6.4 Noncyclic Transformations

149

With this background material, we can proceed to the structure theorem we have been after. It shows that an arbitrary linear transformation is isomorphic to a direct sum of cyclic transformations corresponding to the invariant factors of A. Theorem 6.20. Let A be a linear transformation in Fn . Let d1 (z), . . . , dn (z) be the invariant factors of zI − A. Then A is isomorphic to Sd1 ⊕ · · · ⊕ Sdn . Proof. Since d1 (z), . . . , dn (z) are the invariant factors of zI − A, there exist unimodular polynomial matrices U(z) and V (z) satisfying

This equation implies

U(z)(zI − A) = D(z)V (z).

(6.31)

UKer πzI−A ⊂ Ker πD .

(6.32)

Since U(z) and V (z) are invertible as polynomial matrices, we have also

which implies

U(z)−1 D(z) = (zI − A)V (z)−1 ,

(6.33)

U −1 Ker πD ⊂ Ker πzI−A .

(6.34)

We define now two maps, Φ : Fn −→ XD and Ψ : XD −→ Fn , by

Φ x = πDUx, and

x ∈ Fn ,

Ψ f = πzI−AU −1 f ,

(6.35)

f (z) ∈ XD .

(6.36)

We claim now that Φ is invertible and its inverse is Ψ . Indeed, using (6.34)-(6.36), we compute

Ψ Φ x = πzI−AU −1 πDUx = πzI−AU −1Ux = πzI−A x = x. Similarly, using (6.32), we have, for f ∈ XD ,

ΦΨ f = πDU πzI−AU −1 f = πDUU −1 f = πD f = f . Next, we show that the following diagram is commutative: A - n n F F

Φ

Φ ? XD

SD

-

? XD

150

6 Structure Theory of Linear Transformations

Indeed, for x ∈ Fn , using (6.32) and the fact that zKer πD ⊂ Ker πD , we have SD Φ x = πD z · πDUx = πD zUx = πDUzx = πDU πzI−A zx = πDU(Ax) = Φ Ax. Since Φ is an isomorphism, we have A  SD . But SD  Sd1 ⊕ · · · ⊕ Sdn , and the result follows.  Corollary 6.21. Let A be a linear transformation in an n-dimensional vector space V . Let d1 (z), . . . , dn (z) be the invariant factors of A. Then A is similar to the diagonal block matrix composed of the companion matrices corresponding to the invariant factors, i.e., ⎛  ⎞ Cd1 ⎜ ⎟ . ⎜ ⎟ ⎜ ⎟ A⎜ (6.37) ⎟. . ⎜ ⎟ ⎝ ⎠ .  Cdn Proof. By Theorem 6.20, A is isomorphic to Sd1 ⊕ · · · ⊕ Sdn acting in Xd1 ⊕ · · · ⊕ Xdn . Choosing the standard basis in each Xdi and taking the union of these basis elements, we have [Sd1 ⊕ · · · ⊕ Sdn ]stst = diag ([Sd1 ]stst , . . . , [Sdn ]stst ). We conclude by recalling that [Sdi ]stst = Cd i .



Theorem 6.22. Let A and B be linear transformations in n-dimensional vector spaces U and V respectively. Then the following statements are equivalent. 1. A and B are similar. 2. The polynomial matrices zI − A and zI − B are equivalent. 3. The invariant factors of A and B coincide. Proof. 1. Assume A and B are similar. Let X be an invertible matrix for which XA = BX. This implies X (zI − A) = (zI − B)X and so the equivalence of zI − A and zI − B. 2. The equivalence of the polynomial matrices zI − A and zI − B shows that their invariant factors coincide. 3. Assuming the invariant factors coincide, both A and B are similar to Sd1 ⊕ · · · ⊕ Sdn , and the claim follows from the transitivity of similarity.  Corollary 6.23. Let A be a linear transformation in an n-dimensional vector space U over F. Let d1 (z), . . . , dn (z) be the invariant factors of A, ordered so that di | di−1 . Then

6.5 Diagonalization

151

1. The minimal polynomial of A, mA (z), satisfies mA (z) = d1 (z). 2. dA (z), the characteristic polynomial of A, satisfies n

dA (z) = ∏ di (z).

(6.38)

i=1

Proof. 1. Since A  Sd1 ⊕ · · · ⊕ Sdn , for an arbitrary polynomial p(z) we have p(A)  p(Sd1 ) ⊕ · · · ⊕ p(Sdn ). So p(A) = 0 if and only if p(Sdi ) = 0 for all i, or equivalently, di | p. In particular, d1 | p. But d1 (z) is the minimal polynomial of Sd1 , so it is also the minimal polynomial of Sd1 ⊕ · · · ⊕ Sdn and, by similarity, also of A. 2. Using Proposition 6.20 and the matrix representation (6.37), (6.38) follows.  Next we give a characterization of cyclicity in terms of the characteristic and minimal polynomials. Proposition 6.24. A transformation T in U is cyclic if and only if its characteristic and minimal polynomials coincide. Proof. It is obvious that if T is cyclic, its characteristic and minimal polynomials have to coincide. Conversely, if the characteristic and minimal polynomials coincide, then the only nontrivial invariant factor d(z) is equal to the characteristic polynomial. By Theorem 6.20 we have T  Sd , and since Sd is cyclic, so is T . 

6.5 Diagonalization The maximal simplification one can hope to obtain for the structure of an arbitrary linear transformation in a finite-dimensional vector space is diagonalizability. Unhappily, that is too much to ask. The following theorem gives a complete characterization of diagonalizability. Theorem 6.25. Let T be a linear transformation in a finite-dimensional vector space U . Then T is diagonalizable if and only if mT (z), the minimal polynomial of T , splits into distinct linear factors. Proof. This follows from Proposition 5.15, taken in conjunction with Theorem 6.20. However, we find it of interest to give a direct proof. That the existence of a diagonal representation implies that mT (z), the minimal polynomial of T , splits into distinct linear factors is obvious.

152

6 Structure Theory of Linear Transformations

Assume now that the minimal polynomial splits into distinct linear factors. So k (z − λ ), where the λ are distinct. Let π (z) be the corresponding mT (z) = Πi=1 i i i Lagrange interpolation polynomials. Clearly (z − λi )πi (z) = mT (z). Since for i = j, (z − λi ) | π j (z), we get mT (z) | πi (z)π j (z). (6.39) Similarly, the polynomial πi (z)2 − πi (z) vanishes at all the points λi , i = 1, . . . , k. So mT (z) | πi (z)2 − πi (z).

(6.40)

The two division relations imply now

πi (T )π j (T ) = 0, and

i = j,

πi (T )2 = πi (T ).

(6.41)

(6.42)

It follows that the π (T ) are projections on independent subspaces. We set Ui = Im πi (T ). Then, since the equality 1 = ∑ki=1 πi (z) implies I = ∑ki=1 πi (T ), we get the direct sum decomposition U = U1 ⊕ · · · ⊕ Uk . Moreover, since T πi (T ) = πi (T )T , the subspaces Ui are T -invariant. From the equality (z − λi )πi (z) = mT (z) we get (T − λi I)πi (T ) = 0, or Ui ⊂ Ker (λi I − T ). This can be restated as T |Ui = λi IUi . Choosing bases in the subspaces Ui and taking their union leads to a basis of U made up of eigenvectors. With respect to this basis, T has a diagonal matrix representation.  The following is the basic structure theorem, yielding a spectral decomposition for a linear transformation. The result follows from our previous considerations and can be read off from the Jordan form. Still, we find it of interest to give also another proof. Theorem 6.26. Let T be a linear transformation in a finite-dimensional vector space U . Let mT (z) be the minimal polynomial of T and let mT (z) = p1 (z)ν1 · · · pk (z)νk be the primary decomposition of mT (z). We set Ui = Ker pi (T )νi , i = 1, . . . , k, and define polynomials by πi (z) = Π j =i pνi i (z). Then 1. There exists a unique representation 1=

k

∑ π j (z)a j (z),

(6.43)

j=1

with a j ∈ X 2. We have

νj

pj

. mT | πi π j ,

for i = j,

(6.44)

and mT | πi2 a2i − πi ai .

(6.45)

6.5 Diagonalization

153

3. The operators Pi = πi (T )ai (T ), i = 1, . . . , k, are projection operators satisfying Pi Pj I T Pi Im Pi

= δi j Pj , = ∑ki=1 Pi , = Pi T Pi , = Ker pνi i (T ).

4. Define Ui = Im Pi . Then U = U1 ⊕ · · · ⊕ Uk . 5. The subspaces Ui are T -invariant. 6. Let Ti = T |Ui . Then the minimal polynomial of Ti is pi (z)νi . Proof. 1. Clearly, the πi (z) are relatively prime and pi (z)νi πi (z) = mT (z). By the Chinese remainder theorem, there exists a unique representation of the form 1 = ∑kj=1 π j (z)a j (z), with a j (z) ∈ X ν j . The a j (z) can be computed as follows. Since pj

for i = j we have π pνi π j f = 0, we have i

1 = π pνi i

k

∑ π j a j = π piνi πi ai = πi (S piνi )ai ,

j=1

and hence ai = πi (S pνi )−1 1. That the operator πi (S pνi ) : X pνi −→ X pνi is invertible i

i

i

i

follows, by Theorem 5.18, from the coprimeness of πi (z) and pνi i (z). 2. To see (6.44), note that if j = i, pνi i | π j , and hence mT (z) is a factor of πi (z)π j (z). Naturally, we get also mT | πi ai π j a j . To see (6.45), it suffices to show that for all j, pνi i | π 2j a2j − π j a j , or equivalently, that π 2j a2j and π j a j have the same remainder after division by pνi i . For j = i this is trivial, since pνi i |π j . So we assume j = i and compute

π pνi (πi2 a2i ) = π pνi (πi ai )π pνi πi ai = 1, i

3.

4. 5. 6.

i

i

for π pνi πi ai = πi (S pνi )ai = 1. i i Properties (6.44) and (6.45) have now the following consequences. For i = j we have πi (T )ai (T )π j (T )a j (T ) = 0, and (πi (T )ai (T ))2 = πi (T )ai (T ). This implies that Pi = πi (T )ai (T ) are commuting projections on independent subspaces. Set now Ui = Im Pi = Im πi (T )ai (T ). Equation (6.43) implies I = ∑ki=1 Pi , and hence we get the direct sum decomposition U = U1 ⊕ · · · ⊕ Uk . Since Pi commutes with T , the subspaces Ui actually reduce T . Let Ti = T |Ui . Since pνi i πi = mT , it follows that for all x ∈ U , pνi i (T )Pi x = pνi i (T )πi (T )ai (T )x = 0, i.e., the minimal polynomial of Ti divides pνi i . To see the converse, let q(z) be any polynomial such that q(Ti ) = 0. Then q(T )πi (T ) = 0. It follows that q(z)πi (z) is divisible by the minimal polynomial of T . But that means that pνi i πi | qπi or pνi i | q. We conclude that the minimal polynomial of Ti is pi (z)νi . 

154

6 Structure Theory of Linear Transformations

6.6 Exercises 1. Let T (z) be an n × n nonsingular polynomial matrix. Show that there exists a unimodular matrix P(z) such that P(z)T (z) has the upper triangular form ⎛

t11 (z) ⎜ 0 ⎜ ⎜ ⎜ . ⎜ ⎝ . 0

⎞ . . . t1n (z) .. . . ⎟ ⎟ ⎟ .. . . ⎟ ⎟ .. . . ⎠ . . 0 tnn (z)

with tii (z) = 0 and deg(t ji ) < deg(tii ). 2. Let T be a linear operator on F2 . Prove that any nonzero vector that is not an eigenvector of T is a cyclic vector for T . Prove that T is either cyclic or a multiple of the identity. 3. Let T be a diagonalizable operator in an n-dimensional vector space. Show that if T has a cyclic vector, then T has n distinct eigenvalues. If T has n distinct eigenvalues, and if x1 , . . . , xn is a basis of eigenvectors, show that x = x1 + · · · + xn is a cyclic vector for T . 4. Let T be a linear transformation in the finite-dimensional vector space V . Prove that ImT has a complementary invariant subspace if and only if Im T and Ker T are independent subspaces. Show that if Im T and Ker T are independent subspaces, then Ker T is the unique complementary invariant subspace for Im T . 5. How many possible Jordan forms are there for a 6 × 6 complex matrix with characteristic polynomial (z + 2)4(z − 1)2 . 6. Given a linear transformation A in an n-dimensional space V , let d(z) and m(z) be its characteristic and minimal polynomials respectively. Show that d(z) divides mn (z). 7. Let p(z) = p0 + p1 z + · · · + pk−1 zk−1 . Solve the equation

πzk qp = 1. 8. Let A =

a c bd

be a complex matrix. Apply the Gram-Schmidt process to the

columns of A. Show that this results in a basis for C2 if and only if detA = 0. 9. Let A be an n×n matrix over the field F. We say that A is diagonalizable if there exists a matrix representation of A that is diagonal. Show that diagonalizable and cyclic are independent concepts, i.e., show the existence of matrices that are DC, DC, DC, DC. (Here DC means nondiagonalizable and cyclic.) 10. Let A be an n × n Jordan block matrix, i.e.,

6.6 Exercises

155

⎛

⎞ λ 1 ⎜ ⎟ ⎜ . . ⎟ ⎜ ⎟ A=⎜ . . ⎟, ⎜ ⎟ ⎝ . 1⎠ λ and

⎛

⎞ 01 ⎜ . . ⎟ ⎜ ⎟ ⎜ ⎟ N=⎜ . . ⎟, ⎜ ⎟ ⎝ . 1⎠ 0

i.e., A = λ I + N. Let p be a polynomial of degree r. Show that p(A) =

n−1

1

∑ k! p(k) (λ )N k .

k=0

11. Show that if A is diagonalizable and p(z) ∈ F[z], then p(A) is diagonalizable. 12. Let p(z) ∈ F[z] be of degree n. a. Show that the dimension of the smallest invariant subspace of X p that contains f (z) is n − r, where r = deg( f ∧ p). b. Conclude that f (z) is a cyclic vector of S p (z) if and only if f (z) and p(z) are coprime. 13. Let A : V −→ V and A1 : V1 −→ V1 be linear transformations with minimal polynomials m and m1 respectively. a. Show that if there exists a surjective (onto) map Z : V −→ V1 satisfying ZA = A1 Z, then m1 | m. (Is the converse true? Under what conditions?) b. Show that if there exists an injective (1 to 1) map Z : V1 −→ V satisfying ZA1 = AZ, then m1 | m. (Is the converse true? Under what conditions?) c. Show that the same is true with the minimal polynomials replaced by the characteristic polynomials. 14. Let A : V −→ V be a linear transformation and M ⊂ V an invariant subspace of A. Show that the minimal and characteristic polynomials of A|M divide the minimal and characteristic polynomials of A respectively. 15. Let A : V −→ V be a linear transformation and M ⊂ V an invariant subspace of A. Let d1 , . . . , dn be the invariant factors of A and e1 , . . . , em be the invariant factors of A|M . a. Show that ei | di for i = 1, . . . , m. b. Show that given arbitrary vectors v1 , . . . , vk in V , we have

156

6 Structure Theory of Linear Transformations

dim span {Ai v j | j = 1, . . . , k, i = 0, . . . , n − 1} ≤

k

∑ deg d j .

j=1

16. Let A : V −→ V and assume V = span {Ai v j | j = 1, . . . , k, i = 0, . . . , n − 1}. Show that the number of nontrivial invariant factors of A is ≤ k. 17. Let A(z) and B(z) be polynomial matrices and let C(z) = A(z)B(z). Let ai (z), bi (z), ci (z) be the invariant factors of A, B,C respectively. Show that ai | ci and bi | ci . 18. Let A and B be cyclic linear transformations. Show that if ZA = BZ and Z has rank r, then the degree of the greatest common divisor of the characteristic polynomials of A and B is at least r. 19. Show that if the minimal polynomials of A1 and A2 are coprime, then the only solution to ZA1 = A2 Z is the zero solution. 20. If there exists a rank r solution to ZA1 = A2 Z, what can you say about the characteristic polynomials of A1 and A2 ? 21. Let V be an n-dimensional linear space. Assume T : V −→ V is diagonalizable. Prove a. If T is cyclic, then T has n distinct eigenvalues. b. If T has n distinct eigenvalues, then T is cyclic. c. If b1 , . . . , bn are the eigenvectors corresponding to the distinct eigenvalues, then b = b1 + · · · + bn is a cyclic vector for T . 22. Show that if T 2 is cyclic, then T is cyclic. Is the converse true? 23. Let V be an n-dimensional linear space over the field F. Show that every nonzero vector in V is a cyclic vector of T if and only if the characteristic polynomial of T is irreducible over F. 24. Show that if m(z) is the minimal polynomial of a linear transformation A : V −→ V , then there exists a vector x ∈ V such that {p ∈ F[z] | p(A)x = 0} = mF[z]. 25. Let A : V −→ V be a linear transformation. Show that: a. For each x ∈ V , there exists a smallest A-invariant subspace Mx containing x. b. x is a cyclic vector for A|Mx . 26. Show that if A1 , A2 are cyclic maps with cyclic vectors b1 , b2 respectively and if the minimal polynomials of A1 and A2 are coprime, then b1 ⊕ b2 is a cyclic vector for A1 ⊕ A2 .

6.6 Exercises

157

27. Let A be a cyclic transformation in V and let M ⊂ V be an invariant subspace of T . Show that M has a complementary invariant subspace if and only if the characteristic polynomials of T |M and TV /M are coprime. 28. Show that if T 2 − T = 0, then T is similar to a matrix of the form diag (1, . . . , 1, 0, . . . , 0). 29. The following series of exercises, based on Axler (1995), provides an alternative approach to the structure theory of a linear transformation, which avoids the use of determinants. We do not assume the underlying field to be algebraically closed. Let T be a linear transformation in a finite-dimensional vector space X over the field F. A nonzero polynomial p(z) ∈ F[z] is called a prime of T if p(z) is monic, irreducible and there exists a nonzero vector x ∈ X for which p(T )x = 0. Such a vector x is called a p-null vector. Clearly, these notions generalize eigenvalues and eigenvectors. a. Show that every linear transformation T in a finite-dimensional vector space X has at least one prime. b. Let p(z) be a prime of T and let x be a p-null vector. If q(z) is any polynomial for which q(T )x = 0, then p(z) | q(z). In particular, if deg q < deg p and q(T )x = 0, then necessarily q(z) = 0. c. Show that nonzero pi -null vectors corresponding to distinct primes are linearly independent. d. Let p(z) be a prime of T with deg p = ν . Show that {x | p(T )k x = 0, for some k = 1, 2, . . .} = {x | p(T )[n/ν ] x = 0}. We call elements of Ker p(T )[n/ν ] generalized p-null vectors. e. Show that X is spanned by generalized p-null vectors. f. Show that generalized p-null vectors corresponding to distinct primes are linearly independent. g. Let p1 (z), . . . , pm (z) be the distinct primes of T and let Xi = Ker pi (T )[n/νi ] . Prove i. ii. iii. iv.

X has the direct sum representation X = X1 ⊕ · · · ⊕ Xm . Xi are T -invariant subspaces. The operators pi (T )|Xi are nilpotent. The operator T |Xi has only one prime, namely pi (z).

30. Given a nilpotent operator N, show that it is similar to a matrix of the form ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝

⎞

N1

⎟ ⎟ ⎟ ⎟, ⎟ ⎠

. . . Nk

158

6 Structure Theory of Linear Transformations

with Ni of the form

⎛

⎞ 01 ⎜ . . ⎟ ⎜ ⎟ ⎜ ⎟ . . ⎟. ⎜ ⎜ ⎟ ⎝ . 1⎠ 0

31. Let T be a linear operator in an n-dimensional complex vector space X . Let νi λ1 , . . . , λm be the distinct eigenvalues of T and let m(z) = ∏m i=1 (z − λi ) be its minimal polynomial. Let Hi (z) be the minimal degree solution to the Hermite (k) interpolation problem Hi (λ j ) = δ0 k δi j , i, j = 1, . . . , m, k = 0, . . . , ν j − 1. Define Pk = Hk (T ). Show that for any polynomial p(z), we have p(T ) =

m νk −1

∑∑

k=1 j=0

p( j) (λk ) (T − λk I) j Pk . j!

32. Let T be a linear operator in an n-dimensional vector space X . Let di (z) be the invariant factors of T and let δi = degdi . Show that the commutant of T has dimension equal to ∑ni=1 (2i − 1)δi . Conclude that the commutant of T is L(X) if and only if T is scalar, i.e., T = α I for some α ∈ F. Show also that the dimension of the commutant of T is equal to n if and only if T is cyclic.

6.7 Notes and Remarks The polynomial module structure induced by a linear transformation acting in a vector space is what in functional analysis is called a functional calculus; see Dunford and Schwartz (1958). The Jordan canonical form originated with C. Jordan and K. Weierstrass, using different techniques. Actually, in 1874, Jordan and Kronecker, himself a student of Weierstrass, were quarreling over priorities regarding pencil equivalence and the reduction to canonical forms. Jordan was stressing spectral theory, i.e., eigenvalues and eigenvectors, whereas Weierstrass, defended by Kronecker, was stressing invariant factors and elementary divisors. For a comprehensive discussion of this controversy, see Brechenmacher (2007). The study of a linear transformation on a vector space via the study of the polynomial module structure induced by it on that space appears already in van der Waerden (1931). Although it is very natural, it did not become the standard approach in the literature, most notably in books aimed at a broader mathematical audience. This is probably due to the perception that the concept of module is too abstract. Our approach to the structure theory of linear transformations and the Jordan canonical form, based on polynomial models, rational models, and shift operators,

6.7 Notes and Remarks

159

is a concretization of that approach. It was originated in Fuhrmann (1976), which in turn was based on ideas stemming from Hilbert space operator theory. In this connection, the reader is advised to consult Sz.-Nagy and Foias (1970) and Fuhrmann (1981). The transition from the cyclic case to the noncyclic case, described in Section 6.4, uses a special case of general polynomial model techniques introduced in Fuhrmann (1976).

Chapter 7

Inner Product Spaces

7.1 Introduction In this chapter we focus on the study of vector spaces and linear transformations that relate to notions of distance, angle, and orthogonality. We will restrict ourselves throughout to the case of the real field R or the complex field C.

7.2 Geometry of Inner Product Spaces Definition 7.1. Let U be a vector space over C (or R). An inner product on U is a function (·, ·) : U × U −→ C that satisfies the following conditions: 1. (x, x) ≥ 0, and (x, x) = 0 if and only if x = 0. 2. For α1 , α2 ∈ C and x1 , x2 , y ∈ U we have (α1 x1 + α2 x2 , y) = α1 (x1 , y) + α2 (x2 , y). 3. We have (x, y) = (y, x). We note that these axioms imply the antilinearity of the inner product in the right variable, that is, (x, α1 y1 + α2 y2 ) = α1 (x, y1 ) + α2 (x, y2 ). A function φ (x, y) that is linear in x and antilinear in y and satisfies φ (x, y) = φ (y, x) is called a Hermitian form. Thus the inner product is a Hermitian form in U . We define the norm of a vector x ∈ U by 1

x = (x, x) 2 ,

P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 7, © Springer Science+Business Media, LLC 2012

161

162

7 Inner Product Spaces

where we take the nonnegative square root. A vector x ∈ U is called normalized, or a unit vector, if x = 1. The existence of the inner product allows us to introduce and utilize the all-important notion of orthogonality. We say that x, y ∈ U are orthogonal, and write x ⊥ y, if (x, y) = 0. Given a set S, we write x ⊥ S if (x, s) = 0 for all s ∈ S. We define the set S⊥ by S⊥ = {x | (x, s) = 0, for all s ∈ S} =



{x | (x, s) = 0}.

s∈S

A set of vectors {xi } is called orthogonal if (xi , x j ) = 0 whenever i = j. A set of vectors {xi } is called orthonormal if (xi , x j ) = δi j . Theorem 7.2 (The Pythagorean theorem). Let {xi }ki=1 be an orthogonal set of vectors in U . Then k  k 2  ∑ xi  = ∑ xi 2 . i=1

Proof.  k 2  ∑ xi  = i=1



k

k

i=1

j=1

∑ xi , ∑ x j

i=1



k

k

k

= ∑ ∑ (xi , x j ) = ∑ xi 2 . i=1 j=1

i=1



Theorem 7.3 (The Schwarz inequality). For all x, y ∈ U we have |(x, y)| ≤ x · y.

(7.1)

Proof. If y = 0, the equality holds trivially. So, assuming y = 0, we set e = y/y. We note that x = (x, e)e + (x − (x, e)e) and (x − (x, e)e) ⊥ (x, e)e. Applying the Pythagorean theorem, we get x2 = (x, e)e2 + x − (x, e)e2 ≥ (x, e)e2 = |(x, e)|2 , or |(x, e)| ≤ x. Substituting y/y for e, inequality (7.1) follows.



Theorem 7.4 (The triangle inequality). For all x, y ∈ U we have x + y ≤ x + y.

(7.2)

Proof. We compute x + y2 = (x + y, x + y) = (x, x) + (x, y) + (y, x) + (y, y) = x2 + 2Re(x, y) + y2 ≤ x2 + 2|(x, y)| + y2 ≤ x2 + 2xy + y2 = (x + y)2.



7.2 Geometry of Inner Product Spaces

163

Next, we prove two important identities. The proof of both is computational and we omit it. Proposition 7.5. For all vectors x, y ∈ U we have 1. The polarization identity: 1 (x, y) = {x + y2 − x − y2 + ix + iy2 − ix − iy2} 4

(7.3)

2. The parallelogram identity:   x + y2 + x − y2 = 2 x2 + y2 .

(7.4)

Proposition 7.6 (The Bessel inequality). Let {ei }ki=1 be an orthonormal set of vectors in U . Then k

x2 ≥ ∑ |(x, ei )|2 . i=1

Proof. Let x be an arbitrary vector in U . Obviously, we can write x = ∑ki=1 (x, ei )ei +  k x − ∑i=1 (x, ei )ei . Since the vector x − ∑ki=1 (x, ei )ei is orthogonal to all e j , j = 1, . . . , k, we can use the Pythagorean theorem to obtain x2 = x − ∑ki=1 (x, ei )ei 2 +  ∑ki=1 (x, ei )ei 2 ≥  ∑ki=1 (x, ei )ei 2 = ∑ki=1 |(x, ei )|2 .



Proposition 7.7. An orthogonal set of nonzero vectors {xi }ki=1 is linearly independent. Proof. Assume ∑ki=1 ci xi = 0. Taking the inner product with x j , we get k

0 = ∑ ci (xi , x j ) = c j x j 2 . i=1

This implies c j = 0 for all j.



Since orthogonality implies linear independence, we can search for bases consisting of orthogonal, or even better, orthonormal, vectors. Thus an orthogonal basis for U is an orthogonal set that is also a basis for U . Similarly, an orthonormal basis for U is an orthonormal set that is also a basis for U . The next theorem, generally known as the Gram–Schmidt orthonormalization process, gives a constructive method for computing orthonormal bases. Theorem 7.8 (Gram–Schmidt orthonormalization). Let {x1 , . . . , xk } be linearly independent vectors in U . Then there exists an orthonormal set {e1 , . . . , ek } such that span (e1 , . . . , e j ) = span (x1 , . . . , x j ), 1 ≤ j ≤ k. (7.5)

164

7 Inner Product Spaces

Proof. We prove this by induction. For j = 1 we set e1 = x j

x1 x1  .

Assume we have

j−1 constructed e1 , . . . , e j−1 as required. We consider next = x j − ∑i=1 (x j , ei )ei . Clearly x j = 0, for otherwise, x j ∈ span (e1 , . . . , e j−1 ) = span (x1 , . . . , x j−1 ), contrary to the assumption of linear independence of the xi . We define now

ej =

x j x j 

=

j−1 x j − ∑i=1 (x j , ei )ei j−1 x j − ∑i=1 (x j , ei )ei 

.

Since ei ∈ span (x1 , . . . , x j ) for i = 1, . . . , j − 1 and since e j ∈ span (e1 , . . . , e j−1 , x j ) ⊂ span (x1 , . . . , x j ), we obtain the inclusion span (e1 , . . . , e j ) ⊂ span (x1 , . . . , x j ). To get the inverse inclusion, we observe that span(x1 , . . . , x j−1 ) ⊂ span(e1 , . . . , e j ) follows from the xj =

j−1

j−1

i=1

i=1

∑ (x j , ei )ei + x j − ∑ (x j , ei )ei e j

implies x j ∈ span (e1 , . . . , e j ). So span (x1 , . . . , x j ) ⊂ span (e1 , . . . , e j ), and these two inclusions imply (7.5).  Corollary 7.9. Every finite-dimensional inner product space has an orthonormal basis. Corollary 7.10. Every orthonormal set in a finite-dimensional inner product space U can be extended to an orthonormal basis. Proof. Let {e1 , . . . , em } be an orthonormal set, thus necessarily linearly independent. Hence, by Theorem 2.13, it can be extended to a basis {e1 , . . . , em , xm+1 , . . . , xn } for U . Applying the Gram–Schmidt orthonormalization process, we get an orthonormal basis {e1 , . . . , en }. The first m vectors remain unchanged.  Orthonormal bases are particularly convenient for vector expansions, or alternatively, for computing coordinates. Proposition 7.11. Let {e1 , . . . , en } be an orthonormal basis for a finite-dimensional inner product space U . Then every vector x ∈ U has a unique representation in the form n

x = ∑ (x, ei )ei . i=1

We consider next an important approximation problem. We define the distance of a vector x ∈ U to a subspace M by

δ (x, M ) = inf{x − m|m ∈ M }. A best approximant for x in M is a vector m0 ∈ M that satisfies x − m0  = δ (x, M ).

7.2 Geometry of Inner Product Spaces

165

Theorem 7.12. Let U be a finite-dimensional inner product space and M a subspace. Let x ∈ U . Then 1. A best approximant for x in M exists. 2. A best approximant for x in M is unique. 3. A vector m0 ∈ M is a best approximant for x in M if and only if x − m0 is orthogonal to M . Proof. 1. Let {e1 , . . . , em } be an orthonormal basis for M . Extend it to an orthonormal basis {e1 , . . . , en } for U . We claim that m0 = ∑m i=1 (x, ei )ei is a best approximant. Clearly m0 ∈ M . Any other vector m ∈ M has a unique representation of the form m = ∑m i=1 ci ei . Therefore m

m

m

i=1

i=1

i=1

x − ∑ ci ei = x − ∑ (x, ei )ei + ∑ [(x, ei ) − ci ]ei . Now, the vector x − ∑m i=1 (x, ei )ei is orthogonal to L(e1 , . . . , em ). Hence, applying the Pythagorean theorem, we get m m 2 2 2 x − ∑m i=1 ci ei  = x − ∑i=1 (x, ei )ei  +  ∑i=1 [(x, ei ) − ci ]ei  m 2 ≥ x − ∑i=1 (x, ei )ei  .

2. Let ∑m i=1 ci ei be another best approximant. By the previous computation, we must m 2 2 have  ∑m i=1 [(x, ei ) − ci ]ei  = ∑i=1 |(x, ei ) − ci | , and this happens if and only if ci = (x, ei ) for all i. Thus the two approximants coincide. 3. Assume m0 ∈ M is the best approximant. We saw that m0 = ∑m i=1 (x, ei )ei and hence x − ∑m i=1 (x, ei )ei is orthogonal to M = L(e1 , . . . , em ). Conversely, assume m0 ∈ M and x − m0 ⊥ M . Then, for any vector m ∈ M , we have x − m = (x − m0 ) + (m − m0 ). Since m − m0 ∈ M , using the Pythagorean theorem, we have x − m2 = x − m0 2 + m − m02 ≥ x − m02 . Hence m0 is the best approximant.



In the special case of inner product spaces, we will reserve the notation U = M ⊕ N for the case of orthogonal direct sums, that is, U = M + N and M ⊥ N . We note that M ⊥ N implies M ∩ N = {0}. Theorem 7.13. Let U be a finite-dimensional inner product space and M a subspace. Then we have U = M ⊕ M ⊥. M⊥

(7.6)

are orthogonal. It suffices to show that they Proof. The subspaces M and span U . Thus, let x ∈ U , and let m be its best approximant in M . We can write  x = m + (x − m). By Theorem 7.12, we have x − m ∈ M ⊥ .

166

7 Inner Product Spaces

Inner product spaces are a particularly convenient setting for the development of duality theory. The key to this is the identification of the dual space to an inner product space with the space itself. This is done by identifying a linear functional with an inner product with a vector. Theorem 7.14. Let U be a finite-dimensional inner product space. Then f is a linear functional on U if and only if there exists a vector ξ f ∈ U such that f (x) = (x, ξ f ).

(7.7)

Proof. If f is defined by (7.7), then obviously it is a linear functional on U . To prove the converse, we note first that if f is the zero functional, then we just choose ξ f = 0. Thus, without loss of generality, we can assume that f is a nonzero functional. In this case, M = Ker f is a subspace of codimension 1. This implies dim M ⊥ = 1. Let us choose an arbitrary nonzero vector y ∈ M ⊥ and set ξ f = α y. Since (y, ξ f ) = (y, α y) = α y2 , by choosing α = the same holds, by linearity, for all (x, ξ f ) = 0. Thus (7.7) follows.

f (y) , y2

we get f (y) = (y, ξ f ), and

vectors in M ⊥ . For x ∈ M

it is clear that f (x) = 

7.3 Operators in Inner Product Spaces In this section we study some important classes of operators in inner product spaces. We begin by studying the adjoint transformation. Due to the availability of inner products and the representation of linear functionals, the definition is slightly different from that given for general transformations.

7.3.1 The Adjoint Transformation Theorem 7.15. Let U , V be inner product spaces and let T : U −→ V be a linear transformation. Then there exists a unique linear transformation T ∗ : V −→ U such that (T x, y) = (x, T ∗ y) (7.8) for all x ∈ U and y ∈ V . Proof. Fix a vector y ∈ V . Then f (x) = (T x, y) defines a linear functional on U . Thus, by Theorem 7.14, there exists a unique vector ξ ∈ U such that (T x, y) = (x, ξ ). We define T ∗ by T ∗ y = ξ . Therefore, T ∗ is a map from V to U . It remains to show that T ∗ is a linear map. To this end, we compute

7.3 Operators in Inner Product Spaces

167

(x, T ∗ (α1 y1 + α2 y2 )) = (T x, α1 y1 + α2 y2 ) = α1 (T x, y1 ) + α2 (T x, y2 ) = α1 (x, T ∗ y1 ) + α2 (x, T ∗ y2 ) = (x, α1 T ∗ y1 ) + (x, α2 T ∗ y2 ) = (x, α1 T ∗ y1 + α2 T ∗ y2 ). By the uniqueness of the representing vector, we get T ∗ (α1 y1 + α2 y2 ) = α1 T ∗ y1 + α2 T ∗ y2 .



We will call the transformation T ∗ the adjoint or, if emphasis is needed, the Hermitian adjoint of T . Given a complex matrix A = (ai j ), its adjoint A∗ = (a∗i j ) ˜ is defined by a∗ = a ji , or alternatively, by A∗ = A. ij

Proposition 7.16. Let T : U −→ V be a linear transformation. Then (Im T )⊥ = Ker T ∗ . Proof. We have for x ∈ U , y ∈ V , (T x, y) = (x, T ∗ y). So y ⊥ Im T if and only if T ∗ y ⊥ U , i.e., if and only if y ∈ Ker T ∗ .  Corollary 7.17. Let T be a linear transformation in an inner product space U . Then we have the following direct sum decompositions: U = Im T ⊕ Ker T ∗ , U = Im T ∗ ⊕ KerT. Moreover, rankT = rank T ∗ and dim Ker T = dim Ker T ∗ . Proof. Follows from Theorem 7.13 and Proposition 7.16.



Proposition 7.18. Let T be a linear transformation in U . Then M is a T -invariant subspace of U if and only if M ⊥ is a T ∗ -invariant subspace. Proof. Follows from equation (7.8).



We proceed to study the matrix representation of a linear transformation T with respect to orthonormal bases. These are particularly easy to compute. Proposition 7.19. Let T : U −→ V be a linear transformation and let B = {e1 , . . . , en } and B1 = { f1 , . . . , fm } be orthonormal bases for U and V respectively. Then 1 1. For the matrix representation [T ]B B = (ti j ), we have

ti j = (Te j , fi ).

168

7 Inner Product Spaces

2. We have for the adjoint T ∗ , B1 ∗ [T ∗ ]B B1 = ([T ]B ) .

Proof. 1. The matrix representation of T with respect to the given bases is defined by Te j = ∑m k=1 tki f k . We compute m

(Te j , fi ) = ( ∑ tk j fk , fi ) = k=1

m

m

k=1

k=1

∑ tk j ( fk , fi ) = ∑ tk j δik = ti j .

∗ 2. Let [T ∗ ]B B1 = (ti j ). Then

ti∗j = (T ∗ f j , ei ) = ( f j , Tei ) = (Tei , f j ) = t ji .



Next, we study the properties of the map from T → T ∗ as a function from L(U , V ) into L(V , U ). Proposition 7.20. Let U , V be inner product spaces. Then the adjoint map has the following properties: 1. 2. 3. 4. 5. 6.

(T + S)∗ = T ∗ + S∗. (α T )∗ = α T ∗ . (ST )∗ = T ∗ S∗ . (T ∗ )∗ = T . ∗ =I . For the identity map IU in U , we have IU U −1 ∗ If T : U −→ V is invertible, then (T ) = (T ∗ )−1 .

Proof. 1. Let x ∈ U and y ∈ V . We compute (x, (T + S)∗ y) = ((T + S)x, y) = (T x + Sx, y) = (T x, y) + (Sx, y) = (x, T ∗ y) + (x, S∗y) = (x, T ∗ y + S∗y) = (x, (T ∗ + S∗ )y). By uniqueness we get (T + S)∗ = T ∗ + S∗ . 2. Computing (x, (α T )∗ y) = ((α T )x, y) = (α T x, y) = α (T x, y) = α (x, T ∗ y) = (x, α T ∗ y), we get, using uniqueness, that (α T )∗ = α T ∗ . 3. We compute (x, (ST )∗ y) = (ST x, y) = (T x, S∗ y) = (x, T ∗ S∗ y), or (ST )∗ = T ∗ S∗ .

7.3 Operators in Inner Product Spaces

4.

169

(T x, y) = (x, T ∗ y) = (T ∗ y, x) = (y, T ∗∗ x) = (T ∗∗ x, y),

and this implies T ∗∗ = T . 5. Let IU be the identity map in U . Then (IU x, y) = (x, y) = (x, IU y), ∗ =I . so IU U 6. Assume T : U −→ V is invertible. Then T −1 T = IU and T T −1 = IV . The first ∗ = T ∗ (T −1 )∗ , that is, (T −1 )∗ is a right inverse of T ∗ . equality implies IU = IU The second equality implies that it is a left inverse, and (T −1 )∗ = (T ∗ )−1 follows.  The availability of the norm function on an inner product space allows us to consider a numerical measure of the size of a linear transformation.

Definition 7.21. Let U1 , U2 be two inner product spaces. Given a linear transformation T : U1 −→ U2 , we define its norm by T  = sup T x. x≤1

Clearly, for an arbitrary linear transformation T , the norm T  is finite. For it is defined as the supremum of a continuous function on the unit ball {x ∈ U1 | x ≤ 1}, which is a compact set. It follows that the supremum is actually attained, and thus there exists a, not necessarily unique, unit vector x for which T x = T . The following proposition, whose standard proof we omit, sums up the basic properties of the operator norm. Proposition 7.22. We have the following properties of the operator norm: 1. 2. 3. 4. 5. 6.

T + S ≤ T  + S. α T  = |α | · T . T S ≤ T  · S. T ∗  = T . I = 1. T  = supx=1 T x = supx,y≤1 |(T x, y)| = supx,y=1 |(T x, y)|.

7.3.2 Unitary Operators The extra structure that inner product spaces have over linear spaces, namely the existence of inner products, leads to a new definition, that of an isomorphism of two inner product spaces.

170

7 Inner Product Spaces

Definition 7.23. Let V and W be two inner product spaces over the same field. A linear transformation T : V −→ W is an isometry if it preserves inner products, i.e., if (T x, Ty) = (x, y), for all x, y ∈ V . An isomorphism of inner product spaces is an isomorphism that is isometric. We will also call it an isometric isomorphism or alternatively a unitary isomorphism. Proposition 7.24. Given a linear transformation T : V −→ W , then the following properties are equivalent: 1. T preserves inner products. 2. For all x ∈ V we have T x = x. 3. We have T ∗ T = I. Proof. Assume T preserves inner products, i.e., (T x, Ty) = (x, y), for all x, y ∈ V . Choosing y = x, we get T x2 = (T x, T x) = (x, x) = x2 . If T x = x, then using the polarization identity (7.3), we get (T x, Ty) = =

1 {(T x + Ty)2 − (T x − Ty)2 + i(T x + iTy)2 − i(T x − iTy)2} 4 1 {(x + y)2 − (x − y)2 + i(x + iy)2 − i(x − iy)2} = (x, y). 4

Next observe that if T preserves inner products, then ((I − T ∗ T )x, y) = 0, which implies T ∗ T = I. Finally, if the equality T ∗ T = I holds, then clearly T preserves inner products.  Corollary 7.25. An isometry T : V −→ W maps orthonormal sets into orthonormal sets. The next result is the counterpart, for inner product spaces, of Theorem 4.14. Corollary 7.26. Two inner product spaces are isomorphic if and only if they have the same dimension. Proof. Assume a unitary isomorphism U : V −→ W exists. Since it maps bases into bases, the dimensions of V and W coincide. Conversely, suppose V and W have the same dimension. We choose orthonormal bases {e1 , . . . , en } and { f1 , . . . , fn } in V and W respectively. We define a linear map U : V −→ W by Uei = fi . It is immediate that the operator U so defined is a unitary isomorphism.  Proposition 7.27. Let U : V −→ W be a linear transformation. Then U is unitary if and only if U ∗ = U −1 . Proof. If U ∗ = U −1 , we get U ∗U = UU ∗ = I, i.e., U is an isometric isomorphism. Conversely, if U is unitary, we have U ∗U = I. Since U is an isomorphism and the inverse transformation, if it exists, is unique, we obtain U ∗ = U −1 . 

7.3 Operators in Inner Product Spaces

171

Next, we consider matrix representations. We say that a complex matrix is unitary if A∗ A = I. Here A∗ is defined by a∗i j = a ji. We proceed to consider matrix representations for unitary maps. We say that a complex n × n matrix A is unitary if A∗ A = I, where A∗ is the Hermitian adjoint of A. We expect that the matrix representation of a unitary matrix, when taken with respect to orthonormal bases, should reflect the unitarity property. We state this in the following proposition, omitting the simple proof. Proposition 7.28. Let U : V −→ W be a linear transformation between two inner product spaces. Let B1 = {e1 , . . . , en } and B2 = { f1 , . . . , fn }, be orthonormal bases in V and W respectively. Then U is unitary if and only if its matrix representation 2 [U]B B1 is a unitary matrix. The notion of similarity can be strengthened in the context of inner product spaces. Definition 7.29. Let V and W be two inner product spaces over the same field, and let T : V −→ V and S : W −→ W be linear transformations. We say that T and S are unitarily equivalent if there exists a unitary map U : V −→ W for which UT = SU, or equivalently, for which the following diagram is commutative: V

U

-

T

W S

? V

U

-

? W

The unitary transformations play, in the algebra of linear transformations on an inner product space, the same role that complex numbers of absolute value one play in the complex number field C. This is reflected in the eigenvalues of unitary transformations as well as in their structure. Proposition 7.30. Let U be a unitary operator in an inner product space V . Then 1. 2. 3. 4.

All eigenvalues of U have absolute value one. If Ux = λ x, then U ∗ x = λ x. Eigenvectors corresponding to different eigenvalues are orthogonal. We have Ker (U − λ1 I)ν1 · · · (U − λk I)νk = Ker (U − λ1 I) · · · (U − λk I).

5. The minimal polynomial mU (z) of U splits into distinct linear factors.

172

7 Inner Product Spaces

Proof. 1. Let λ be an eigenvalue of U and x a corresponding eigenvector. Then Ux = λ x implies (Ux, x) = (λ x, x) = λ (x, x). Now, using the unitarity of U, we have x2 = Ux2 = λ x2 = |λ |2 x2 . This implies |λ | = 1. 2. Assume Ux = λ x. Using the fact that U ∗U = UU ∗ = I, we compute U ∗ x − λ x2 = (U ∗ x − λ x,U ∗ x − λ x) = (U ∗ x,U ∗ x) − λ (x,U ∗ x) − λ (x,Ux) + |λ |2x2 = (UU ∗ x, x) − λ (Ux, x) − λ (x,Ux) + x2 = x2 − λ (λ x, x) − λ (x, λ x) + x2 = x2 − 2|λ |2x2 + x2 = 0. 3. Let λ , μ be distinct eigenvalues and x, y the corresponding eigenvectors. Then

λ (x, y) = (Ux, y) = (x,U ∗ y) = (x, μ y) = μ (x, y). Hence, (λ − μ )(x, y) = 0. Since λ = μ , we conclude that (x, y) = 0 or x ⊥ y. 4. Since U is unitary, U and U ∗ commute. Therefore also U − λ I and U ∗ − λ I commute. Assume now that (U − λ I)ν x = 0. Without loss of generality we may n assume that (U − λ I)2 x = 0. This implies 0 = (U ∗ − λ I)2 (U − λ I)2 x = [(U ∗ − λ I)2 n

n

n−1

(U − λ I)2

n−1

]2 x.

So 0 = ([(U ∗ − λ I)2

n−1

(U − λ I)2

n−1

]2 x, x) = (U ∗ − λ I)2

n−1

(U − λ I)2

n−1

x2 ,

or (U ∗ − λ I)2 (U − λ I)2 x = 0. This implies in turn that (U − λ I)2 x = 0, n−1 and hence (U − λ I)2 x = 0. By repeating the argument we conclude that (U − λ I)x = 0. Next, we use the fact that all factors U − λi I commute. Assuming (U − λ1 I)ν1 · · · (U − λk I)νk x = 0, we conclude n−1

n−1

n−1

0 = (U − λ1 I)(U − λ2 I)ν2 · · · (U − λk I)νk x = (U − λ2 I)ν2 (U − λ1 I)(U − λ3 I)ν3 · · · (U − λk I)νk x. This implies (U − λ1 I)(U − λ2 I)(U − λ3 I)ν3 · · · (U − λk I)νk x = 0. Proceeding inductively, we get (U − λ1 I) · · · (U − λk I)x = 0. 5. Follows from part 4. 

7.3 Operators in Inner Product Spaces

173

7.3.3 Self-adjoint Operators Probably the most important class of operators in an inner product space is that of self-adjoint operators. An operator T in an inner product space U is called selfadjoint, or Hermitian, if T ∗ = T . Similarly, a complex matrix A is called selfadjoint or Hermitian if it satisfies A∗ = A, or equivalently ai j = a ji . The next proposition shows that self-adjoint operators in an inner product space play in L(U ) a role similar to that of the real numbers in the complex field. Proposition 7.31. Every linear transformation T in an inner product space can be written in the form T = T1 + iT2 with T1 , T2 self-adjoint. Such a representation is unique. Proof. Write 1 1 T = (T + T ∗ ) + i (T − T ∗ ) 2 2i and note that T1 = 12 (T + T ∗ ) and T2 = 2i1 (T − T ∗ ) are both self-adjoint. Conversely, if T = T1 + iT2 with T1 , T2 self-adjoint, then T ∗ = T1 − iT2 . By elimination we get that T1 , T2 have necessarily the form given previously.  In preparation for the spectral theorem, we prove a few properties of self-adjoint operators that are of intrinsic interest. Proposition 7.32. Let T be a self-adjoint operator in an inner product space U . Then 1. All eigenvalues of T are real. 2. Eigenvectors corresponding to different eigenvalues are orthogonal. 3. We have Ker (T − λ1 I)ν1 · · · (T − λk I)νk = Ker (T − λ1 I) · · · (T − λk I). 4. The minimal polynomial mT (z) of T splits into distinct linear factors. Proof. 1. Let λ be an eigenvalue of T and x a corresponding eigenvector. Then T x = λ x implies (T x, x) = (λ x, x) = λ (x, x). Now, using the self-adjointness of (T x,x) T , we have (T x, x) = (x, T x) = (T x, x), that is, (T x, x) is real. So is λ = (x,x) . 2. Let λ , μ be distinct eigenvalues of T , and x, y corresponding eigenvectors. Using the fact that eigenvalues of T are real, we get (T x, y) = (λ x, y) = λ (x, y) and (T x, y) = (x, Ty) = (x, μ y) = μ (x, y) = μ (x, y). By subtraction we get (λ − μ )(x, y) = 0, and since λ = μ , we conclude that (x, y) = 0, or x ⊥ y.

174

7 Inner Product Spaces

3. We prove this by induction on the number of factors. So, let k = 1 and assume that (T − λ1 I)ν1 x = 0. Then, by multiplying this equation by (T − λ1 I)μ , we may n assume without loss of generality that (T − λ1 I)2 x = 0. Then we get n

0 = ((T − λ1 I)2 x, x) = ((T − λ1 I)2

n−1

x, (T − λ1I)2

n−1

x) = (T − λ1 I)2

n−1

x,

n−1

which implies (T − λ1 I)2 x = 0. Repeating this argument, we finally obtain (T − λ1 I)x = 0. Assume now the statement holds for up to k − 1 factors and k (T − λ I)νi x = 0. Therefore (T − λ I)νk Π k−1 (T − λ I)νi x = 0. By the that Πi=1 i i k i=1 argument for k = 1 we conclude that k−1 k−1 (T − λi I)νi x = Πi=1 (T − λi I)νi [(T − λk I)x]. 0 = (T − λiI)Πi=1

By the induction hypothesis we get k−1 k 0 = Πi=1 (T − λi I)[(T − λk I)x] = Πi=1 (T − λiI)x, k (T − λ I) = 0. or, since x is arbitrary, Πi=1 i 4. Follows from the previous part.



Theorem 7.33. Let T be self-adjoint. Then there exists an orthonormal basis consisting of eigenvectors of T . k (z − λ ), with Proof. Let mT (z) be the minimal polynomial of T . Then mT (z) = Πi=1 i the λi distinct. Let πi (z) be the corresponding Lagrange interpolation polynomials. We observe that the πi (T ) are orthogonal projections. That they are projections has been proved in Theorem 6.25. That they are orthogonal projections follows from their self-adjointness, noting that, given a real polynomial p(z) and a selfadjoint operator T , necessarily p(T ) is self-adjoint. This implies that U = Ker (λ1 I − T ) ⊕ · · · ⊕ Ker (λk I − T ) is an orthogonal direct sum decomposition. Choosing an orthonormal basis in each subspace Ker (λi I − T ), we get an orthonormal basis made of eigenvectors. 

As a consequence, we obtain the spectral theorem. Theorem 7.34. Let T be self-adjoint. Then it has a unique representation of the form s

T = ∑ λi Pi , i=1

where the λi ∈ R are distinct and Pi are orthogonal projections satisfying s

Pi Pj = δi j Pj , ∑ Pi = I. i=1

7.3 Operators in Inner Product Spaces

175

Proof. Let λi be the distinct eigenvalues of T . We define the Pi to be the orthogonal projections on Ker (λi I − T ). Conversely, if such a representation exists, it follows that necessarily, the λi are the eigenvalues of T and Im Pi = Ker (λi I − T ).  In the special case of self-adjoint operators, the computation of the norm has a further characterization. Proposition 7.35. Let T be a self-adjoint operator in a finite-dimensional inner product space U . Then 1. We have T  = supx≤1 |(T x, x)| = supx=1 |(T x, x)|. 2. Let λ1 , . . . , λn be the eigenvalues of T ordered so that |λi | ≥ |λi+1 |. Then T  = |λ1 |, i.e., the norm equals the modulus of the largest eigenvalue. Proof. 1. Since for x satisfying x ≤ 1, we have |(T x, x)| ≤ T x · x ≤ T  · x2 ≤ T , it follows that supx=1 |(T x, x)| ≤ supx≤1 |(T x, x)| ≤ T . To prove the converse inequality, we argue as follows. We use the following version of the polarization identity: 4Re (T x, y) = (T (x + y), x + y) − (T(x − y), x − y), which implies 1 sup |(T z, z)|[x + y2 + x − y2], 4 z≤1 1 |(Re (T x, y)| ≤ sup |(T z, z)|[2x2 + 2y2] ≤ sup |(T z, z)|. 4 z≤1 z≤1

|(Re (T x, y)| ≤

Choosing y = TT xx it follows that T x ≤ supz≤1 |(T z, z)|. The equality supz≤1 |(T z, z)| = supz=1 |(T z, z)| is obvious. 2. Let λ1 be the eigenvalue of T of largest modulus and x1 a corresponding, normalized, eigenvector. Then |λ1 | = |(λ1 (x1 , x1 )| = |(λ1 x1 , x1 )| = |(T x1 , x1 )| ≤ T  · x1 2 = T . So the absolute values of all eigenvalues of T are bounded by T . Assume now that x is a vector for which T x = T . We will show that it is an eigenvector of T corresponding either to T  or to −T . The equality sup |(T x, x)| = T  implies either sup(T x, x) = T  or inf(T x, x) = −T . We assume the first is satisfied. Thus there exists a unit vector x for which (T x, x) = T . We compute

176

7 Inner Product Spaces

0 ≤ T x − T x2 = T x2 − 2T (T x, x) + T 2 x2 ≤ T 2 x2 − 2T 2 + T 2 x2 = 0. So necessarily T x = T  · x, or T  = λ1 . If inf(T x, x) = −T , the same argument can be used. 

7.3.4 The Minimax Principle For self-adjoint operators we have a very nice characterization of eigenvalues, known as the minimax principle. Theorem 7.36. Let T be a self-adjoint operator on an n-dimensional inner product space U. Let λ1 ≥ · · · ≥ λn be the eigenvalues of T . Then

λk =

min

max{(T x, x) | x = 1}.

{M |dim M =n−k+1} x∈M

(7.9)

Proof. Let {e1 , . . . , en } be an orthonormal basis of U consisting of eigenvectors of T . Let now M be an arbitrary subspace of U of dimension n − k + 1. Let Mk = L(e1 , . . . , ek ). Then since dim Mk + dim Nk = k + (n − k + 1) = n + 1, their intersection Mk ∩ Nk is nontrivial and contains a unit vector x. For this vector we have k

k

i=1

i=1

(T x, x) = ∑ λi |(x, ei )|2 ≥ λk ∑ |(x, ei )|2 = λk . This shows that min{M |dim M =n−k+1} maxx∈M {(T x, x) | x = 1} ≥ λk . To complete the proof, we need to exhibit at least one subspace on which the reverse inequality holds. To this end, let Mk = L(ek , . . . , en ). Then for any x ∈ M with x = 1, we have n

n

i=k

i=k

(T x, x) = ∑ λi |(x, ei )|2 ≤ λk ∑ |(x, ei )|2 ≤ λk x2 = λk . Therefore, we have n

n

i=k

i=k

max{(T x, x)| x = 1} = ∑ λi |(x, ei )|2 ≤ λk ∑ |(x, ei )|2 ≤ λk x2 = λk .

x∈M



7.3.5 The Cayley Transform The classes of unitary and self-adjoint operators generalize the sets of complex numbers of absolute value one and the set of real numbers respectively. Now the fractional linear transformation w = z−i z+i maps the upper half-plane onto the unit

7.3 Operators in Inner Product Spaces

177

disk, and in particular the real line onto the unit circle. Naturally, one wonders whether this can be extended to a map of self-adjoint operators onto unitary operators. The next theorem focuses on this map. Theorem 7.37. Let A be a self-adjoint operator in a finite-dimensional inner product space V . Then 1. For each vector x ∈ V , we have (A + iI)x2 = (A − iI)x2 = Ax2 + x2. 2. The operators A + iI, A − iI are both injective, hence invertible. 3. The operator U defined by U = (A − iI)(A + iI)−1

(7.10)

is a unitary operator for which 1 is not an eigenvalue. The operator U is called the Cayley transform of A. 4. Given a unitary map U in a finite-dimensional inner product space such that 1 is not an eigenvalue, the operator A defined by A = i(I + U)(I − U)−1

(7.11)

is a self-adjoint operator. This map is called the inverse Cayley transform. Proof. 1. We compute (A + iI)x2 = Ax2 + x2 + (ix, Ax) + (Ax, ix) = Ax2 + x2 = (A − iI)x2 . 2. The injectivity of both A + iI and A − iI is an immediate consequence of the previous equality. This shows the invertibility of both operators. 3. Let x ∈ V be arbitrary. Then there exists a unique vector z such that x = (A + iI)z or z = (A + iI)−1x. Therefore (A − iI)(A + iI)−1x = (A − iI)z = (A + iI)z = x. This shows that U defined by (7.10) is unitary. To see that 1 is not an eigenvalue of U, assume Ux = x. Thus (A − iI)(A + iI)−1 x = x = (A + iI)(A + iI)−1x and hence 2i(A + iI)−1 x = 0. This implies x = 0. 4. Assume U is a unitary map such that 1 is not an eigenvalue. Then I − U is invertible. Defining A by (7.11), and using the fact that U ∗ = U −1 , we compute A∗ = −i(I + U ∗ )(I − U ∗)−1 = −i(I + U −1)UU −1 (I − U −1 )−1 = −i(U + I)(U − I)−1 = i(I + U)(I − U)−1 = A. So A is self-adjoint.



178

7 Inner Product Spaces

7.3.6 Normal Operators Our analysis of the classes of unitary and self-adjoint operators in inner product spaces showed remarkable similarities. In particular, both classes admitted the existence of orthonormal bases made up of eigenvectors. One wonders whether the set of all operators having this property can be characterized. In fact, this can be done, and the corresponding class is that of normal operators, which we proceed to introduce. Definition 7.38. Let T be a linear transformation in a finite-dimensional inner product space. We say that T is normal if it satisfies T ∗T = T T ∗.

(7.12)

Proposition 7.39. The operator T is normal if and only if for every x ∈ V , we have T x = T ∗ x.

(7.13)

Proof. We compute, assuming T is normal, T x2 = (T x, T x) = (T ∗ T x, x) = (T T ∗ x, x) = (T ∗ x, T ∗ x) = T ∗ x2 , which implies (7.13). Conversely, assuming the equality (7.13), we have (T T ∗ x, x) = (T ∗ T x, x), and hence, by Proposition 7.35, we get T T ∗ = T ∗ T , i.e., T is normal.  Corollary 7.40. Let T be a normal operator. Let α be an eigenvalue of T . Then T x = α x implies T ∗ x = α x. Proof. If T is normal, so is T − α I, and we apply Proposition 7.39.



Proposition 7.41. Let T be a normal operator in a complex inner product space. If for some complex number λ , we have (T − λ I)ν x = 0, then (T − λ I)x = 0. Proof. The proof is exactly as in the case of unitary operators and is omitted.



Lemma 7.42. Let T be a normal operator and assume λ , μ are distinct eigenvalues of T . If x, y are corresponding eigenvectors, then x ⊥ y. Proof. We compute

λ (x, y) = (λ x, y) = (T x, y) = (x, T ∗ y) = (x, μ y) = μ (x, y). Hence (λ − μ )(x, y) = 0, and since λ = μ , we conclude that (x, y) = 0. Thus x, y are orthogonal.  The following is known as the spectral theorem for normal operators.

7.3 Operators in Inner Product Spaces

179

Theorem 7.43. Let T be a normal operator in a finite-dimensional complex inner product space V . Then there exists an orthonormal basis of V consisting of eigenvectors of T . k (z − λ )νi be the primary decomposition of the minimal Proof. Let mT (z) = Πi=1 i polynomial of T . We set Mi = Ker (λi I − T )νi . By Proposition 7.41 we have Mi = Ker (λi I −T ), i.e., it is the eigenspace corresponding to the eigenvalue λi . By Lemma 7.42, the subspaces Mi are mutually orthogonal. We apply now Theorem 6.26 to conclude that V = M1 ⊕ · · · ⊕ Mk , where now this is an orthogonal direct sum decomposition. Choosing an orthonormal basis in each subspace Mi and taking their union, we get an orthonormal basis for V made of eigenvectors. 

Theorem 7.44. Let T be a normal operator in a finite-dimensional inner product space V . Then 1. A subspace M ⊂ V is invariant under T if and only if it is invariant under T ∗ . 2. Let M be an invariant subspace for T . Then the direct sum decomposition V = M ⊕ M ⊥ reduces T . 3. T reduced to an invariant subspace M , that is, T |M is a normal operator. Proof. 1. Let M be invariant under T . Since T |M has at least one eigenvalue and a corresponding eigenvector, say T x1 = λ1 x1 , then also T ∗ x1 = λ1 x1 . Let M1 be the subspace spanned by x1 . We consider M  M1 = M ∩ M1⊥ . This is also invariant under T . Proceeding by induction, we conclude that M is spanned by eigenvectors of T hence, by Proposition 7.41, by eigenvectors of T ∗ . This shows that M is T ∗ invariant. The converse follows by symmetry. 2. Since M is invariant under both T and T ∗ , so is M ⊥ . 3. Let {e1 , . . . , em } be an orthonormal basis of M consisting of eigenvectors. Thus Tei = λi ei and T ∗ ei = λi ei . Let x = ∑m i=1 αi ei ∈ M . Then ∗ m T ∗ T x = T ∗ T ∑m i=1 αi ei = T ∑i=1 αi λi ei m = ∑i=1 αi |λi |2 ei = T ∑m i=1 αi λi ei m ∗ ∗ = T T ∑i=1 αi ei = T T x.

This shows that (T |M )∗ = T ∗ |M .



Theorem 7.45. The operator T is normal if and only if T ∗ = p(T ) for some polynomial p. Proof. If T ∗ = p(T ), then obviously T and T ∗ commute, i.e., T is normal. Conversely, if T is normal, there exists an orthonormal basis {e1 , . . . , en } consisting of eigenvectors corresponding to the eigenvalues λ1 , . . . , λn . Let p be any polynomial that interpolates the values λi at the points λi . Then p(T )ei = p(λi )ei = λi ei = T ∗ ei . This shows that T ∗ = p(T ).



180

7 Inner Product Spaces

7.3.7 Positive Operators We consider next a subclass of self-adjoint operators, that of positive operators. Definition 7.46. An operator T on an inner product space V is called nonnegative if for all x ∈ V , we have (T x, x) ≥ 0. The operator T is called positive if it is nonnegative and (T x, x) = 0 for x = 0 only. Similarly, a complex Hermitian matrix A is called nonnegative if for all x ∈ Cn , we have (Ax, x) ≥ 0. Proposition 7.47. 1. A Hermitian operator T in an inner product space is nonnegative if and only if all its eigenvalues are nonnegative. 2. A Hermitian operator T in an inner product space is positive if and only if all its eigenvalues are positive. Proof. 1. Assume T is Hermitian and nonnegative. Let λ be an arbitrary eigenvalue of T and x a corresponding eigenvector. Then

λ x2 = λ (x, x) = (λ x, x) = (T x, x) ≥ 0, which implies λ ≥ 0. Conversely, assume T is Hermitian and all its eigenvalues are nonnegative. Let {e1 , . . . , en } be an orthonormal basis consisting of eigenvectors corresponding to the eigenvalues λi ≥ 0. An arbitrary vector x has the expansion x = ∑ni=1 (x, ei )ei ; hence (T x, x) = (T ∑ni=1 (x, ei )ei , ∑nj=1 (x, e j )e j ) = ∑ni=1 ∑nj=1 (x, ei )(x, e j )(Tei , e j ) = ∑ni=1 ∑nj=1 (x, ei )(x, e j )(λi ei , e j ) = ∑ni=1 λi |(x, ei )|2 ≥ 0. 2. The proof is the same except for the inequalities being strict.



An easy way of producing nonnegative operators is to consider operators of the form A∗ A or AA∗ . The next result shows that this is the only way. Proposition 7.48. T ∈ L(U ) is nonnegative if and only if T = A∗ A for some operator A in U . Proof. That A∗ A is nonnegative is immediate. Conversely, assume T is nonnegative. By Theorem 7.33, there exists an orthonormal basis B = {e1 , . . . , en } in U made out of eigenvectors of T corresponding to the eigenvalues λ1 , . . . , λn . Since T is nonnegative, all the λi are nonnegative and 1

have nonnegative square roots λi 2 . Define a linear transformation S in U by letting 1

Sei = λi 2 ei . Clearly, S2 ei = λi ei = Tei . The operator S so defined is self-adjoint, for given x, y ∈ U , we have

7.3 Operators in Inner Product Spaces

181

(Sx, y) = (S ∑ni=1 (x, ei )ei , ∑nj=1 (y, e j )e j ) 1

= (∑ni=1 λi2 (x, ei )ei , ∑nj=1 (y, e j )e j ) 1

= (∑ni=1 (x, ei )ei , ∑nj=1 λi2 (y, e j )e j ) = (∑ni=1 (x, ei )ei , S ∑nj=1 (y, e j )e j ) = (x, Sy). So T = S2 = S∗ S.



Uniqueness of a nonnegative square root will be proved in Theorem 7.51. Definition 7.49. Given vectors x1 , . . . , xk in an inner product space V , we define the corresponding Gram matrix, or Gramian, G = (gi j ) by gi j = (xi , x j ). Clearly, a Gramian is always a Hermitian matrix. The next results connects Gramians with positivity. Theorem 7.50. Let V be a complex n-dimensional inner product space. A necessary and sufficient condition for a k × k, with k ≤ n, Hermitian matrix G to be nonnegative definite is that it be the Gram matrix of k vectors in V . Proof. Assume G⎛is a ⎞ Gramian, i.e., there exist vectors x1 , . . . , xk ∈ V for which ξ1 ⎜ ⎟ gi j = (xi , x j ). Let ⎝ ... ⎠ ∈ Ck . Then we compute

ξk (Gξ , ξ ) = ∑ni=1 ∑nj=1 gi j ξ j ξ i = ∑ni=1 ∑nj=1 (xi , x j )ξ j ξ i = (∑ni=1 ξ i xi , ∑nj=1 ξ i x j ) ≥ 0, which shows that G is nonnegative. To prove the converse, assume G is nonnegative. By Theorem 7.34, there exists a k × k unitary matrix U, with columns u1 , . . . , uk , for which U ∗ GU = diag (δ1 , . . . , δk ), with δi ≥ 0. Let γi be the nonnegative square root of δi . We define Γ = diag (γ1 , . . . , γk ). Thus we have U ∗ GU = Γ ∗Γ , or with R = Γ U −1 , G = R∗ R. Let r1 , . . . , rk be the columns of R. Then it follows that gi j = (r j , ri ). i.e., G is the Gramian of h vectors in Ck . To show that G is also the Gramian of k vectors in V , we choose an arbitrary orthonormal basis e1 , . . . , en in V . We define a map S : Ck −→ V by Sui = ei and extend it linearly. Clearly, S is isometric, and with xi = γi ei ∈ V , it follows that G is also the Gramian of x1 , . . . , xk .  Effectively, in the proof of Proposition 7.48, we have constructed a nonnegative square root. We formalize this. Theorem 7.51. A nonnegative operator T has a unique nonnegative square root. Proof. The existence of a nonnegative square root has been proved in Proposition 7.48.

182

7 Inner Product Spaces

To prove uniqueness, let λ1 ≥ · · · ≥ λn be the eigenvalues of T and B = {e1 , . . . , en } an orthonormal basis in U made out of eigenvectors. Let A be an arbitrary nonnegative square root of T , i.e., T = A2 . We compute 1

1

0 = (λi I − A2 )ei = (λi 2 I + A)(λi2 I − A)ei . 1

1

Now, in case λi > 0, the operators λi 2 I + A are invertible, and hence Aei = λi 2 ei . In case λi = 0 we compute Aei 2 = (Aei , Aei ) = (A2 ei , ei ) = (Tei , ei ) = 0. So Aei = 0. Thus a nonnegative square root is completely determined by T , whence uniqueness. 

7.3.8 Partial Isometries Definition 7.52. Let U1 , U2 be inner product spaces. An operator U : U1 −→ U2 is called a partial isometry if there exists a subspace M ⊂ U1 such that  Ux =

x, x ∈ M , 0,

x ⊥ M.

The space M is called the initial space of U and the image of U, UM , the final space. In the following proposition we collect the basic facts on partial isometries. Proposition 7.53. Let U1 , U2 be inner product spaces, and U : U1 −→ U2 a linear transformation. Then 1. U is a partial isometry if and only if U ∗U is an orthogonal projection onto the initial space of U. 2. U is a partial isometry with initial space M if and only if U ∗ is a partial isometry with initial space UM . 3. UU ∗ is the orthogonal projection on the final space of U and U ∗U is the orthogonal projection on the initial space of U. Proof. 1. Assume U is a partial isometry with initial space M . Let P be the orthogonal projection of U1 on M . Then for x ∈ M , we have (U ∗Ux, x) = Ux2 = x2 = (x, x) = (Px, x).

7.3 Operators in Inner Product Spaces

183

On the other hand, if x ⊥ M , then (U ∗Ux, x) = Ux2 = 0 = (Px, x). So ((U ∗U − P)x, x) = 0 for all x, and hence U ∗U = P. Conversely, suppose P = U ∗U is a, necessarily orthogonal, projection in U1 . Let M = {x|U ∗Ux = x}. Then for any x ∈ U1 , Ux2 = (U ∗Ux, x) = (Px, x) = Px2 . This shows that U is a partial isometry with initial space M . 2. Assume U is a partial isometry with initial space M . Note that we have U2 = ImU ⊕ KerU ∗ = UM ⊕ KerU ∗ . Obviously U ∗ |KerU ∗ = 0, whereas if x ∈ M , then U ∗Ux = Px = x. So U ∗Ux = x = Ux, i.e., U ∗ is a partial isometry with initial space UM . By the previous part, it follows that UU ∗ is the orthogonal projection on UM , the final space of U. 3. From Part 1, it follows that UU ∗ is the orthogonal projection on the initial space of U ∗ , which is UM , the final space of U. The rest follows by duality. 

7.3.9 The Polar Decomposition In analogy with the polar representation of a complex number in the form z = reiθ , we have a representation of an arbitrary linear transformation in an inner product space as the product of a unitary operator and a nonnegative one. Theorem 7.54. Let U1 , U2 be two inner product spaces and let T : U1 −→ U2 be a linear transformation. Then there exist partial isometries V : U1 −→ U2 and W : U2 −→ U1 for which 1

1

T = V (T ∗ T ) 2 = (T T ∗ ) 2 W.

(7.14)

The partial isometry V is uniquely defined if we require KerV = Ker T . Similarly, the partial isometry W is uniquely defined if we require KerW = Ker T ∗ . Proof. For x ∈ U1 , we compute 1

1

1

(T ∗ T ) 2 x2 = ((T ∗ T ) 2 x, (T ∗ T ) 2 x) = (T ∗ T x, x) = (T x, T x) = T x2 .

(7.15)

Note that we have 1

Im (T ∗ T ) 2 = Im T ∗ , 1

Ker (T ∗ T ) 2 = Ker T.

(7.16)

184

7 Inner Product Spaces

We define V : U1 −→ U2 by V (T ∗ T ) 2 x = T x, 1

V z = 0,

x ∈ U1 , z ∈ Ker T.

(7.17) 1

Recalling the direct sum representation U1 = Im (T ∗ T ) 2 ⊕ Ker T , it follows from 1 (7.15) that V is a well-defined partial isometry with Im (T ∗ T ) 2 as initial space 1 and Im T = Im (T T ∗ ) 2 as final space. The other representation in (7.14) follows by duality.  A remark on uniqueness is in order. It is clear from the construction of V that it is 1 uniquely determined on Im (T ∗ T ) 2 . Therefore V is uniquely determined if Ker T = 1 Ker (T ∗ T ) 2 = {0}. Noting that the definition of V on Ker T was a choice we made, it follows that we could make a different choice, provided Im T has a nontrivial orthogonal complement. Since (Im T )⊥ = Ker T ∗ , Ker T ∗ = {0} is also a sufficient condition for the uniqueness of V . In general, if dim U2 ≥ dim U1 , we can assume V to be a, not necessarily unique, isometry. If dim U1 ≥ dim U2 , we can assume W to be a, not necessarily unique, coisometry, i.e., W ∗ an isometry. In case we have dim U1 = dim U2 , we can assume both V and W to be unitary. In this case, V and W are uniquely determined if and only if T is invertible.

7.4 Singular Vectors and Singular Values In this section we study the basic properties of singular vectors and singular values. We use this to give a characterization of singular values as approximation numbers. These results will be applied, in Chapter 12, to Hankel norm approximation problems. Starting from a polar decomposition of a linear transformation, and noting that the norm of a unitary operator, or for that matter also of isometries and nontrivial 1 1 partial isometries, is 1, it seems plausible that the operators (T ∗ T ) 2 and (T T ∗ ) 2 provide a measure of the size of T . Proposition 7.55. Let U1 , U2 be two inner product spaces and let T : U1 −→ U2 be 1 1 a linear transformation. Let (T ∗ T ) 2 and (T T ∗ ) 2 be the nonnegative square roots 1 1 of T ∗ T and T T ∗ respectively. Then (T ∗ T ) 2 and (T T ∗ ) 2 have the same nonzero eigenvalues, including multiplicities. 1

Proof. Let μ be a nonzero eigenvalue of (T ∗ T ) 2 and x = 0 a corresponding eigenvector. Clearly, this implies that also T ∗ T x = μ 2 x. Define y = Tμx . We proceed to compute T T ∗y = T T ∗

Tx T ∗ Tx T = (T T x) = (μ 2 x) = μ 2 = μ 2 y. μ μ μ μ

7.4 Singular Vectors and Singular Values

185

Note that x = 0 implies y = 0, and this shows that μ 2 is an eigenvalue of T T ∗ . In turn, 1 1 this implies that μ is an eigenvalue of (T T ∗ ) 2 . Let now Vi = {x ∈ U1 |(T ∗ T ) 2 x = 1 μi x} and Wi = {y ∈ U2 |(T T ∗ ) 2 y = μi y}. We define maps Φi : Vi −→ Wi and Ψi : Wi −→ Vi by Tx Φi x = , x ∈ Vi , μi (7.18) T ∗y Ψi y = , y ∈ Wi . μi We compute

 Φi x2 =

Tx Tx , μi μi

 =

1 ∗ (T T x, x) = x2 , μi2

i.e., Φi is an isometry. The same holds for Ψi by symmetry. Finally, it is easy to check that Ψi Φi = I and ΦiΨi = I, i.e., both maps are unitary. Hence, Vi , Wi have 1 the same dimension, and so the multiplicity of μi as an eigenvalue of (T ∗ T ) 2 and 1  (T T ∗ ) 2 is the same. This leads us to the following definition. Definition 7.56. Let T : U1 −→ U2 be a linear transformation. A pair of vectors {φ , ψ }, with φ ∈ U1 and ψ ∈ U2 , is called a Schmidt pair of T , corresponding to the nonzero singular value μ , if T φ = μψ , T ∗ ψ = μφ ,

(7.19)

holds. We also refer to φ , ψ as the singular vectors of T and T ∗ respectively. The minimax principle leads to the following characterization of singualr values. Theorem 7.57. Let U1 , U2 be inner product spaces and let T : U1 −→ U2 be a linear transformation. Let μ1 ≥ · · · ≥ μn ≥ 0 be its singular values. Then

μk =

T x . min max {M |codim M =k−1} x∈M x

Proof. The μi2 are the eigenvalues of T ∗ T . Applying the minimax principle, we have

μk2 = min{M |codim M =k−1} maxx∈M = min{M |codim M =k−1} maxx∈M which is equivalent to the statement of the theorem.

(T ∗ T x, x) (x, x) T x2 , x2 

186

7 Inner Product Spaces

With the use of the polar decomposition and the spectral representation of self-adjoint operators, we get a convenient representation of arbitrary operators. Theorem 7.58. Let T : U1 −→ U2 be a linear transformation and let μ1 ≥ · · · ≥ μr be its nonzero singular values and let {φi , ψi } be the corresponding orthonormal sets of Schmidt pairs. Then for every pair of vectors x ∈ U1 , y ∈ U2 , we have r

T x = ∑ μi (x, φi )ψi

(7.20)

i=1

and r

T ∗ y = ∑ μi (y, ψi )φi .

(7.21)

i=1

Proof. Let μ1 ≥ · · · ≥ μr > 0 the nonzero singular values and {φi , ψi } the corresponding Schmidt pairs. We have seen that with V defined by V φi = ψi , and of course V ∗ ψi = φi , we have 1 T = V (T ∗ T ) 2 . Now, for x ∈ U1 , we have r

(T ∗ T ) 2 x = ∑ μi (x, φi )φi , 1

i=1

and hence 1

T x = V (T ∗ T ) 2 x = V

r

r

i=1

i=1

∑ μi (x, φi )φi = ∑ μi (x, φi )ψi .

T∗

can be derived analogously, or alternatively, can be The other representation for obtained by computing the adjoint of T using (7.20).  The availability of the previously obtained representations for T and T ∗ leads directly to an extremely important characterization of singular values in terms of approximation properties. The question we address is the following. Given a linear transformation of rank m between two inner product spaces, how well can it be approximated, in the operator norm, by transformations of lower rank. We have the following characterization. Theorem 7.59. Let T : U1 −→ U2 be a linear transformation of rank m. Let μ1 ≥ · · · ≥ μm be its nonzero singular values. Then

μk = inf{T − T  | rank T ≤ k − 1}.

(7.22)

Proof. By Theorem 7.58, we may assume that T x = ∑ni=1 μi (x, φi )ψi . We define a linear transformation T by T x = ∑k−1 i=1 μi (x, φi )ψi . Obviously rank(T ) = k − 1 and n (T − T )x = ∑i=k μi (x, φi )ψi , and therefore we conclude that T − T  = μk . Conversely, let us assume rank(T ) = k − 1, i.e., codimKer T = k − 1. By Theorem 7.57, we have

7.5 Unitary Embeddings

187

T x (T − T )x μk ≤ max = max ≤ T − T . x x∈Ker T x x∈Ker T



7.5 Unitary Embeddings The existence of nonnegative square roots is a central tool in modern operator theory. In this section we discuss some related embedding theorems. The basic question we address is, given a linear transformation A in an inner product space U, when  can  it be embedded in a 2 × 2 block unitary operator matrix of the form V=

A B CD

.

This block matrix is to be considered   initially   to be an  operator defined on the  A B x Ax + By = . It is clear that if V given inner product space U ⊕U, by CD y Cx + Dy     x x before is unitary, then, obseving that P defined by P = is an orthogonal y

0

projection, it follows that A = PV |U⊕{0} , and we have A = PV |U⊕{0}  ≤ V  = 1, that is, A is necessarily a contraction. The following theorem focuses on the converse. Theorem 7.60. A linear transformation A can be embedded in a 2 × 2 block unitary matrix if and only if it is a contraction. Proof. That embeddability in a unitary matrix implies that A is contractive has been shown before. Thus, assume A is contractive, i.e., Ax ≤ x for all x ∈ U. Since x2 − Ax2 = (x, x) − (Ax, Ax) = (x, x) − (A∗ Ax, x) 1 = ((I − A∗ A)x, x) = (I − A∗ A) 2 x2 . Here we used the fact that the nonnegative operator I − A∗ A has a unique nonnega1 ∗ tive square root. Thisimplies given  that, setting C = (I − A A) 2 , the   transformation A C

A

is isometric, i.e., satisfies ( A∗ C∗ ) = I. C   1 A∗ Similarly, using the requirement that ( A B ) = I, we set B = (I − AA∗ ) 2 . ∗

by the block matrix

B

Computing now

188

7 Inner Product Spaces



I 0 0I



 =

1

A (I − AA∗ ) 2 1 (I − A∗ A) 2 D

1



1

(I − A∗ A) 2 A∗ 1 (I − AA∗ ) 2 D∗

1

 , 1

we have (I − A∗ A) 2 A∗ + D(I − AA∗ ) 2 = 0. Using the equality (I − A∗ A) 2 A∗ = 1 1 A∗ (I − AA∗ ) 2 , we get (A∗ + D)(I − AA∗ ) 2 = 0. This indicates that we should choose D = −A∗ . It remains to check that   1 A (I − AA∗ ) 2 (7.23) V= 1 (I − A∗ A) 2 −A∗ 

is a required embedding.

The embedding (7.23) is not unique. In fact, if K and L are unitary operators in   1

U, then

A (I − AA∗ ) 2 L 1 K(I − A∗ A) 2 −KA∗ L

parametrize all such embeddings.

The embedding presented in the previous theorem is possibly redundant, since it requires a space of twice the dimension of the original space U. For example, if A is unitary to begin with, then no extension is necessary. Thus the dimension of the minimal space where a unitary extension is possible seems to be related to the distance of the contraction A from unitary operators. This is generally measured by 1 1 the ranks of the operators (I − A∗ A) 2 and (I − AA∗ ) 2 . The next theorem handles this situation. However, before embarking on that route, we give an important dilation result. This is an extension of a result originally obtained in Halmos (1950). That construction played a central role, via the Sch¨affer matrix, see Sz.-Nagy and Foias (1970), in the theory of unitary dilations.   A B Theorem 7.61. Given a contraction A in Cn , there exists a unitary matrix , CD

with B injective and C surjective, if and only if this matrix has the form 

1

A (I − AA∗ ) 2 V 1 U(I − A∗ A) 2 −UA∗V

 ,

(7.24) 1

where U : Cn −→ C p is a partial isometry with initial space Im (I − A∗ A) 2 and 1 where V : C p −→ Cn is an isometry with final space Im (I − AA∗ ) 2 . Proof. Let P

1

Im (I−AA∗ ) 2

and P

1

Im (I−A∗ A) 2

be the orthogonal projections on the

appropriate spaces. Assume U,V are partial isometries as described in the theorem. Then we have UU ∗ = I,

U ∗U = P 1, Im (I−A∗ A) 2

V ∗V = I,

VV ∗ = P 1. Im (I−AA∗ ) 2

(7.25)

7.5 Unitary Embeddings

189

These identities imply the following: VV ∗ (I − AA∗) 2 = (I − AA∗ ) 2 , 1

1

1

1

(I − AA∗) 2 VV ∗ = (I − AA∗) 2 .

(7.26)

We compute now the product 

A (I − AA∗ ) 2 V 1 U(I − A∗ A) 2 −UA∗V 1



A∗ (I − A∗ A) 2 U ∗ 1 V ∗ (I − AA∗) 2 −V ∗ AU ∗ 1

 .

We use the identities in (7.26). So 1

1

1

1

AA∗ + (I − AA∗) 2 VV ∗ (I − AA∗ ) 2 = AA∗ + (I − AA∗ ) 2 (I − AA∗ ) 2 = I. Next, 1

1

1

1

A(I − A∗A) 2 U ∗ − (I − AA∗) 2 VV ∗ AU ∗ = A(I − A∗ A) 2 U ∗ − (I − AA∗ ) 2 AU ∗ = 0. Similarly, 1

1

1

1

U(I − A∗ A) 2 A∗ − UA∗VV ∗ (I − AA∗ ) 2 = U[(I − A∗ A) 2 A∗ − A∗ (I − AA∗ ) 2 ] = 0. Finally, 1

1

U(I − A∗ A) 2 (I − A∗A) 2 U ∗ +UA∗VV ∗ AU ∗ = U(I − A∗ A)U ∗ +UA∗ AU ∗ = UU ∗ = I. This shows that the matrix in (7.24) is indeed unitary. Conversely, suppose we are given a contraction A. If C satisfies A∗ A + C∗C = I, then C∗C = I − A∗ A. This in turn implies that for every vector x, we have 1

Cx2 = (I − A∗ A) 2 x2 . 1

Therefore, there exists a partial isometry U, defined on Im (I − A∗A) 2 , for which 1

C = U(I − A∗ A) 2 .

(7.27)

Obviously, ImC ⊂ ImU. Since C is surjective this forces U to be surjective. Thus UU ∗ = I and U ∗U = P 1 . In a similar fashion, we start from the equality ∗ Im (I−A A) 2

AA∗ + BB∗ = I and see that for every x,

1

B∗ x2 = (I − AA∗ ) 2 x2 .

190

7 Inner Product Spaces

So we conclude that there exists a partial isometry V ∗ with initial space Im (I − 1 AA∗ ) 2 for which 1 B∗ = V ∗ (I − AA∗) 2 , or 1

B = (I − AA∗ ) 2 V. Since B is injective, so is V . So we

get V ∗V

= I,

(7.28)

and VV ∗

Next we use the equality DB∗ + CA∗ = 0, to get 1

=P 1. Im (I−AA∗ ) 2

1

1

DV ∗ (I − AA∗ ) 2 + U(I − A∗ A) 2 A∗ = (DV ∗ + UA∗ )(I − AA∗ ) 2 = 0, so DV ∗ +UA∗ |

1

1

Im (I−AA∗ ) 2

1

= 0. Now we have Cn = Im (I − AA∗ ) 2 ⊕ Ker (I − AA∗ ) 2 . 1

1

1

Moreover, the identity A(I − A∗ A) 2 = (I − AA∗ ) 2 A implies AIm (I − A∗ A) 2 ⊂ 1 1 1 Im (I − AA∗ ) 2 as well as AKer (I − A∗ A) 2 ⊂ Ker (I − AA∗ ) 2 . Similarly, A∗ Ker (I − 1 1 1 AA∗ ) 2 ⊂ Ker (I − A∗ A) 2 . Letting now x ∈ Ker (I − AA∗ ) 2 , then A∗ x ∈ Ker (I − 1 1 A∗ A) 2 . Since U is a partial isometry with initial space Im (I − A∗ A) 2 , necessarily 1 UA∗ x = 0. On the other hand V ∗ is a partial isometry with initial space (I − AA∗ ) 2 ∗ ∗ and hence V ∗ | 1 = 0. So DV + UA | 1 = 0, and hence we can Ker (I−AA∗ ) 2 Ker (I−AA∗ ) 2 ∗ ∗ ∗ conclude that DV + UA = 0. Using the fact that V V = I, this implies D = −UA∗V.

(7.29)

It is easy to check now, as in the sufficiency part, that with B,C, D satisfying (7.27), (7.28), and (7.29) respectively, all other equations are satisfied. 

7.6 Exercises 1. Show that the rational function q(z) = ∑nk=−n qk zk is nonnegative on the unit circle if and only if q(z) = p(z)p (z) for some polynomial p(z). Here p (z) = p(z−1 ). This theorem, the simplest example of spectral factorization, is due to Fej´er. 2. Show that a polynomial q(z) is nonnegative on the imaginary axis if and only if for some polynomial p(z), we have q(z) = p(z)p(−z). 3. Given vectors x1 , . . . , xk in an inner product space U , the Gram determinant is defined by G(x1 , . . . , xk ) = det((xi , x j )). Show that x1 , . . . , xk are linearly independent if and only if detG(x1 , . . . , xk ) = 0. 4. Let X be an inner product space of functions. Let f0 , f1 , . . . be a sequence of linearly independent functions in X and let φ0 , φ1 , . . . be the sequence of

7.6 Exercises

191

functions obtained from it by way of the Gram–Schmidt orthonormalization procedure. a. Show that

   ( f0 , f 0 ) . . . . ( fn , f 0 )      . .... .     . .... .   φn (x) =  .   . .... .    ( f0 , fn−1 ) . . . . ( fn , fn−1 )     f0 (x) . . . . f n (x) 

b. Let n

φn (x) = ∑ cn,i fi (x), i=0

and define D−1 = 1,    ( f , f ) . . . . ( fn , f )  0   0 0  . .... .     .... .   . Dn =  .  . .... .     . .... .    ( f0 , fn ) . . . . ( f n , fn )  Show that

 cn,n =

Dn−1 Dn

1 2

.

5. In the space of real polynomials R[x], an inner product is defined by (p, q) =

6. 7.

8. 9.

 1 −1

p(x)q(x)dx.

Apply the Gram-Schmidt orthogonalization procedure to the sequence of polynomials 1, x, x2 , . . . . Show that up to multiplicative constants, the resulting n 2 n . polynomials coincide with the Legendre polynomials Pn (x) = 2n1n! d (xdx−1) n Show that the rank of a skew-symmetric matrix in a real inner product space is even. Let K be a skew-Hermitian operator, i.e., K ∗ = −K. Show that U = (I + K)−1 (I − K) is unitary. Show that every unitary operator is of the form U = λ (I + K)−1(I − K), where λ is a complex number of absolute value 1. Show that if A and B are normal and Im A ⊥ Im B, then A + B is normal. Show that A is normal if and only if for some unitary U, we have A∗ = AU.

192

7 Inner Product Spaces

10. Let N be a normal operator and let AN = NA. Show that AN ∗ = N ∗ A. 11. Given a normal operator N and any positive integer k, show that there exists an operator A satisfying Ak = N. 12. Show that a circulant matrix A ∈ Cn×n is normal. 13. Let T be a real orthogonal matrix, i.e., it satisfies T˜ T = I. Show that T is similar to a block diagonal matrix diag (I, R1 , . . . , Rk ), where  Ri =

cos θi sin θi − sin θi cos θi

 .

14. Show that a linear operator A in an inner product space having eigenvalues λ1 , . . . , λn is normal if and only if Trace(A∗ A) = ∑ni=1 |λi |2 . 15. Given contractive operators A and B in finite-dimensional inner product spaces U1 andU2 respectively, show that there exists an operator X : U2 −→ U1 for  which

AX 0 B

1

1

is contractive if and only if X = (I − AA∗ ) 2 C(I − B∗ B) 2 and

C : U2 −→ U1 is a contraction. 16. Define the Jacobi tridiagonal matrices by ⎞ 0 a1 −b1 . . ⎜ −c a . . ⎟ ⎟ ⎜ 1 2 ⎟ ⎜ Jk = ⎜ . . . . . ⎟, ⎟ ⎜ ⎝ . . . −bk−1 ⎠ 0 . . −ck−1 ak ⎛

assuming bi , ci > 0. Let Dk (λ ) = det(λ I − Jk ). a. Show that the following recurrence relation is satisfied: Dk (λ ) = (λ − ak )Dk−1 (λ ) − bk−1ck−1 Dk−2 (λ ). b. Show that Jk is similar to a self-adjoint transformation. 17. Singular value decomposition. Let A ∈ Cm×n and let σ1 ≥ · · · ≥ σr be its nonzero singular values. Show that there exist unitary matrices U ∈ Cm×m and V ∈ Cn×n such that   D0 A= , 0 0 where D = diag (σ1 , . . . , σr ). Compare   with Theorem 7.58. 18. Show that the eigenvalues of

0 A A∗ 0

are σ1 , . . . , σn , −σ1 , . . . , −σn .

7.7 Notes and Remarks

193

7.7 Notes and Remarks Most of the results in this chapter go over in one way or another to the context of Hilbert spaces, where, in fact, most of them were proved initially. Young (1988) is a good, very readable modern source for the basics of Hilbert space theory. The spectral theorem in Hilbert space was one of the major early achievements of functional analysis and operator theory. For an extensive source on these topics and the history of the subject, see Dunford and Schwartz (1958, 1963). Singular values were introduced by E. Schmidt in 1907. Because of the importance of singular values as approximation numbers, equation (7.22) is sometimes taken as the definition of singular values. In this connection, see Young (1988).

Chapter 8

Tensor Products and Forms

8.1 Introduction In this chapter, we discuss the general theory of quadratic forms and the notion of congruence and its invariants, as well as the applications of the theory to the analysis of special forms. We will focus primarily on quadratic forms induced by rational functions, most notably the Hankel and Bezout forms, because of their connection to system-theoretic problems such as stability and signature symmetric realizations. These forms use as their data different representations of rational functions, namely power series and coprime factorizations respectively. But we will discuss also the partial fraction representation in relation to the computation of the Cauchy index of a rational function and the proof of the Hermite–Hurwitz theorem and the continued fraction representation as a tool in the computation of signatures of Hankel matrices as well as in the problem of Hankel matrix inversion. Thus, different representations of rational functions, i.e., different encodings of the information carried by a rational function, provide efficient starting points for different methods. The results obtained for rational functions will be applied to root-location problems for polynomials in the next chapter. In the latter part of the chapter, we approach the study of bilinear forms from the point of view of tensor products and their representations as spaces of homomorphisms. This we specialize to the tensor product of polynomial and rational models, emphasizing concrete, functional identifications for the tensor products. Since polynomial models carry two natural structures, namely that of vector spaces over the underlying field F as well as that of modules over the polynomial ring F[z], it follows that there are two related tensor products. The relation between the two tensor products illuminates the role of the Bezoutians and the Bezout map. In turn, this leads to the characterization of the maps that intertwine two polynomial models, yielding a generalization of Theorem 5.17.

P.A. Fuhrmann, A Polynomial Approach to Linear Algebra, Universitext, DOI 10.1007/978-1-4614-0338-8 8, © Springer Science+Business Media, LLC 2012

195

196

8 Tensor Products and Forms

8.2 Basics 8.2.1 Forms in Inner Product Spaces Given a linear transformation T in a finite-dimensional complex inner product space U , we define a field-valued function φ on U × U by

φ (x, y) = (T x, y).

(8.1)

Clearly, φ is linear in the variable x and antilinear in the variable y. Such a function will be called a sesquilinear form or just a form. If the field is the field R of real numbers, then a form is actually linear in both variables, i.e., a bilinear form. It might seem that the forms defined by (8.1) are rather special. This is not the case, as is seen from the following. Theorem 8.1. Let U be a finite-dimensional complex inner product space. Then φ is a form on U if and only if there exists a linear transformation T in U such that (8.1) holds. The operator T is uniquely determined by φ . Proof. We utilize Theorem 7.14 on the representation of linear functionals on inner product spaces. Thus, if we fix a vector y ∈ U , the function φ (x, y) is a linear functional, hence given by an inner product with a vector ηy , that is, φ (x, y) = (x, ηy ). It is clear that ηy depends linearly on y. Therefore, there exists a linear transformation S in U for which ηy = Sy. We complete the proof by defining T = S∗ . To see uniqueness, assume T1 , T2 represent the same form, i.e., (T1 x, y) = (T2 x, y). This implies ((T1 − T2 )x, y) = 0, for all x, y ∈ U . Choosing y = (T1 − T2 )x, we get (T1 − T2 )x = 0 for all x ∈ U . This shows that T2 = T1 .  Suppose we choose a basis B = { f 1 , . . . , fn } in U . Then arbitrary vectors in U can be written as x = ∑nj=1 ξ j f j and y = ∑ni=1 ηi fi . In terms of the coordinates, we can compute the form by

φ (x, y) = (T x, y) = (T ∑nj=1 ξ j f j , ∑ni=1 ηi fi ) = ∑nj=1 ∑ni=1 ξ j ηi (T f j , fi ) = ∑nj=1 ∑ni=1 φi j ξ j ηi , where we define φi j = (T f j , fi ). We call the matrix (φi j ) the matrix representation of the form φ in the basis B, and denote this matrix by [φ ]B B . Using the standard inner product in Cn , we can write B B φ (x, y) = ([φ ]B B [x] , [x] ).

Next, we consider how the matrix of a quadratic form changes with a change of  B basis in U . Let B = {g1, . . . , gn } be another basis in U . Then [x]B = [I]B B [x] and therefore

8.2 Basics

197 







B B B B B B B φ (x, y) = ([φ ]B B [x] , [y] ) = ([φ ]B [I]B [x] , [I]B [y] ) 













∗ B B B B B B B = (([I]B B ) [φ ]B [I]B [x] , [y] ) = ([φ ]B  [x] , [y] ),

which shows that







B ∗ B B [ φ ]B B = ([I]B ) [φ ]B [I]B ,

(8.2)

i.e., in a change of basis, the matrix of a quadratic form changes by congruence. The next result clarifies the connection between the matrix representation of a form and the matrix representation of the corresponding linear transformation. We say that a complex square matrix A1 is congruent to a matrix A if there exists a nonsingular matrix R such that A1 = R∗ AR. In case we deal with the real field, the ˜ Hermitian adjoint R∗ is replaced by the transpose R. The following proposition is standard, and we omit the proof. Proposition 8.2. In the ring of square real or complex matrices, congruence is an equivalence relation. A quadratic form in a complex inner product space U is a function of the form

φˆ (x) = φ (x, x),

(8.3)

where φ (x, y) is a symmetric bilinear form on U . We say that a form on U is a symmetric form if we have φ (x, y) = φ (y, x) for all x, y ∈ U and a Hermitian form if we have the additional property

φ (x, y) = φ (y, x). Proposition 8.3. A form φ on a complex inner product space U is Hermitian if and only if there exists a Hermitian operator T in U for which φ (x, y) = (T x, y). Proof. Assume T is Hermitian, i.e., T ∗ = T . Then

φ (x, y) = (T x, y) = (x, Ty) = (Ty, x) = φ (y, x). Conversely, assume φ (x, y) = φ (y, x). Let T be the linear transformation for which φ (x, y) = (T x, y). We compute (T x, y) = φ (x, y) = φ (y, x) = (Ty, x) = (x, Ty), and this for all x, y ∈ U . This shows that T is Hermitian.



Proposition 8.4. Let U be a complex inner product space and φ a Hermitian form on U . Let T be the uniquely defined linear transformation in U for which φ (x, y) = B∗ (T x, y), and let B = { f1 , . . . , fn } be a basis for U . Then [φ ]B B = [T ]B .

198

8 Tensor Products and Forms ∗

Proof. Let B ∗ = {g1 , . . . , gn } be the basis dual to B. Then [T ]B B = (ti j ) where the ti j are defined through T f j = ∑nk=1 tk j gk . Computing  (T f j , fi ) =



n

∑ tk j gk , fi

k=1

=

n

∑ tk j δik = ti j ,

k=1



the result follows.

We are interested in the possibility of reducing a Hermitian form, using congruence transformations, to a diagonal form. Theorem 8.5. Let φ be a Hermitian form on a finite-dimensional complex inner product space U . Then there exists an orthonormal basis B = {e1 , . . . , en } in U for which [φ ]B B is diagonal with real entries. Proof. Let T : U −→ U be the Hermitian operator representing the form φ . By Theorem 7.33, there exists an orthonormal basis B = {e1 , . . . , en } in U consisting of eigenvectors of T corresponding to the real eigenvalues λ1 , . . . , λn . With respect to this basis, the matrix of the form is given by

φi j = (Te j , ei ) = λ j (e j , ei ) = λ j δi j .



Just as positive operators are a subset of self-adjoint operators, so positive forms are a subset of Hermitian forms. We say that a form φ on a complex inner product space U is nonnegative if it is Hermitian and φ (x, x) ≥ 0 for all x ∈ U . We say that a form φ on an inner product space U is positive if it is Hermitian and φ (x, x) > 0 for all 0 = x ∈ U . Corollary 8.6. Let φ be a nonnegative Hermitian form on the complex inner product space U . Then there exists an orthogonal basis B = { f1 , . . . , fn } for which the matrix [φ ]B B is diagonal with 

φii =

1, 0,

0 ≤ i ≤ r, r < i ≤ n.

Proposition 8.7. A form φ on a complex inner product space U is positive if and only if there exists a positive operator P for which φ (x, y) = (Px, y). Of course, to check positivity of a form, it suffices to compute all the eigenvalues and check them for positivity. However, computing eigenvalues is not, in general, an algebraic process. So our next goal is to search for a computable criterion for positivity.

8.2 Basics

199

8.2.2 Sylvester’s Law of Inertia If we deal with real forms, then due to the different definition of congruence, we cannot apply the complex result directly. The proof of Theorem 8.5 is particularly simple. However, it uses the spectral theorem, and that theorem is constructive only if we have access to the eigenvalues of the appropriate self-adjoint operator. Generally, this is not the case. Thus we would like to re-prove the diagonalization by congruence result in a different way. It is satisfying that this can be done algebraically, and the next theorem, known as the Lagrange reduction method, accomplishes this. Theorem 8.8. Let φ be a symmetric bilinear form on a real inner product space U . Then there exists a basis B = { f1 , . . . , fn } of U for which [φ ]B B is diagonal. Proof. Let φ (x, y) be a symmetric bilinear form on U . We prove the theorem by induction on the dimension n of U . If n = 1, then obviously [φ ]B B , being a 1 × 1 matrix, is diagonal. Assume we have proved the theorem for spaces of dimension ≤ n − 1. Let now φ be a symmetric bilinear form in an n-dimensional space U . Assume B = { f1 , . . . , fn } is an arbitrary basis of U . We consider two cases. In the first, we assume that not all the diagonal elements of [φ ]B B are zero, i.e., there exists an index i for which φ ( f i , f i ) = 0. The second case is the complementary one, in which all the φ ( fi , fi ) are zero. Case 1: Without loss of generality, using symmetric transpositions of rows and columns, we will assume that φ ( f1 , f1 ) = 0. We define a new basis B  = { f1 , . . . , fn }, where ⎧ i = 1, f , ⎪ ⎪ ⎨ 1  fi = φ ( f1 , fi ) ⎪ ⎪ ⎩ fi − f1 , 1 < i ≤ n. φ ( f1 , f1 ) Clearly, B  is indeed a basis for U and

φ ( f1 , fi ) =

⎧ ⎨ φ ( f1 , f1 ), i = 1, ⎩

1 < i ≤ n.

0,

Thus, with the subspace M = L( f2 , . . . , fn ), we have  [φ ]

B B

=



φ ( f1 , f1 )

0

0

[φ |M]B0

B

.

0

Here φ |M is the restriction of φ to M and B0 = { f2 , . . . , fn } is a basis for M. The proof, in this case, is completed using the induction hypothesis.

200

8 Tensor Products and Forms

Case 2: Assume now that φ ( fi , fi ) = 0 for all i. If φ ( fi , f j ) = 0 for all i, j, then φ is the zero form, and hence [φ ]B B is the zero matrix, which is certainly diagonal. So we assume there exists a pair of distinct indices i, j for which φ ( fi , f j ) = 0. Without loss of generality, we will assume that i = 1, j = 2. To simplify notation we will write a = φ ( f1 , f2 ). First, we note that defining f1 = f1 +2 f2 and f2 = f1 −2 f2 , we have 1 a φ ( f1 , f1 ) = φ ( f1 + f2 , f1 + f2 ) = , 4 2 1 φ ( f1 , f2 ) = φ ( f1 + f2 , f1 − f2 ) = 0, 4 1 a φ ( f2 , f2 ) = φ ( f1 − f2 , f1 − f2 ) = − . 4 2 We choose the other vectors in the new basis to be of the form fi = fi − αi f1 − βi f2 , with the requirement that for i = 3, . . . , n, we have φ ( f1 , fi ) = φ ( f2 , fi ) = 0. Therefore we get the pair of equations 0 = φ ( f1 + f2 , fi − αi f1 − βi f2 ) = φ ( f1 , fi ) + φ ( f2 , fi ) − αi φ ( f2 , f1 ) − βi φ ( f2 , f1 ) 0 = φ ( f1 − f2 , fi − αi f1 − βi f2 ) = φ ( f1 , fi ) − φ ( f2 , fi ) + αi φ ( f2 , f1 ) − βi φ ( f2 , f1 ). This system has the solution αi = { f1 , . . . , fn }, where

φ ( f2 , fi ) a

and βi =

φ ( f1 , fi ) . a

f1 + f2 , 2 f1 − f2 , f2 = 2 φ ( f2 , fi ) φ ( f1 , fi ) f1 − f2 . fi = fi − a a

We define now B  =

f1 =

(8.4)

It is easy to check that B  is a basis for U . With respect to this basis we have the block diagonal matrix ⎞

⎛a



[φ ]B B

0 ⎜2 a ⎜0 − =⎜ 2 ⎝

B

[φ |M]B0

⎟ ⎟ ⎟. ⎠

0

Here B0 = { f3 , . . . , fn } and M = L( f3 , . . . , fn ). We complete the proof by applying the induction hypothesis to φ |M.  Corollary 8.9. Let φ be a quadratic form defined on a real inner product space U . Let B = { f1 , . . . , fn } be an arbitrary basis of U and let B  = {g1 , . . . , gn } be a

8.2 Basics

201

basis of U that diagonalizes φ . Let φi = φ (gi , gi ). If ⎞ ξ1 ⎜ ⎟ [x]B = ⎝ ... ⎠ , ⎛

⎞ η1 ⎜ ⎟ = ⎝ ... ⎠ , ⎛

[x]B



ξn

ηn



and [I]B B = (ai j ), then

n

n

i=1

j=1

φ (x, x) = ∑ φi ( ∑ ai j ξ j )2 .

(8.5)

Proof. We have 















B B B B B B B B B B φ (x, x) = ([φ ]B B [x] , [x] ) = ([φ ]B  [x] , [x] ) = ([φ ]B [I]B [x] , [[I]B x] ). 



n B Also [x]B = [I]B B [x] implies ηi = ∑ j=1 ai j ξ j . So

  n n  B B φ (x, x) = [φ ]B = ∑ φi ηi2 = ∑ φi B  [x] , [x] i=1

i=1



n

∑ ai j ξ j

2 .

j=1

 We say that (8.5) is a representation of the quadratic form as a sum of squares. Such a representation is far from unique. In different representations, the diagonal elements as well as the linear forms may differ. We have, however, the following result, known as Sylvester’s law of inertia, which characterizes the congruence invariant. Theorem 8.10. Let U be a real inner product space and φ (x, y) a symmetric bilinear form on U . Then there exists a basis B = { f1 , . . . , fn } for U such that the matrix [φ ]B B is diagonal, with φ ( fi , f j ) = εi δi j , with ⎧ ⎨ 1, 0 ≤ i ≤ k, εi = −1, k < i ≤ r, ⎩ 0, r < i ≤ n. Moreover, the numbers k and r are uniquely determined by φ . Proof. By Theorem 8.8, there exists a basis B such that [φ ]B B is real and diagonal, with λ1 , . . . , λn the diagonal elements. Without loss of generality, we can assume 1 λi > 0 for 1 ≤ i ≤ k, λi < 0 for k < i ≤ r and λi = 0 for r < i ≤ n. We set μi = |λi | 2 and εi = sign λi . Hence, we have, for all i, λi = εi μi2 . We consider now the basis 1 B1 = {μ1−1 e1 , . . . , μn−1 en }. Clearly, with respect to this basis, the matrix [φ ]B B1 has the required form.

202

8 Tensor Products and Forms

To prove the uniqueness of k and r, we note first that r is equal to the rank of [ φ ]B B . So we have to prove uniqueness for k only. Suppose we have another basis B  = {g1 , . . . , gn } such that φ (g j , gi ) = εi δi j , where ⎧ ⎨ 1, 0 ≤ i ≤ k ,  εi = −1, k < i ≤ r, ⎩ 0, r < i ≤ n. Without loss of generality, we assume k < k . Given x ∈ U , we have x = ∑ni=1 ξi fi = ∑ni=1 ηi gi , that is, the ξi and ηi are the coordinates of x with respect to B and B  respectively. Computing the quadratic form, we get n

n

i=1

i=1

φ (x, x) = ∑ εi ξi2 = ∑ εi ηi2 , or

2 ξ12 + · · · + ξk2 − ξk+1 − · · · − ξr2 = η12 + · · · + ηk2 − ηk2 +1 − · · · − ηr2 .

This can be rewritten as 2 ξ12 + · · · + ξk2 + ηk2 +1 + · · · + ηr2 = η12 + · · · + ηk2 + ξk+1 + · · · + ξr2 .

(8.6)

Now we consider the set of equations ξ1 = · · · = ξk = ηk +1 = · · · = ηr = 0. We have here r − k + k = r − (k − k) < r equations. Thus there exists a nontrivial vector x for which not all the coordinates ξk+1 = · · · = ξr are zero. However, by equation (8.6), we get ξk+1 = · · · = ξr = 0, which is a contradiction. Thus we must have k ≥ k, and by symmetry we get k = k.  Definition 8.11. The numbers r and s = 2k − r determined by Sylvester’s law of inertia are called the rank and signature respectively of the quadratic form φ . We will denote the signature of a quadratic form by σ (φ ). Similarly, the signature of a symmetric (Hermitian) matrix A will be denoted by σ (A). The triple (π , ν , δ ), with r−s π = r+s 2 , ν = 2 , δ = n − r, is called the inertia of the form, and denoted by In (A). Corollary 8.12. 1. Two quadratic forms on a real inner product space U are congruent if and only if they have the same rank and signature. 2. Two n × n symmetric matrices are congruent if and only if they have the same rank and signature. Proposition 8.13. Let A be an m × m real symmetric matrix, X an m × n matrix of full row rank, and B the n × n symmetric matrix defined by ˜ B = XAX. Then the rank and signature of A and B coincide.

(8.7)

8.2 Basics

203

Proof. The statement concerning ranks is trivial. From (8.7) we obtain the two ˜ Let us choose a basis for Rn that is inclusions Ker B ⊃ Ker X and Im B ⊂ Im X. compatible with the direct sum decomposition ˜ Rn = Ker X ⊕ Im X.   In that basis, X = 0 X1 with X1 invertible and  B=

00 0 B1



 =

0 X˜1



  A 0 X1

or B1 = X˜1 AX1 . Thus σ (B) = σ (B1 ) = σ (A), which proves the theorem.



The positive definiteness of a quadratic form can be determined from any of its matrix representations. The following theorem gives a determinantal characterization for positivity. Theorem 8.14. Let φ be a symmetric bilinear form on an n-dimensional vector space U and let [φ ]B B = (φi j ) be its matrix representation with respect to the basis B. Then the following statements are equivalent: 1. φ is positive definite. ˜ 2. [φ ]B B = (φi j ) = X X, with X invertible and lower triangular. 3. All principal minors are positive, i.e.,  φ  11  .    .   .   φk1

. . . . .

. . . . .

. . . . .

 φ1k  .   .  > 0,  .   φkk 

for k = 1, . . . , n. Proof. (2) ⇒ (1) This is obvious. (3) ⇒ (2) We prove this by induction on k.For k  = 1, this is immediate. Assume it holds for all i = 1, . . . , k − 1 and that A =

A11 b b˜ d

is k × k with A11 ∈ R(k−1)×(k−1) positive.

By the induction hypothesis, there exists a matrix X ∈ R(k−1)×(k−1) for which A11 = X1 X˜1 , with X1 nonsingular. We write 

A11 b b˜ d



 =

X1 0 0 1



I X1−1 b 1 −1 d b˜ X



 1 0 X , 0 1

204

8 Tensor Products and Forms

 with

I v v˜ d



> 0, where v = X1−1 b. Using the computation 

I v v˜ d



 =

I 0 v˜ d



I 0 0e



I v 01



 =

 I v , v˜ e + vv ˜

i.e., e + vv ˜ = d and necessarily ˜ > 0, and setting e = ε 2 , it follows that   e =d − vv  I 0 ˜ where X = X1 0 . A = X X, 0 1

v˜ ε

(1) ⇒ (3) Assume all principal minors have positive determinants. Let (φi j ) be the matrix of the qudratic form. Let R be the matrix that reduces (φi j ) to upper triangular form. If we do not normalize the diagonal elements, then R is lower triangular with all elements on the diagonal equal to 1. Thus Rφ R˜ is diagonal with the determinants of the principal minors on the digonal. Since Rφ R˜ is positive by assumption, so is φ .  The same result holds for Hermitian forms, where the transpose is replaced by the Hermitian adjoint. Theorem 8.15. 1. Let v1 , . . . , vk ∈ Cn , λ1 , . . . , λk ∈ R, and let A be the Hermitian matrix ∑ki=1 λi vi v∗i . Then A is Hermitian congruent to Λ = diag (λ1 , . . . , λk , 0, . . . , 0). 2. If A1 and A2 are Hermitian and rank (A1 + A2 ) = rank (A1 ) + rank(A2 ), then

σ (A1 + A2 ) = σ (A1 ) + σ (A2).

Proof. 1. Extend {v1 , . . . , vk } to a basis {v1 , . . . , vn } of Cn . Let V be the matrix whose columns are v1 , . . . , vn . Then V Λ V ∗ = A. 2. Let λ1 , . . . , λk be the nonzero eigenvalues of A1 , with corresponding eigenvectors v1 , . . . , vk and let λk+1 , . . . , λk+l be the nonzero eigenvalues of A2 , with corresponding eigenvectors vk+1 , . . . , vk+l . Thus A1 = ∑ki=1 λi vi v∗i and A2 = ∗ ∑k+l i=k+1 λi vi vi . By our assumption, Im A1 ∩ Im A2 = {0}. Therefore, the vectors v1 , . . . , vk+l are linearly independent. Hence, by part (1), A1 + A2 is Hermitian congruent to diag (λ1 , . . . , λk+l , 0, . . . , 0).  In the sequel, we will focus our interest on some important quadratic forms induced by rational functions.

8.3 Some Classes of Forms We pass now from the general theory of forms to some very important examples. We will focus mainly on the Hankel and Bezoutian forms and their interrelations. These forms arise out of different representations of rational functions. Since positivity is

8.3 Some Classes of Forms

205

not essential to the principal results, we assume that the field F is arbitrary. Assume V is a finite-dimensional F-vector space. A billinear form φ (x, y) on V × V is called symmetric if, for all x, y ∈ V , we have

φ (x, y) = φ (y, x).

(8.8)

8.3.1 Hankel Forms We begin our study of special forms by focusing on Hankel operators and forms. Though our interest is mainly in Hankel operators with rational symbols, we need to enlarge the setting in order to give Kronecker’s characterization of rationality in terms of infinite Hankel matrices. Let g(z) = ∑nj=−∞ g j z j ∈ F((z−1 )) and let π+ and π− be the projections, defined in (1.23), that correspond to the direct sum representation F((z−1 )) = F[z] ⊕ z−1 F[[z−1 ]]. We define the Hankel operator Hg : F[z] −→ z−1 F[[z−1 ]] by Hg f = π− g f ,

f (z) ∈ F[z].

(8.9)

Here g(z) is called the symbol of the Hankel operator. Clearly, Hg is a well-defined linear transformation. We recall, from Chapter 1, that F[z], z−1 F[[z−1 ]] and F((z−1) ) carry also natural F[z]-module structures. In F[z] this is simply given in terms of polynomial multiplication. However, z−1 F[[z−1 ]] is not an F[z]-submodule of F((z−1) ). Therefore, we define for h(z) ∈ z−1 F[[z−1 ]], p · h = π− ph. For the polynomial z we denote the corresponding operators by S+ : F[z] −→ F[z] and S− : z−1 F[[z−1 ]] −→ z−1 F[[z−1 ]], defined by 

(S+ f )(z), = z f (z)

f (z) ∈ F[z],

(S− h)(z), = π− zh(z)

h(z) ∈ z−1 F[[z−1 ]].

(8.10)

Proposition 8.16. Let g(z) ∈ F((z−1 )) and let Hg : F[z] −→ z−1 F[[z−1 ]] be the Hankel operator defined in (8.9). Then 1. Hg is an F[z]-module homomorphism. 2. Any F[z]-module homomorphism from F[z] to z−1 F[[z−1 ]] is a Hankel operator. 3. Let g1 (z), g2 (z) ∈ F((z−1 ). Then Hg1 = Hg2 if and only if g1 (z) − g2 (z) ∈ F[z]. Proof. 1. It suffices to show that Hg S+ = S− Hg .

(8.11)

206

8 Tensor Products and Forms

This we do by computing Hg S+ f = π− gz f = π− z(g f ) = π− zπ− (g f ) = S− Hg f . Here we used the fact that S+ Ker π− ⊂ Ker π− . 2. Let H : F[z] −→ z−1 F[[z−1 ]] be an F[z]-module homomorphism. We set g = H1 ∈ z−1 F[[z−1 ]]. Then, since H is a homomorphism, for any polynomial f (z) we have H f = H( f · 1) = f · H1 = π− f g = πg f = Hg f . Thus we have identified H with a Hankel operator. 3. Since Hg1 − Hg2 = Hg1 −g2 , it suffices to show that Hg = 0 if and only if g(z) ∈ F[z]. Assume therefore Hg = 0. Then π− g1 = 0, which means that g(z) ∈ F[z]. The converse is obvious.  Equation (8.11) expresses the fact that Hg : F[z] −→ z−1 F[[z−1 ]] is an F[z]-module homomorphism, where both spaces are equipped with the F[z]-module structure. We call (8.11) the functional equation of Hankel operators . Corollary 8.17. Let g(z) ∈ F((z−1 )). Then Hg as a map from F[z] to z−1 F[[z−1 ]] is not invertible. Proof. Were Hg invertible, it would follow from (8.11) that S+ and S− are similar transformations. This is impossible, since for S− each α ∈ F is an eigenvalue, whereas S+ has no eigenvalues at all.  Corollary 8.18. Let Hg be a Hankel operator. Then Ker Hg is a submodule of F[z] and Im Hg a submodule of z−1 F[[z−1 ]]. 

Proof. Follows from (8.11).

Submodules of F[z] are ideals, hence of the form qF[z]. Also, we have a characterization of finite-dimensional submodules of F− (z) ⊂ z−1 F[[z−1 ]], the subspace of rational, strictly proper functions. This leads to Kronecker’s theorem. Theorem 8.19 (Kronecker). Let g(z) ∈ z−1 F[[z−1 ]]. 1. A Hankel operator Hg has finite rank if and only if g(z) is a rational function. p(z) 2. If g(z) = q(z) with p(z), q(z) coprime, then Ker H p = qF[z]

(8.12)

Im H p = X q .

(8.13)

q

and q

8.3 Some Classes of Forms

207

Proof. Assume g(z) is rational. So g(z) = p(z)/q(z), and we may assume that p(z) and q(z) are coprime. Clearly qF[z] ⊂ Ker Hg , and because of coprimeness, we actually have the equality qF[z] = Ker Hg . This proves (8.12). Now we have F[z] = Xq ⊕ qF[z] as F-vector spaces and hence Im Hg = {π− q−1 p f | f ∈ Xq }. We compute π− q−1 p f = q−1 (qπ− q−1 p f ) = q−1 π− qp f ∈ X q . Using coprimeness, and Theorem 5.18, we have {πq p f | f ∈ Xq } = Xq and therefore Im Hg = X q , which shows that Hg has finite rank. Conversely, assume Hg has finite rank. So M = Im Hg is a finite-dimensional subspace of z−1 F[[z−1 ]]. Let A = S− |M and let q(z) be the minimal polynomial of A. Therefore, for every f (z) ∈ F[z] we have 0 = π− qπ− g f = π− qg f . Choosing f (z) = 1 we conclude that q(z)g(z) = p(z) is a polynomial and hence g(z) = p(z)/q(z) is rational.  In terms of expansions in powers of z, since g(z) = ∑∞j=1 g j z− j , the Hankel operator has the infinite matrix representation ⎛

g1 ⎜g ⎜ 2 ⎜ H = ⎜ g3 ⎜ ⎝. .

g2 g3 g4 . .

g3 g4 g5 . .

. . . . .

⎞ . .⎟ ⎟ ⎟ .⎟. ⎟ .⎠ .

(8.14)

Any infinite matrix (gi j ), satisfying gi j = gi+ j−1 is called a Hankel matrix. Clearly, it is the matrix representation of the Hankel operator Hg with respect to the basis {1, z, z2 , . . .} in F[z] and {z− j | j ≥ 1} in z−1 F[[z−1 ]]. Thus H has finite rank if and only if the associated Hankel operator has finite rank. In terms of Hankel matrices, Kronecker’s theorem can be restated as follows. Theorem 8.20 (Kronecker). An infinite Hankel matrix H has finite rank if and only if its symbol g(z) = ∑∞j=1 g j z− j is a rational function. Definition 8.21. Given a proper rational function g(z), we define its McMillan degree, δ (g), by δ (g) = rank Hg . (8.15) Proposition 8.22. Let g(z) = p(z)/q(z), with p(z), q(z) coprime, be a proper rational function. Then δ (g) = deg q. (8.16) Proof. By Theorem 8.19 we have Im H p/q = X q , and so

δ (g) = rank Hg = dim X q = deg q.

(8.17) 

208

8 Tensor Products and Forms

Proposition 8.23. Let gi (z) = pi (z)/qi (z) be rational functions, with pi (z), qi (z) coprime. Then 1. We have Im Hg1 +g2 ⊂ Im Hg1 + Im Hg2

(8.18)

δ (g1 + g2 ) ≤ δ (g1 ) + δ (g2).

(8.19)

δ (g1 + g2 ) = δ (g1 ) + δ (g2 )

(8.20)

and

2. We have if and only if q1 (z), q2 (z) are coprime. Proof. 1. Obvious. 2. We have g1 (z) + g2(z) =

q1 (z)p2 (z) + q2 (z)p1 (z) . q1 (z)q2 (z)

Assume q1 (z), q2 (z) are coprime. Then by Proposition 5.25, we have X q1 + X q2 = X q1 q2 . On the other hand, the coprimeness of q1 (z), q2 (z) implies that of q1 (z)q2 (z), q1 (z)p2 (z) + q2 (z)p1 (z). Therefore Im Hg1 +g2 = X q1 q2 = X q1 + X q2 = Im Hg1 + ImHg2 . Conversely, assume δ (g1 + g2) = δ (g1 ) + δ (g2 ). This implies Im Hg1 ∩ Im Hg2 = X q1 ∩ X q2 = {0}. But invoking once again Proposition 5.25, this implies the coprimeness of q1 (z), q2 (z).  For the special case of the real field, the Hankel operator defines a quadratic form on R[z]. Since R[z]∗ = z−1 R[[z−1 ]], it follows from our definition of self-dual maps that Hg is self-dual. Thus we have also an induced quadratic form on R[z] given by ∞

∞

[Hg f , f ] = ∑ ∑ gi+ j+1 fi f j .

(8.21)

i=0 j=0

Since f (z) is a polynomial, the sum in (8.21) is well defined. Thus, with the Hankel map is associated the infinite Hankel matrix (8.14). With g(z) we also associate the finite Hankel forms Hk , k = 1, 2, . . . , defined by the matrices

8.3 Some Classes of Forms

209

⎛

g1 ⎜ . ⎜ ⎜ Hk = ⎜ . ⎜ ⎝ . gk

. . . . .

⎞ . . gk .. . ⎟ ⎟ ⎟ . . . ⎟. ⎟ .. . ⎠ . . g2k−1

In particular, assuming as we did that p(z) and q(z) are coprime and that deg(q) = n, then rank(Hk ) = deg(q) = n = δ (g) for k ≥ n. Naturally, not only the rank information for Hn is derivable from g(z) through its polynomial representation but also the signature information. This is most conveniently done by the use of Bezoutians or the Cauchy index. We turn next to these topics.

8.3.2 Bezoutians A quadratic form is associated with each symmetric matrix. We will focus now on quadratic forms induced by polynomials and rational functions. In this section we present the basic theory of Bezout forms, or Bezoutians. Bezoutians are a special class of quadratic forms that were introduced to facilitate the study of root location of polynomials. Except for the study of signature information, we do not have to restrict the field. The study of Bezoutians is intimately connected to that of Hankel forms. The study of this connection will be undertaken in Section 8.3.5. Let p(z) = ∑ni=0 pi zi and q(z) = ∑ni=0 qi zi be two polynomials and let z and w be two, generally noncommuting, variables. Then p(z)q(w) − q(z)p(w) = ∑ni=0 ∑nj=0 pi q j (zi w j − z j wi ) = ∑0≤i< j≤n (pi q j − qi p j )(zi w j − z j wi ). Observe now that j−i−1

zi w j − z j wi =

∑

ν =0

zi+ν (w − z)w j−ν −1 .

Thus, equation (8.22) can be rewritten as p(z)q(w) − q(z)p(w) =

∑

j−i−1

(pi q j − p j qi )

0≤i< j≤n

∑

ν =0

zi+ν (w − z)w j−ν −1 ,

(8.22)

210

8 Tensor Products and Forms

or n

q(z)p(w) − p(z)q(w) = ∑

n

∑ bi j zi−1 (z − w)w j−1 .

(8.23)

i=1 j=1

The last equality is obtained by changing the order of summation and properly defining the coefficients bi j . Equation (8.23) can also be written as n n q(z)p(w) − p(z)q(w) = ∑ ∑ bi j zi−1 w j−1 . z−w i=1 j=1

(8.24)

Definition 8.24. 1. The matrix (bi j ) defined by equation (8.23) or (8.24) is called the Bezout form associated with the polynomials q(z) and p(z) or just the Bezoutian of q(z) and p(z) and is denoted by B(q, p). The function q(z)p(w)−p(z)q(w) z−w is called the generating function of the Bezoutian form. 2. If g(z) = p(z)/q(z) is the unique coprime representation of a rational function g(z), with q(z) monic, then we will write B(g) for B(q, p). We shall call B(g) the Bezoutian of g(z). It should be noted that equation (8.23) holds even when the variables z and w are noncommuting. This observation is extremely useful in the derivation of matrix representations of Bezoutians. Note that B(q, p) defines a bilinear form on Fn by n

B(q, p)(ξ , η ) = ∑

n

∑ bi j ξi η j ,

ξ , η ∈ Fn .

(8.25)

i=1 j=1

No distinction will be made in the notation between the matrix and the bilinear form. The following theorem summarizes the elementary properties of the Bezoutian. Theorem 8.25. Let q(z), p(z) ∈ F[z] with max(deg q, deg p) ≤ n. Then 1. The Bezoutian matrix B(q, p) is a symmetric matrix. 2. The Bezoutian B(q, p) is linear in q(z) and p(z), separately. 3. B(p, q) = −B(q, p). It is these basic properties of the Bezoutian, in particular the linearity in both variables, that turn the Bezoutian into such a powerful tool. In conjunction with the Euclidean algorithm, it is the starting point for many fast matrix inversion algorithms. Lemma 8.26. Let q(z) and r(z) be two coprime polynomials and p(z) an arbitrary nonzero polynomial. Then

and

rank (B(qp, rp)) = rank (B(q, r))

(8.26)

σ (B(qp, rp)) = σ (B(q, r)).

(8.27)

8.3 Some Classes of Forms

211

Proof. The Bezoutian B(qp, rp) is determined by the polynomial expansion of p(z)

q(z)r(w) − r(z)q(w) p(w), z−w

˜ which implies B(qp, pr) = XB(q, r)X with ⎛

p0 . ⎜ . ⎜ ⎜ X =⎜ ⎜ ⎝

. . pm ... ... .. p0

⎞ ⎟ . ⎟ ⎟ .. ⎟. ⎟ ⎠ ... . . . pm 

The result follows now by an application of Theorem 8.13 .

The following theorem is central in the study of Bezout and Hankel forms. It gives a characterization of the Bezoutian as a matrix representation of an intertwining map. Thus, the study of Bezoutians is reduced to that of intertwining maps of the form p(Sq ), studied in Chapter 5. These are easier to handle and yield information on Bezoutians. This also leads to a conceptual theory of Bezoutians, in contrast to the purely computational approach that has been prevalent in their study for a long time. Theorem 8.27. Let p(z), q(z) ∈ F[z], with deg p ≤ deg q. Then the Bezoutian B(q, p) of q(z) and p(z) satisfies B(q, p) = [p(Sq )]stco . (8.28) Here Bst and Bco are the standard and control bases of Xq respectively defined in Proposition 5.3. Proof. Note that ⎧ ⎪ 0 ⎪ ⎨ π+ w−k (z − w)w j−1 = −1 ⎪ ⎪ ⎩ (z − w)w j−k−1

j k.

So, from equation (8.23), it follows that n

q(z)π+ w−k p(w) − p(z)π+ w−k q(w)|w=z = ∑ bik zi−1 , i=1

or, with {e1 (z), . . . , en (z)} the control basis of Xq , and pk (z) defined by pk (z) = π+ z−k p(z),

212

8 Tensor Products and Forms

we have n

q(z)pk (z) − p(z)ek (z) = ∑ bik zi−1 . i=1

Applying the projection πq , we obtain n

πq pek = p(Sq )ek = ∑ bik zi−1 ,

(8.29)

i=1

which, from the definition of a matrix representation, completes the proof.



Corollary 8.28. Let p(z), q(z) ∈ F[z], with deg p < deg q. Then the last row and column of the Bezoutian B(q, p) consist of the coefficients of p(z). Proof. By equation (8.29) and the fact that the last element of the control basis satisfies en (z) = 1, we have n

n−1

i=1

i=0

∑ binzi−1 = (πq pen )(z) = (πq p)(z) = p(z) = ∑ pi zi ,

(8.30)

i.e., bin = pi−1 ,

i = 1, . . . , n.

(8.31)

The statement for rows follows from the symmetry of the Bezoutian.  As an immediate consequence, we derive some well-known results concerning Bezoutians. Corollary 8.29. Let p(z), q(z) ∈ F[z], with deg p ≤ deg q. Assume p(z) has a factorization p(z) = p1 (z)p2 (z). Let q(z) = ∑ni=0 qi zi be monic, i.e., qn = 1. Let Cq and Cq be the companion matrices of q(z), given by (5.9) and (5.10) respectively, and let K be the Hankel matrix ⎛ ⎞ q1 . . . qn−1 1 ⎜. . ..1 0⎟ ⎜ ⎟ ⎜ ⎟ . .. .⎟ ⎜. K=⎜ (8.32) ⎟. ⎜. . .. .⎟ ⎜ ⎟ ⎝ qn−1 1 ⎠ 1 0... 0 Then 1. 2.

B(q, p1 p2 ) = B(q, p1 )p2 (Cq ) = p1 (Cq )B(q, p2 ).

(8.33)

B(q, p) = K p(Cq ) = p(Cq )K.

(8.34)

8.3 Some Classes of Forms

3.

213

B(q, p)Cq = Cq B(p, q).

(8.35)

Proof. Note that for the standard and control bases of Xq , we have Cq = [Sq ]stst and C = C˜ q = [Sq ]co . Note also that the matrix K in equation (8.32) satisfies K = B(q, 1) q

co

and is a Bezoutian as well as a Hankel matrix. 1. This follows from the equality p(Sq ) = p1 (Sq )p2 (Sq ) and the fact that B(q, p1 p2 ) = [(p1 p2 )(Sq )]stco = [p1 (Sq )p2 (Sq )]stco st st = [p1 (Sq )]stco [p2 (Sq )]co co = [p1 (Sq )]st [p2 (Sq )]co .

2. This follows from Part (1) for the trivial factorization p(z) = p(z) · 1. 3. From the commutativity of Sq and p(Sq ), it follows that st st [p(Sq )]stco [Sq ]co co = [Sq ]st [p(Sq )]co . st  We note that [Sq ]co co = ([Sq ]st ).



Representation (8.34) for the Bezoutian is sometimes referred to as the Barnett factorization. Already in Section 3.4, we derived a determinantal test, using the resultant, for the coprimeness of two polynomials. Now, with the introduction of the Bezoutian, we have another such test. We can classify the resultant approach as geometric, for basically it gives a condition for a polynomial model to be a direct sum of two shift-invariant subspaces. The Bezoutian approach, on the other hand, is arithmetic, inasmuch as it is based on the invertibility of intertwining maps. We shall see in the sequel yet another such test, given in terms of Hankel matrices, and furthermore, we shall clarify the connection between the various tests. Theorem 8.30. Given two polynomials p(z), q(z) ∈ F[z], then 1. dim(Ker B(q, p)) is equal to the degree of the g.c.d. of q(z) and p(z). 2. B(q, p) is invertible if and only if q(z) and p(z) are coprime. Proof. Part (1) follows from Theorem 5.18. Part (2) follows from Theorem 5.18 and Theorem 8.27. 

8.3.3 Representation of Bezoutians Relation (8.23) leads easily to some interesting representation formulas for the Bezoutian. For a polynomial a(z) of degree n, we define the reverse polynomial a (z) by a (z) = a(z−1 )zn . (8.36)

214

8 Tensor Products and Forms

Theorem 8.31. Let p(z) = ∑ni=0 pi zi and q(z) = ∑ni=0 qi zi be polynomials of degree n. Then the Bezoutian has the following representations: ˜  (S) − q(S)p ˜  (S)]J B(q, p) = [p(S)q ˜ − q(S)p(S)] ˜ = J[p(S)q (S)   ˜ ˜ = −J[p (S)q(S) − q (S)p(S)] ˜ − q(S)p(S)]J. ˜ = −[p (S)q(S)

(8.37)

Proof. We define the n × n shift matrix S by ⎛

01 ⎜0 . ⎜ ⎜ ⎜0 . S=⎜ ⎜0 . ⎜ ⎝0 . 0 .

. . 1. . . . . . . . .

. . . 1 . .

⎞ 0 0⎟ ⎟ ⎟ 0⎟ ⎟ 0⎟ ⎟ 1⎠ 0

(8.38)

and the n × n transposition matrix J by ⎛

0 ⎜0 ⎜ ⎜ J = ⎜0 ⎜ ⎝0 1

. . . . . . 1. . .

. 1 . . .

⎞ 1 0⎟ ⎟ ⎟ 0⎟. ⎟ 0⎠ 0

(8.39)

˜ From Note also that for an arbitrary polynomial a(z), we have Ja(S)J = a(S). equation (8.23), we easily obtain q(z)p (w) − p(z)q (w) = [q(z)p(w−1 ) − p(z)q(w−1 )]wn = ∑ni=1 ∑nj=1 bi j zi−1 (z − w−1 )w− j+1 wn = ∑ni=1 ∑nj=1 bi j zi−1 (zw − 1)wn− j . In this identity, we substitute now z = S˜ and w = S. Thus, we get for the central term ⎛

1 ⎜0 ⎜ ˜ − I) = − ⎜ (SS ⎜0 ⎜ ⎝0 0

0. 0. 0. 0. 0.

. . . . .

. . . . .

⎞ 0 0⎟ ⎟ ⎟ 0 ⎟, ⎟ 0⎠ 0

8.3 Some Classes of Forms

215

˜ − I)Sn− j is the matrix whose only nonzero term is −1 in the i, n − j and S˜i−1 (SS ˜  (S) − q(S)p ˜  (S)}J. The other position. This implies the identity B(q, p) = {p(S)q identities are similarly derived.  From the representations (8.37) of the Bezoutian we obtain, by expanding the matrices, the Gohberg–Semencul formulas, of which we present one only: ˜  (S) − q(S)p ˜  (S))J B(q, p) = (p(S)q ⎞⎛

⎡⎛

p0 ⎢⎜ . ⎢⎜ ⎜ =⎢ ⎢⎜ . ⎣⎝ . pn−1 ⎡⎛

p0 ⎢⎜ . ⎢⎜ ⎜ =⎢ ⎢⎜ . ⎣⎝ . pn−1

p0 . p0 . . p0 . . . p0

⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎠⎝

qn qn−1 . . . .. .. .

⎞ ⎛ q1 q0 ⎜. . ⎟ ⎟ ⎜ ⎜ . ⎟ ⎟ − ⎜. . ⎠ ⎝. qn

⎞⎛

q1 ⎟⎜ . p0 ⎟⎜ ⎟⎜ . . p0 ⎟⎜ ⎠⎝ q . . p0 n−1 . . . p0 qn

qn−1

⎞⎛ . .. . . . q0

⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎠⎝

pn pn−1 . . . .. .. .

⎞⎤ p1 ⎥ . ⎟ ⎟⎥ ⎥ . ⎟ ⎟⎥ J . ⎠⎦ pn

⎞ ⎛ ⎞⎛ . . qn−1 qn q0 p1 ⎟ ⎜. ⎟⎜ . . . qn ⎟ ⎜ ⎟⎜ ⎟ − ⎜. ⎟⎜ . . . . ⎟ ⎜ ⎟⎜ ⎝ ⎠ ⎠⎝ p . .. qn n−1 qn−1 . . . q0 pn

⎞⎤ . . pn−1 pn ⎟⎥ . . pn ⎟⎥ ⎟⎥. . . ⎟⎥ ⎠⎦ pn

(8.40) Given two polynomials p(z) and q(z) of degree n, we let their Sylvester resultant, Res (p, q), be defined by ⎛

p0 . ⎜ . ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ Res (p, q) = ⎜ ⎜ q0 . ⎜ ⎜ . ⎜ ⎜ ⎜ ⎝

. . pn−1 ... ... .. p0 . . qn−1 ... ... .. q0

pn . . . p1 qn . . . q1

⎞ ⎟ . ⎟ ⎟ .. ⎟ ... ⎟ ⎟ ⎟ . . . pn ⎟ ⎟. ⎟ ⎟ ⎟ . ⎟ ⎟ .. ⎟ ... ⎠ . . . qn

(8.41)

It has been proved, in Theorem 3.18, that the resultant Res (p, q) is nonsingular if and only if p(z) and q(z) are coprime. Equation (8.41) can be rewritten as the 2 × 2 block matrix   ˜ p(S) p (S) Res (p, q) = ˜ , q(S) q (S) where the reverse polynomials p (z) and q (z) are defined in equation (8.36).

216

8 Tensor Products and Forms

Based on the preceeding, we can state the following. Theorem 8.32. Let p(z) = ∑ni=0 pi zi and q(z) = ∑ni=0 qi zi be polynomials of degree n. Then     0J 0 B(p, q)  Res (p, q) Res (p, q) = . (8.42) −J 0 −B(p, q) 0 Proof. By expanding the left side of (8.42) we have 

   ˜ ˜ q(S) ˜ 0 J p(S) p (S) p(S) ˜ q(S) q (S) −J 0 p (S) q (S)   ˜ − q(S)J ˜ p (S)) ˜ ˜ ˜ p(S)) (p(S)Jq ˜  (S) (p(S)Jq(S) − q(S)J = ˜ − q(S)J p (S)) ˜ (p (S)Jq(S) − q(S)J p(S)) (p (S)Jq (S)

Now ˜ ˜ p(S) = J[p(S)q(S) − q(S)p(S)] = 0. p(S)Jq(S) − q(S)J Similarly, ˜ − q (S)J p (S) ˜ = [p (S)q (S) − q(S)p (S)]J = 0. p (S)Jq (S) In the same way, ˜  (S) ˜ − q(S)J ˜ p (S) ˜ = J[p(S)q (S) ˜ − q(S)p(S)] ˜ = B(q, p). p(S)Jq Finally, we compute ˜  (S) − q(S)p(S)] ˜ p (S)Jq(S) − q(S)J p(S) = J[p(S)q = −B(q, p). 

This proves the theorem.

The nonsingularity of both the Bezoutian and resultant matrices is equivalent to the coprimeness of the two generating polynomials. As an immediate corollary to the previous theorem, we obtain the equality, up to a sign, of their determinants. Yet another determinantal test for coprimeness, given in terms of Hankel matrices, will be given in Corollary 8.41. Corollary 8.33. Let p(z) and q(z) be polynomials of degree n. Then | det Res (p, q)| = | det B(p, q)|.

(8.43)

In particular, the nonsingularity of either matrix implies that of the other. As we already know, these conditions are equivalent to the coprimeness of the polynomials p(z) and q(z). Actually, we can be more precise about the relationship between the determinants of the resultant and Bezoutian matrices.

8.3 Some Classes of Forms

217

Corollary 8.34. Let p(z) and q(z) be polynomials of degree n. Then 1. 2.

n(n+1) 2

detRes (q, p).

(8.44)

det p(Sq ) = det p(Cq ) = det Res (p, q).

(8.45)

det B(q, p) = (−1)

˜  (S) − q(S)p ˜  (S)}J of the Proof. 1. From the representation B(q, p) = {p(S)q Bezoutian we get, by applying Lemma 3.17, det B(q, p) = det Res (q, p) det J. Since B(q, p) = −B(p, q), it follows that det B(q, p) = (−1)n det B(p, q). On the n(n−1)

other hand, for J the n × n matrix defined by (8.39), we have det J = (−1) 2 . Therefore (8.44) follows. 2. We can compute the determinant of a linear transformation in any basis. Thus det p(Sq ) = det p(Cq ) = det p(Cq ). st Also, by Theorem 8.27, B(q, p) = [p(Sq )]stco = [I]stco [p(Sq )]co co , and since det[I]co =

(−1)

n(n−1) 2

, we get det p(Sq ) = (−1)

n(n−1) 2

= (−1)

n(n−1) 2

det B(q, p) (−1)n det B(p, q)

n(n−1) 2

(−1)n (−1) = (−1) = det Res (p, q).

n(n+1) 2

det Res (p, q) 

8.3.4 Diagonalization of Bezoutians Since the Bezoutian of a pair of polynomials is a symmetric matrix, it can be diagonalized by a congruence transformation. The diagonalization procedure described here is based on polynomial factorizations and related direct sum decompositions. We begin by analyzing a special case, in which the congruence transformation is given in terms of a Vandermonde matrix. This is related to Proposition 5.15. In the proof of the following proposition, we shall make use of four distinct bases of the polynomial model Xq . These are the standard basis Bst , the control basis Bco , the interpolation basis Bin consisting of the Lagrange interpolation polynomials, and the spectral basis Bsp = {s1 (z), . . . , sn (z)}, where si (z) = Π j=i (z − λ j ). Proposition 8.35. Let q(z) be a monic polynomial of degree n having distinct zeros λ1 , . . . , λn , and let p(z) be a polynomial of degree ≤ n. Let V (λ1 , . . . , λn ) be the Vandermonde matrix

218

8 Tensor Products and Forms

⎛

1 ⎜. ⎜ ⎜ V (λ1 , . . . , λn ) = [I]in st = ⎜ . ⎜ ⎝. 1

λ1 . . . λn

⎞ . . λ1n−1 .. . ⎟ ⎟ ⎟ . . . ⎟. ⎟ .. . ⎠ . . λnn−1

Then the Bezoutian B(q, p) satisfies the following identity: V B(q, p)V˜ = R, where R is the diagonal matrix diag (r1 , . . . , rn ) and ri = p(λi )si (λi ) = p(λi )q (λi ). Proof. The trivial operator identity, I p(Sq )I = p(Sq ), implies the matrix equality st co in [I]in st [p(Sq )]co [I]sp = [p(Sq )]sp .

(8.46)

Since Sq si = λi si , it follows that p(Sq )pi = p(λi )pi = p(λi )pi (λi li . Now si (λi ) = ∏ j=i (λi − λ j ), but q (z) = ∑ni=1 si (z), and hence q (λi ) = ∏ j=i (λi − λ j ) = si (λi ). ∗ = Bin . Thus R = [p(Sq )]in Next, we use the duality relations Bst∗ = Bco and Bsp sp , and the result follows.  The previous proposition uses a factorization of q(z) into distinct linear factors. However, such a factorization does not always exist. In case it does exist, then the polynomials pi (z) = ∏ j=i (z − λ j ), which, up to a constant factor, are equal to the Lagrange interpolation polynomials, form a basis for Xq . Moreover, we have Xq = p1 Xz−λ1 ⊕ · · · ⊕ dn Xz−λn . The fact that the sum is direct is implied by the assumption that the zeros λi are distinct, or that the polynomials z − λi are pairwise coprime. In this case the internal direct sum decomposition is isomorphic to the external direct sum Xz−λ1 ⊕ · · · ⊕ Xz−λn . This can be siginificantly extended and generalized. To this end, let q1 (z), . . . , qs (z) be monic polynomials in F[z]. Let Xq1 ⊕ · · · ⊕ Xqs be the external direct sum of the corresponding polynomial models. We would like to compare this space with Xq1 ···qs . We define polynomials by di (z) = ∏ j=i q j (z) and the map Z : Xq1 ⊕ · · · ⊕ Xqs −→ Xq1 ···qs by ⎛

⎞ f1 s ⎜ ⎟ Z ⎝ ... ⎠ = ∑ di fi . fs

(8.47)

i=1

Then we have the following result, related to the Chinese remainder theorem.

8.3 Some Classes of Forms

219

Proposition 8.36. Let Z : Xq1 ⊕ · · · ⊕ Xqs −→ Xq1 ···qs be defined by (8.47). Let BST be the standard basis in Xq1 ···qs and let Bst be the basis of Xq1 ⊕ · · ·⊕ Xqs constructed from the union of the standard bases in the Xqi . Then 1. The adjoint map to Z, that is, Z ∗ : Xq1 ···qs −→ Xq1 ⊕ · · · ⊕ Xqs , is given by ⎛

⎞ π q1 f ⎜ ⎟ Z ∗ f = ⎝ ... ⎠ .

(8.48)

πq s f 2. The map Z defined by (8.47) is invertible if and only if the qi (z) are mutually coprime. 3. For the case s = 2 we have [Z]ST st = Res (q2 , q1 ),

(8.49)

where Res (q2 , q1 ) is the resultant matrix defined in (3.16) and BST is the standard basis of Xq1 ⊕ Xq2 constructed from the union of the standard bases of Xq1 and Xq2 . 4. Assume qi (z) = z − λi. Then ⎛ ⎞ 1 λ1 . . λ1s−1 ⎜. . .. . ⎟ ⎜ ⎟ ⎜ ⎟ [Z ∗ ]stST = ⎜ . . . . . ⎟ . (8.50) ⎜ ⎟ ⎝. . .. . ⎠ 1 λn . . λns−1 5. We have [Z ∗ ]stST = [Z ∗ ]stin [I]in ST and [Z ∗ ]stin = I. Proof. 1. We compute Z f , g = ∑si=1 di fi , g = [q−1 ∑si=1 di fi , g] = ∑si=1 [q−1 di fi , g] s = ∑si=1 [q−1 i f i , g] = ∑i=1  f i , πqi g =  f , Z ∗ g. 2. Note that Z ∗ is exactly the map given by the Chinese remainder theorem, namely, it maps an element of Xq into the vector of its remainders modulo the polynomials qi (z). It follows that Z ∗ is an isomorphism if and only if the qi (z) are mutually coprime. Finally, the invertibility of Z ∗ is equivalent to the invertibility of Z. 3. In this case, d1 (z) = q2 (z) and d2 (z) = q1 (z).

220

8 Tensor Products and Forms

4. This follows from the fact that for an arbitrary polynomial p(z), we have πz−λi p = p(λi ). 5. That [I]in ST is the Vandermonde matrix has been proved in Corollary 2.36. That [Z ∗ ]stin = I follows from the fact that for the Lagrange interpolation polynomials, we have πi (λ j ) = δi j .  Next, we proceed to study a factorization of the map Z defined in (8.47). This leads to, and generalizes, from a completely different perspective, the classical formula, given by (3.17), for the computation of the determinant of the Vandermonde matrix. Proposition 8.37. Let q1 (z), . . . , qs (z) be polynomials in F[z]. Then, 1. The map Z : Xq1 ⊕ · · · ⊕ Xqs −→ Xq1 ···qs , defined in (8.47), has the factorization Z = Zs−1 · · · Z1 ,

(8.51)

where Zi : Xq1 ···qi ⊕ Xqi+1 ⊕ · · · ⊕ Xqs −→ Xq1 ···qi+1 ⊕ Xqi+2 ⊕ · · · ⊕ Xqs is defined by ⎛

⎛

⎞ (q1 · · · qi ) fi+1 + qi+1 g ⎜ fi+1 ⎟ ⎜ ⎟ fi+2 ⎜ ⎟ ⎜ ⎟ Z⎜ . ⎟ = ⎜ ⎟. .. ⎝ .. ⎠ ⎝ ⎠ . g

⎞

fs

fs

2. Choosing the standard basis in all spaces, we get  [Zi ]stst =

 Res (qi+1 , qi · · · q1 ) 0 . 0 I

3. For the determinant of [Zi ]stst we have i

det[Zi ]stst = ∏ det Res (qi+1 , q j ). j=1

4. For the determinant of [Z]stst we have det[Z]stst =

∏

det Res (q j , qi ).

(8.52)

1≤i< j≤s

5. For the special case qi (z) = z − λi , we have detV (λ1 , . . . , λs ) =

∏

(λ j − λi).

1≤i< j≤s

(8.53)

8.3 Some Classes of Forms

221

Proof. 1. We prove this by induction. For s = 2 this is trivial. Assume we have proved it for all integers ≤ s. Then we define the map Z  : Xq1 ⊕ · · · ⊕ Xqs+1 −→ Xq1 ···qs ⊕ Xqs+1 by ⎛

⎞ f1   ⎜ .. ⎟ s f d ∑ ⎜ ⎟ j j=1 j . Z ⎜ . ⎟ = ⎝ fs ⎠ fs+1 fs+1 Here di (z) = ∏ j=i,s+1 q j (z). Now ds+1 (z) = ∏ j=s+1 q j (z) and qs+1 (z)di (z) = qs+1 (z)

∏

j=i,s+1

q j (z) = ∏ q j (z) = di (z). j=i

So ⎛ ⎞ f1 ⎜ ⎜ .. ⎟ s s+1 ⎜ ⎜ ⎟ Zs Z  ⎜ . ⎟ = (q1 · · · qs ) fs+1 + qs+1 ∑ d j f j = ∑ d j f j = Z ⎜ ⎝ ⎝ fs ⎠ j=1 j=1 ⎛

fs+1

⎞ f1 .. ⎟ . ⎟ ⎟. fs ⎠ fs+1

2. We use Proposition 8.36. 3. From equation (8.49) it follows that det[Zi ]stst = det Res (qi+1 , qi · · · q1 ). Now we use (8.45), i.e., the equality det Res (q, p) = det p(Sq ), to get det Res (qi+1 , qi · · · q1 ) = (−1)(∑ j=1 m j )mi+1 det Res (qi · · · q1 , qi+1 ) i

= (−1)(∑ j=1 m j )mi+1 det(qi · · · q1 )(Sqi+1 ) i

= ∏ij=1 (−1)m j mi+1 det(q j )(Sqi+1 ) = ∏ij=1 (−1)m j mi+1 detRes (q j , qi+1 ) = ∏ij=1 det Res (qi+1 , q j ). 4. Clearly, by the factorization (8.51) and the product rule of determinants, we have s−1 i st det[Z]stst = ∏s−1 j=1 det[Zi ]st = ∏ j=1 ∏ j=1 detRes (qi+1 , q j )

= ∏1≤i< j≤s det Res (q j , qi ).

222

8 Tensor Products and Forms

5. By Proposition 8.36, we have in this case V (λ1 , . . . , λs ) = [Z ∗ ]stST . Now det Z ∗ = det Z implies that ST  ST detV (λ1 , . . . , λs ) = det[Z ∗ ]stST = det[Z ∗ ]CO co = det [Z]st = det[Z]st .

Also, in this case,    −λ −λ j  . det Res (z − λi , z − λ j ) =  i 1 1  Thus (8.53) follows from (8.50). This agrees with the computation of the Vandermonde determinant given previously in Chapter 3.  We can proceed now with the block diagonalization of the Bezoutian. Here we assume an arbitrary factorization of q(z). Proposition 8.38. Let q(z) = q1 (z) · · · qs (z) and let Z : Xq1 ⊕ · · · ⊕ Xqs −→ Xq1 ···qs be the map defined by (8.47). Then 1. The following is a commutative diagram: Xq1 ⊕ · · · ⊕ Xqs

Z

(pd1 )(Sq1 ) ⊕ · · · ⊕ (pds)(Sqk ) ? Xq1 ⊕ · · · ⊕ Xqs 

- Xq ···q s 1 p(Sq )

Z∗

? Xq1 ···qs

i.e., Z ∗ p(Sq )Z = (pd1 )(Sq1 ) ⊕ · · · ⊕ (pds )(Sqk ).

(8.54)

2. Let BCO be the control basis in Xq1 ···qs and let Bco be the basis of Xq1 ⊕ · · · ⊕ Xqs constructed from the ordered union of the control bases in the Xqi , i = 1, . . . , s. We have V˜ B(q, p)V = diag (B(q1 , pd1 ), . . . , B(qs , pds )), where di (z) = ∏ j=i q j (z) and V = [Z]CO co .

8.3 Some Classes of Forms

223

Proof. 1. We compute ⎛

⎞ f1 ⎜ ⎟ Z ∗ p(Sq )Z ⎝ ... ⎠ = Z ∗ p(Sq ) ∑sj=1 d j f j = Z ∗ πq p ∑sj=1 d j f j fs ⎛

⎞ ⎛ ⎞ ⎛ ⎞ πq1 ∑sj=1 πq pd j f j (pd1 )(Sq1 ) f1 πq1 πq pd1 f1 ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ .. .. .. =⎝ ⎠=⎝ ⎠=⎝ ⎠ . . .

πqs ∑sj=1 πq pd j f j

πqs πq pds fs

(pds )(Sqs ) fs

⎛

⎞ f1 ⎜ ⎟ = ((pd1 )(Sq1 ) ⊕ · · · ⊕ (pds )(Sqk )) ⎝ ... ⎠ . fs 2. We start from (8.54) and take the following matrix representations: ST st [Z]CO [Z ∗ ]stST [p(Sq )]CO co = [(pd1 )(Sq1 ) ⊕ · · · ⊕ (pds )(Sqk )]co .

The control basis in Xq1 ⊕ · · ·⊕ Xqs refers to the ordered union of the control bases  CO . in the Xq . Of course, we use Theorem 4.38 to infer the equality [Z ∗ ]st = [Z] ST

i

co



8.3.5 Bezout and Hankel Matrices Given a strictly proper rational function g(z), we have considered two different p(z) representations of it. The first is the coprime factorization g(z) = q(z) , where we assume deg q = n, whereas the second is the expansion at infinity, i.e., the gi representation g(z) = ∑∞ i=1 zi . We saw that these two representations give rise to two different matrices, namely the Bezoutian matrix on the one hand and the Hankel matrix on the other. Our next goal is to study the relationship between these two objects. We already saw that the analysis of Bezoutians is facilitated by considering them as matrix representations of intertwining maps; see Theorem 5.18. The next theorem interprets the finite Hankel matrix in the same spirit. Theorem 8.39. Let p(z), q(z) be polynomials, with q(z) monic and deg p < deg q = p(z) gi q n. Let g(z) = q(z) = ∑∞ i=1 zi , and let H : Xq −→ X be defined by H f = Hg f ,

f ∈ Xq .

224

8 Tensor Products and Forms

Let ρq : X q −→ Xq be the map defined by (5.38), i.e., ρq h = qh. Let Bst , Bco be n (z) 1 (z) , . . . , eq(z) } be the rational the standard and control bases of Xq and let Brc = { eq(z) q control basis of X defined as the image of the control basis in Xq under the map ρq−1 . Then ⎛

1.

⎞ . . . gn ⎟ .... ⎟ ⎟ .... ⎟. ⎟ ⎠ .... . . . g2n−1

g1 ⎜. ⎜ ⎜ Hn = [H]rc st = ⎜ . ⎜ ⎝. gn

(8.55)

2. Hn is invertible if and only if p(z) and q(z) are coprime. 3. If p(z) and q(z) are coprime, then Hn−1 = B(q, a), where a(z) arises out of any solution to the Bezout equation a(z)p(z) + b(z)q(z) = 1. Proof. 1. We compute, using the representation of πq given in (5.35),

ρq−1 p(Sq ) f = q−1 πq (p f ) = q−1 qπ− q−1 p f = π− g f = Hg f = H f , i.e., we have H = ρq−1 p(Sq ).

(8.56)

This implies that the following diagram is commutative: p(Sq )

Xq

- Xq

@ H

@

@

ρq−1

@

@

?

@ R @

Xq To compute the matrix representation of H, let hi j be the i j element of [H]rc st . Then we have, by the definition of a matrix representation of a linear transformation, that n

Hz j−1 = π− pq−1 z j−1 = ∑ hi j ei /q. i=1

So n

∑ hi j ei = qπ−q−1 pz j−1 = πq pz j−1.

i=1

8.3 Some Classes of Forms

225

Using the fact that Bco is the dual basis of Bst under the pairing (5.64), we have hk j = ∑ni=1 hi j ei , zk−1  = πq pz j−1 , zk−1  = [q−1 qπ− q−1 pz j−1 , zk−1 ] = [π− q−1 pz j−1 , zk−1 ] = [gz j−1 , zk−1 ] = [g, z j+k−2 ] = g j+k−1 . 2. The invertibility of H and (8.55) implies the invertibility of Hn . 3. From the equality H = ρq−1 p(Sq ) it is clear that H is invertible if and only if p(Sq ) is, and this, by Theorem 5.18, is the case if and only if p(z) and q(z) are coprime.  The Bezout and Hankel matrices corresponding to a rational function g(z) are both symmetric matrices arising from the same data. It is therefore natural to conjecture that not only are their ranks equal but also the signatures of the corresponding quadratic forms. The next theorem shows that indeed they are congruent, and moreover, it provides the relevant congruence transformations. Theorem 8.40. Let g(z) = p(z)/q(z) with p(z) and q(z) coprime polynomials −i n n−1 + satisfying deg p < deg q = n. Assume g(z) = ∑∞ i=1 gi z and q(z) = z + qn−1 z · · ·+q0 . Then the Hankel matrix Hn and the Bezoutian B(q, p) = (bi j ) are congruent. Specifically, we have 1. ⎛

g1 ⎜. ⎜ ⎜ ⎜. ⎜ ⎝. gn

⎞ . . . gn ⎟ .... ⎟ ⎟ .... ⎟ ⎟ ⎠ .... . . . g2n−1 ⎛

⎜ ⎜ ⎜ =⎜ ⎜ ⎝ 1

⎞⎛ b11 1 ⎜ ⎟ . ψ1 ⎟ ⎜ . ⎟⎜ ... ⎟⎜. ⎟⎜ ⎠⎝. . ... ψ1 . . ψn−1 bn1

⎞⎛ . . . b1n ⎜ .... ⎟ ⎟⎜ ⎟⎜ . . . . ⎟⎜ ⎟⎜ . . . . ⎠⎝ . . . . bnn 1 ψ1

⎞ 1 . ψ1 ⎟ ⎟ ⎟ ... ⎟. ⎟ ⎠ ... . . ψn−1

(8.57)

Here, the ψi are given by the polynomial characterization of Proposition 5.41.

226

8 Tensor Products and Forms

2. In the same way, we have ⎛

b11 ⎜. ⎜ ⎜ ⎜. ⎜ ⎝. bn1

⎞ . . . b1n .... ⎟ ⎟ ⎟ .... ⎟ ⎟ .... ⎠ . . . bnn ⎛

q1 ⎜. ⎜ ⎜ = ⎜. ⎜ ⎝ qn−1 1

. . qn−1 . .1 . . 1

1

⎞⎛

g1 ⎟⎜ . ⎟⎜ ⎟⎜ ⎟⎜ . ⎟⎜ ⎠⎝ . gn

⎞⎛ . . . gn q1 ⎟⎜ . .... ⎟⎜ ⎟⎜ .... ⎟⎜ . ⎟⎜ ⎠ ⎝ qn−1 .... 1 . . . g2n−1

. . qn−1 . .1 . . 1

1

⎞ ⎟ ⎟ ⎟ ⎟. ⎟ ⎠

(8.58)

Note that equation (8.58) can be written more compactly as B(q, p) = B(q, 1)H p/qB(q, 1).

(8.59)

3. We have rank (Hn ) = rank (B(q, p)) and

σ (Hn ) = σ (B(q, p)). Proof. 1. To this end, recall that (8.56) and the definition of the bases Bst , Bco , and Brc enables us to compute a variety of matrix representations, depending on the choice of bases. In particular, it implies −1 rc st co −1 rc co st co [H]rc st = [ρq ]st [p(Sq )]co [I]st = [ρq ]co [I]st [p(Sq )]co [I]st .

It was proved in Theorem 8.39 that ⎛

g1 ⎜. ⎜ ⎜ [H]rc st = Hn = ⎜ . ⎜ ⎝. gn

⎞ . . . gn ⎟ .... ⎟ ⎟ .... ⎟. ⎟ ⎠ .... . . . g2n−1

st Obviously, [ρq−1 ]rc co = I and, using matrix representations for dual maps, [I]co =  ∗ ]st = [I] st . Here, we used the fact that B and B are dual bases. So we see  [I st co co

co

that [I]stco is symmetric. In fact, it is a Hankel matrix, easily computed to be

8.3 Some Classes of Forms

227

⎛

q1 ⎜. ⎜ ⎜ ⎜. st [I]co = ⎜ ⎜. ⎜ ⎝ qn−1 1

. . . qn−1 .... ... .. .

1

⎞ ⎟ ⎟ ⎟ ⎟ ⎟. ⎟ ⎟ ⎠

Finally, from the identification of the Bezoutian as the matrix representation of an intertwining map for Sq , i.e., from B(q, p) = [p(Sq )]stco , and defining R = [I]co st = co  [I]st , we obtain the equality ˜ p)R. Hn = RB(q,

(8.60)

2. Follows from part 1 by inverting the congruence matrix. See Proposition 5.41. 3. Follows from the congruence relations.  The following corollary gives another determinantal test for the coprimeness of two polynomials, this time in terms of a Hankel matrix. Corollary 8.41. Let g(z) = p(z)/q(z) with p(z) and q(z) polynomials satisfying −i deg p < deg q = n. If g(z) = ∑∞ i=1 gi z and ⎛

g1 ⎜. ⎜ ⎜ Hn = ⎜ . ⎜ ⎝. gn

⎞ . . . gn ⎟ .... ⎟ ⎟ .... ⎟, ⎟ ⎠ .... . . . g2n−1

then detHn = detB(q, p).

(8.61)

As a consequence, p(z) and q(z) are coprime if and only if det Hn = 0. Proof. Clearly, det R = (−1)n(n−1)/2, and so (detR)2 = 1, and hence (8.61) follows from (8.60).  The matrix identities appearing in Theorem 8.40 are interesting also for other reasons. They include in them some other identities for the principal minors of Hn , which we state in the following way.

228

8 Tensor Products and Forms

Corollary 8.42. Under the previous assumption we have, for k ≤ n, ⎛

g1 ⎜. ⎜ ⎜ ⎜. ⎜ ⎝. gk

. . . . .

. . . . .

. . . . .

⎞ gk ⎟ . ⎟ ⎟ . ⎟ ⎟ ⎠ . g2k−1

⎞⎛ b(n−k+1)(n−k+1) 1 ⎟⎜ ⎜ ψ . . ⎜ 1 ⎟⎜ ⎟⎜ ⎜ =⎜ . ⎟⎜ ... ⎟⎜ ⎜ ⎠⎝ ⎝ . ... . bn(n−k+1) 1 ψ1 . . ψk−1 ⎛

⎞⎛ . . . b(n−k+1)n ⎟⎜ ... . ⎟⎜ ⎟⎜ ... . ⎟⎜ ⎟⎜ ⎠⎝ . ... . ... bnn 1 ψ1

⎞ 1 . ψ1 ⎟ ⎟ ⎟ ⎟. ... ⎟ ⎠ ... . . ψk−1 (8.62) 

In particular, we obtain the following. Corollary 8.43. The rank of the Bezoutian of the polynomials q(z) and p(z) is equal to the rank of the Hankel matrix H p/q . Proof. Equation (8.62) implies the following equality: det(gi+ j−1 )ki, j=1 = det(bi j )ni, j=n−k+1 .

(8.63) 

At this point, we recall the connection, given in equation (8.61), between the determinant of the Bezoutian and that of the resultant of the polynomials p(z) and q(z). In fact, not only can this be made more precise, we can find relations between the minors of the resultant and those of the corresponding Hankel and Bezout matrices. We state this as follows. Theorem 8.44. Let q(z) be of degree n and p(z) of degree ≤ n. Then the lower k × k right-hand corner of the Bezoutian B(q, p) is given by ⎛

b (n−k+1)(n−k+1) ⎜ . ⎜ ⎜ . ⎜ ⎜ ⎝ . bn(n−k+1)

⎞ . . . b(n−k+1)n ⎟ ... . ⎟ ⎟ ... . ⎟ ⎟ ⎠ ... . ... bnn

8.3 Some Classes of Forms

229

⎛

⎞⎛ qn−k+1 . . pn−2k+1 ⎜ ⎟ ... . ⎟⎜ . ⎟⎜ ... . ⎟⎜ . ⎟⎜ ⎠ ⎝ qn−1 ... . . . . pn−k qn

⎞ . qn−1 qn ⎟ . . . ⎟ ⎟ . . ⎟ ⎟ ⎠ qn

⎛

⎞⎛ pn−k+1 . . qn−2k+1 ⎟⎜ . ... ⎟⎜ ⎟⎜ ... ⎟⎜ . ⎟⎜ ⎠ ⎝ pn−1 ... . . . qn−k pn

⎞ . pn−1 pn ⎟ . . . ⎟ ⎟ . . ⎟ ⎟ ⎠ pn

pn−k ⎜ . ⎜ ⎜ =⎜ . ⎜ ⎝ . pn−1 qn−k ⎜ . ⎜ ⎜ −⎜ . ⎜ ⎝ . qn−1 ⎡⎛

⎞⎛ qn qn−1 . . . pn−2k+1 ⎟⎜ . ... . ⎟⎜ ⎟⎜ ... . ⎟⎜ ⎟⎜ ⎠⎝ ... . . . . pn−k

⎛

⎞⎛ pn pn−1 . . . qn−2k+1 ⎟⎜ . ... ⎟⎜ ⎟⎜ ... ⎟⎜ ⎟⎜ ⎠⎝ ... . . . qn−k

pn−k ⎢⎜ . ⎢⎜ ⎢⎜ = ⎢⎜ . ⎢⎜ ⎣⎝ . pn−1 qn−k ⎜ . ⎜ ⎜ −⎜ . ⎜ ⎝ . qn−1

.. .. .. .

⎞ qn−k+1 ⎟ . ⎟ ⎟ . ⎟ ⎟ ⎠ . qn

⎞⎤ . . pn−k+1 ⎟⎥ ... ⎟⎥ ⎟⎥ ... ⎟⎥ Jk , ⎟⎥ ⎠⎦ .. pn

(8.64)

where we take p j = q j = 0 for j < 0. Moreover, we have det(gi+ j )ki, j=1 = det(bi j )ni, j=n−k+1   pn−k  .  .  .   p =  n−1  pn        

qn−k . . . qn−1 qn

pn−k−1 pn−k . . . pn−1 pn

qn−k−1 qn−k . . . . qn

. . pn−k ... ... ... ... pn−1 pn

 qn−2k−1       qn−k   .  .  .   .   .  qn−1   qn

(8.65)

230

8 Tensor Products and Forms

Proof. We use equation (8.40) to infer (8.64). Using Lemma 3.17, we have  p  n−k .   .  .   pn−1  det(bi j )ni, j=n−k+1 = detJk =  pn          

. . . . . pn−1 .

. . pn−2k+1 ... ... ... . . pn−k . . pn−k+1 ... ... ... . pn−1 pn

qn−k . . . qn−1 qn

 . . . qn−2k+1   ....   ....    ....   . . . qn−k  . . . qn−k+1  .  ....   ...    ...  . qn−1   qn

k(k−1)

Since det Jk = (−1) 2 , we can use this factor to rearrange the columns of the determinant on the right to get (8.65). An application of equation (8.63) yields equality (8.65).  Equation (8.65) will be useful in the derivation of the Hurwitz stability test.

8.3.6 Inversion of Hankel Matrices In this section we study the problem of inverting a finite nonsingular Hankel matrix ⎛

g1 ⎜. ⎜ ⎜ Hn = ⎜ . ⎜ ⎝. gn

⎞ . . . gn ⎟ .... ⎟ ⎟ .... ⎟. ⎟ ⎠ .... . . . g2n−1

(8.66)

We already saw, in Theorem 8.39, that if the gi arise out of a rational function −i g(z) = p(z)/q(z), with p(z), q(z) coprime, via the expansion g(z) = ∑∞ i=1 gi z , the corresponding Hankel matrix is invertible. Our aim here is to utilize the special structure of Hankel matrices in order to facilitate the computation of the inverse. What is more, we would like to provide a recursive algorithm, one that uses the information in the nth step for the computation in the next one. This is a natural entry point to the general area of fast algorithms for computing with specially structured matrices, of which the Hankel matrices are just an example. The method we shall employ is to use the coprime factorization of g(z) as the data of the problem. This leads quickly to concrete representations of the inverse. Going back to the data, given in the form of the coefficients g1 , . . . , g2n−1 , we construct all possible rational functions with rank equal to n and whose first 2n − 1 coefficients

8.3 Some Classes of Forms

231

are prescribed by the gi . Such a rational function is called a minimal rational extension of the sequence g1 , . . . , g2n−1 . We use these rational functions to obtain inversion formulas. It turns out, as is to be expected, that the nature of a specific minimal extension is irrelevant as far as the inversion formulas are concerned. Finally, we will study the recursive structure of the solution, that is, how to use the solution to the inversion problem for a given n when the amount of data is increased. Suppose now that rather than having a rational function g(z) as our data, we are given only numbers g1 , . . . , g2n−1 for which the matrix Hn is invertible. We consider i this to correspond to the rational function ∑2n−1 i=1 gi /z . We look now for all other possible minimal rational extensions. It turns out that, preserving the rank condition, there is relatively little freedom in choosing the coefficients gi , i > 2n − 1. In fact, we have the following. Proposition 8.45. Given g1 , . . . , g2n−1 for which the Hankel matrix Hn is invertible, any minimal rational extension is completely determined by the choice of ξ = g2n . Proof. Since Hn is invertible, there exists a unique solution to the equation ⎛

g1 ⎜. ⎜ ⎜ ⎜. ⎜ ⎝. gn

⎞⎛ ⎞ ⎛ ⎞ . . . gn x0 gn+1 ⎟⎜ . ⎟ ⎜ . ⎟ .... ⎟⎜ ⎟ ⎜ ⎟ ⎟⎜ ⎟ ⎜ ⎟ .... ⎟⎜ . ⎟ = ⎜ . ⎟. ⎟⎜ ⎟ ⎜ ⎟ ⎠⎝ . ⎠ ⎝ . ⎠ .... . . . g2n−1 xn−1 g2n

Since the rank of the infinite Hankel matrix does not increase, we have ⎛

g1 ⎜. ⎜ ⎜. ⎜ ⎜. ⎜ ⎜ ⎜ gn ⎜ ⎜ gn+1 ⎜ ⎜. ⎜ ⎝. .

⎞ ⎞ ⎛ . . . gn gn+1 ⎟ ⎟ ⎜ .... ⎟⎛ ⎞ ⎜ . ⎟ ⎟ ⎟ ⎜ x0 .... ⎟ ⎜ . ⎟ ⎜ ⎟ ⎟ ⎜ ⎟ .... ⎟⎜ . ⎟ ⎜ . ⎟ ⎟⎜ ⎟ ⎟ ⎜ . . . g2n−1 ⎟ ⎜ . ⎟ = ⎜ g2n ⎟ , ⎟⎜ ⎟ ⎟ ⎜ ⎝ . ⎠ ⎜ g2n+1 ⎟ . . . g2n ⎟ ⎟ ⎟ ⎜ ⎟ xn−1 ⎜ . ⎟ .... ⎟ ⎟ ⎜ ⎠ ⎝ . ⎠ .... ....

.

which shows that the gi , with i > 2n, are completely determined by the choice of g2n , via the recurrence relation gk = ∑n−1  i=0 xi gk−n+i . Incidentally, it is quite easy to obtain the monic-denominator polynomial of a minimal rational extension. Theorem 8.46. Given g1 , . . . , g2n−1 for which the Hankel matrix Hn is invertible. Let gξ (z) = pξ (z)/qξ (z) be any minimal rational extension of the sequence g1 , . . . , g2n−1 determined by ξ = g2n , pξ (z) and qξ (z) coprime, and qξ (z) monic. Let g(z) = p(z)/q(z) be the extension corresponding to ξ = 0. Also, let a(z) and

232

8 Tensor Products and Forms

b(z) be the unique polynomials of degree < n and < deg p respectively that solve the Bezout equation a(z)p(z) + b(z)q(z) = 1. (8.67) Then, 1. The polynomial a(z) is independent of ξ , and its coefficients are given as a solution of the system of linear equations ⎛

g1 ⎜. ⎜ ⎜ ⎜. ⎜ ⎝. gn

⎛ ⎞ ⎞⎛ ⎞ a0 0 . . . gn ⎜. ⎟ ⎟⎜ . ⎟ .... ⎜ ⎟ ⎟⎜ ⎟ ⎜ ⎟ ⎟⎜ ⎟ .... ⎟⎜ . ⎟ = −⎜. ⎟. ⎜ ⎟ ⎟⎜ ⎟ ⎝0⎠ ⎠⎝ . ⎠ .... . . . g2n−1 1 an−1

(8.68)

2. The polynomial b(z) is independent of ξ and has the representation n

b(z) = − ∑ gi ai (z), i=1

where ai (z), i = 1, . . . , n − 2, are the polynomials defined by ai (z) = π+ z−i a. 3. The coefficients of the polynomial qξ (z) = zn + qn−1 (ξ )zn−1 + · · · + q0 (ξ ) are solutions of the system of linear equations ⎛

g1 ⎜. ⎜ ⎜ ⎜. ⎜ ⎝. gn

⎞⎛ ⎞ ⎛ ⎞ q0 (ξ ) . . . gn gn+1 ⎟⎜ . ⎟ ⎜. ⎟ .... ⎟⎜ ⎟ ⎜ ⎟ ⎟⎜ ⎟ ⎜ ⎟ .... ⎟⎜ . ⎟. ⎟ = −⎜ . ⎟⎜ ⎟ ⎜ ⎟ ⎠⎝ . ⎝ g2n−1 ⎠ ⎠ .... qn−1 (ξ ) . . . g2n−1 ξ

(8.69)

The polynomial qξ can also be given in determinantal form as   g  1  .   −1  . qξ (z) = (det Hn )   .   gn   gn+1 4. We have qξ (z) = q(z) − ξ a(z). 5. We have pξ (z) = p(z) + ξ b(z).

 . . . gn 1  ... . z   ... . .  . ... . .   . . . g2n−1 zn−1  ... ξ zn 

(8.70)

8.3 Some Classes of Forms

233

6. With respect to the control basis polynomials corresponding to qξ (z), i.e., ei (ξ , z) = π+ z−i qξ = qi (ξ ) + qi+1(ξ )z + · · · + zn−i , i = 1, . . . , n, the polynomial pξ (z) has the representation n

pξ (z) = ∑ gi ei (ξ , z).

(8.71)

i=1

Proof. 1. Let pξ (z), qξ (z) be the polynomials in the coprime representation gξ (z) = pξ (z)/qξ (z) corresponding to the choice g2n = ξ . Let a(z) and b(z) be the unique polynomials, assuming dega < deg qξ , that solve the Bezout equation a(z)pξ (z) + b(z)qξ (z) = 1. We rewrite this equation as a(z)

pξ (z) 1 + b(z) = , qξ (z) qξ (z)

(8.72)

and note that the right-hand side has an expansion of the form 1 σn+1 1 = n + n+1 + · · · . qξ (z) z z Equating coefficients of z−i , i = 1, . . . , n, in (8.72), we have a0 g1 + · · · + an−1gn a0 g2 + · · · + an−1gn+1 . . . a0 gn−1 + · · · + an−1g2n−2 a0 gn + · · · + an−1g2n−1

= 0, = 0,

= 0, = −1,

which is equivalent to (8.68). Since g1 , . . . , g2n−1 do not depend on ξ , this shows that the polynomial a(z) is also independent of ξ . 2. Equating nonnegative indexed coefficients in (8.72), we obtain an−1 g1 = −bn−2 , an−1 g2 + an−2 g1 = −bn−3 , . . . an−1 gn−1 + · · · + a1 g1 = −b0 ,

234

8 Tensor Products and Forms

or in matrix form

⎛ ⎞⎛ ⎞ ⎞ g0 bn−2 an−1 ⎜ . ⎟ ⎟⎜ . ⎟ ⎜ . . ⎜ ⎟⎜ ⎟ ⎟ ⎜ ⎜ ⎟⎜ ⎟ ⎟ ⎜ . ⎟⎜ . ⎟ = −⎜ . ⎟. ⎜ . ⎜ ⎟⎜ ⎟ ⎟ ⎜ ⎝ . ⎠ ⎠⎝ . ⎠ ⎝ . . a1 . . . an−1 gn−1 b0 ⎛

(8.73)

This means that the polynomial b(z) is independent of ξ and has the representation b(z) = − ∑ni=1 gi ai (z). 3. From the representation gξ (z) = pξ (z)/qξ (z), we get pξ (z) = qξ (z)gξ (z). In the last equality we equate coefficients of z−1 , . . . , z−n to get the system of equations (8.69). Since Hn is assumed to be invertible, this system has a unique solution, and the solution obviously depends linearly on the parameter ξ . This system can be solved by inverting Hn to get ⎞ ⎛ ⎞ ⎛ ⎞ q0 (ξ ) gn+1 gn+1 ⎟ ⎜. ⎜ . ⎟ ⎜ . ⎟ ⎟ ⎜ ⎜ ⎟ ⎟ adjHn ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ −1 ⎜ ⎟ = −Hn ⎜ . ⎟ = − ⎜. ⎜ . ⎟. ⎟ ⎜ ⎜ ⎟ ⎟ detHn ⎜ ⎠ ⎝. ⎝ g2n−1 ⎠ ⎝ g2n−1 ⎠ ξ ξ qn−1 (ξ ) ⎛

  Multiplying on the left by the polynomial row vector 1 z · · ·zn−1 , and using equation (3.14), we get (ξ ) qξ (z) = zn + qn−1(ξ )zn−1 + · · · + q0⎛ ⎞ gn+1 ⎜ . ⎟ ⎜ ⎟   adj H ⎜ ⎟ n = zn − 1 z · · ·zn−1 ⎜ . ⎟ ⎟ det Hn ⎜ ⎝ g2n−1 ⎠ ξ ⎡

⎞⎤ gn+1 ⎢ ⎜ . ⎟⎥ ⎢ ⎜ ⎟⎥   ⎢ ⎜ ⎟⎥ −1 n n−1 = (det Hn ) ⎢det Hn z − 1 z · · ·z adjHn ⎜ . ⎟⎥ ⎢ ⎜ ⎟⎥ ⎣ ⎝ g2n−1 ⎠⎦ ξ    g1 . . . gn gn+1     . ... . .     .   . ... . = (det Hn )−1  ,  . ... . .     gn . . . g2n−1 ξ     1 z . . zn−1 zn 

which is equivalent to (8.70).

⎛

8.3 Some Classes of Forms

235

4. By writing the last row of the determinant in (8.70) as 

     gn+1 . . . ξ zn = gn+1 . . . 0 zn + 0 . . 0 ξ 0 ,

and using the linearity of the determinant as a function of each of its columns, we get    g1 . . . gn 1    . ... . z     .   . ... . qξ (z) = (det Hn )−1    . ... . .     gn . . . g2n−1 zn−1     gn+1 . . . ξ zn    g1   .    . = (det Hn )−1   .   gn   gn+1   g1   .    . +(detHn )−1   .   gn  0

 . . . gn 1  ... . z   ... . .   ... . .   . . . g2n−1 zn−1  ... 0 zn  .. .. .. .. .. ..

 . gn 1  . . z   . . .   . . .   . g2n−1 zn−1  0 ξ 0 

= q(z) − ξ r(z), where

 g  1  .   r(z) =  .   .   gn

 . . . gn−1 1  ... . z   ... . . .  ... . .   . . . g2n−2 zn−1 

We will show that r(z) = a(z). From the Bezout equations a(z)p(z) + b(z)q(z) = 1 and a(z)pξ (z) + b(z)qξ (z) = 1 it follows by subtraction that a(z)(pξ (z) − p(z)) + b(z)(qξ (z) − q(z)) = 0. Since qξ (z) = q(z) − ξ r(z), we must have a(z)(pξ (z) − p(z)) = ξ b(z)r(z). The coprimeness of a(z) and b(z) implies that a(z) is a divisor of r(z). Setting r(z) = a(z)d(z), it follows that pξ (z) = p(z) + ξ b(z)d(z) and qξ (z) = q(z) − ξ a(z)d(z). Now

236

8 Tensor Products and Forms

p(z) + ξ b(z)d(z) p(z) ξ (a(z)p(z) + b(z)q(z))d(z) − = q(z) − ξ a(z)d(z) q(z) q(z)(q(z) − ξ a(z)d(z)) ξ d(z) = . q(z)(q(z) − ξ a(z)d(z))

gξ (z) − g(z) =

Since gξ (z) and g(z) have expansions in powers of z−1 agreeing for the first 2n − 1 terms, and the next term has to be ξ z−2n , this implies d = 1. 5. Follows from pξ (z) = p(z) + ξ b(z)d(z), by substituting d(z) = 1. 6. Equating positively indexed coefficients in the equality pξ (z) = qξ (z)gξ (z), we have pn−1 (ξ ) = g1 , pn−2 (ξ ) = g2 + qn−1 (ξ )g1 , . . . p0 (ξ ) = gn + qn−1 (ξ )gn−1 + · · · + q1 (ξ )g1 , or

⎞ q1 (ξ ) . . . qn−1 (ξ ) 1 ⎛ ⎞ p 0 (ξ ) ⎟ g1 ⎜ . . .. . ⎟⎜ . ⎟ ⎟ ⎜ ⎜ . ⎟⎜ ⎟ ⎟ ⎜ ⎜ . . .. ⎟⎜ ⎟ ⎟ ⎜ ⎜ . ⎟⎜ . ⎟. ⎟=⎜ ⎜ ⎟⎜ ⎟ ⎟ ⎜ ⎜ . . . ⎟⎝ . ⎠ ⎠ ⎜ ⎝ . ⎠ ⎝ qn−1 (ξ ) 1 gn pn−1 (ξ ) 1 ⎛

⎞

⎛

This is equivalent to pξ (z) = g1 e1 (ξ , z) + · · · + gn en (ξ , z), 

which proves (8.71).

Incidentally, parts (4) and (5) of the theorem provide a nice parametrization of all solutions to the minimal partial realization problem arising out of a nonsingular Hankel matrix. Corollary 8.47. Given the nonsingular Hankel matrix Hn , if g(z) = p(z)/q(z) is the minimal rational extension corresponding to the choice g2n = 0, then the minimal rational extension corresponding to g2n = ξ is given by gξ (z) =

p(z) + ξ b(z) , q(z) − ξ a(z)

where a(z), b(z) are the polynomials arising from the solution of the Bezout equation a(z)p(z) + b(z)q(z) = 1. This theorem explains the mechanism of how the particular minimal rational extension chosen does not influence the computation of the inverse, as of course it

8.3 Some Classes of Forms

237

should not. Indeed, we see that B(qξ , a) = B(q − ξ a, a) = B(q, a) − ξ B(a, a) = B(q, a), and so Hn−1 = B(qξ , a) = B(q, a).

8.3.7 Continued Fractions and Orthogonal Polynomials The Euclidean algorithm provides a tool not only for the computation of a g.c.d. of a pair of polynomials but also, if coprimeness is assumed, for the solution of the corresponding Bezout equation. The application of the Euclidean algorithm yields a representation of a rational function in the form of a continued fraction. It should be noted that a slight modification of the process makes the method applicable to an arbitrary (formal) power series. This is the theme of the present section. Given an irreducible representation p(z)/q(z) of a strictly proper rational function g(z), we define, using the division rule for polynomials, a sequence of polynomials qi (z), nonzero constants βi , and monic polynomials ai (z), by 

q−1 (z) = q(z), q0 (z) = p(z), qi+1 (z) = ai+1 (z)qi (z) − βi qi−1 (z),

(8.74)

with deg qi+1 < deg qi . The procedure ends when qn = 0, and then qn−1 is the g.c.d. of p(z) and q(z). Since p(z) and q(z) are assumed coprime, qn−1 (z) is a nonzero constant. The pairs {βi−1 , ai (z)} are called the atoms of g(z). We define αi = deg ai . For i = 0, we rewrite equation (8.74) as

β0 p q0 = = . q q−1 a1 − q1 q0 Thus g1 = q1 /q0 is also a strictly proper rational function, and furthermore, this is an irreducible representation. By an inductive argument, this leads to the continued fraction representation of g(z) in terms of the atoms βi−1 and ai (z):

β0

g(z) =

.

β1

a1 (z) − a2 (z) −

β2 . a3 (z) − . .

βn−2 an−1 (z) −

βn−1 an (z)

(8.75)

238

8 Tensor Products and Forms

There are two ways to truncate the continued fraction. The first one is to consider the top part, i.e., let

β0

gi (z) =

.

β1

a1 (z) −

(8.76)

β2

a2 (z) −

βi−2

..

a3 (z) − .

ai−1 (z) −

βi−1 ai (z)

The other option is to consider the rational functions γi defined by the bottom part, namely βi . (8.77) γi (z) = βi+1 ai+1 (z) − βn−2 . ai+2 (z) − . . βn−1 an−1 (z) − an (z) p(z) . We also set γn (z) = 0. Clearly, we have gn (z) = γ0 (z) = g(z) = q(z) Just as a rational function determines a unique continued fraction, it is also completely determined by it.

Proposition 8.48. Let g(z) be a rational function having the continued fraction representation (8.75). Then {βi , ai+1 }n−1 i=0 are the atoms of g(z). Proof. Assume g(z) = p(z)/q(z). Then g(z) =

β0 p(z) = , q(z) a1 − γ1

with γi strictly proper. This implies p(z)γ1 = a1 (z)p(z) − β0 q(z). This shows that r(z) = p(z)γ1 (z) is a polynomial, and since γ1 (z) is strictly proper, we have deg r < deg p. It follows that {β0 , a1 (z)} is the first atom of g(z). The rest follows by induction.  Fractional linear transformations are crucial to the analysis of continued fractions and play a central role in a wide variety of parametrization problems. Definition 8.49. A fractional linear transformation, or a M¨obius transformation, is a function of the form f (w) =

aw + b , cw + d

8.3 Some Classes of Forms

239

under the extra condition  ad  − bc = 0. With this fractional linear transformation we associate the matrix

ab cd

.

The following lemma relates composition of fractional linear transformations to matrix multiplication. w+bi , i = 1, 2, be two fractional linear transformations. Lemma 8.50. Let fi (w) = aciiw+d i Then their composition is given by ( f1 ◦ f 2 )(w) = aw+b cw+d , where



ab cd



 =

a 1 b1 c 1 d1



a 2 b2 c2 d2

 . 

Proof. By computation.

We define now two sequences of polynomials {Qk (z)} and {Pk (z)} by the three term recurrence formulas  Q−1 (z) = 0, Q0 (z) = 1, (8.78) Qk+1 (z) = ak+1 (z)Qk (z) − βk Qk−1 (z), and



P−1 (z) = −1, P0 (z) = 0, Pk+1 (z) = ak+1 (z)Pk (z) − βk Pk−1 (z).

(8.79)

The polynomials Qi (z) and Pi (z) so defined are called the Lanczos polynomials of the first and second kind respectively. Theorem 8.51. Let g(z) be a strictly proper rational function having the continued fraction representation (8.75). Let the Lanczos polynomials be defined by (8.78) and (8.79), and define the polynomials Ak (z), Bk (z) by Ak (z) =

Qk−1 (z) , β0 · · · βk−1

−Pk−1(z) Bk (z) = . β0 · · · βk−1

(8.80)

Then 1. For the rational functions γi (z), i = 1, . . . , n, we have ai+1 (z)γi (z) − βi = γi (z)γi+1 (z).

(8.81)

2. Defining the rational functions Ei (z) by Ei (z) = Qi (z)Pn (z)/Qn (z) − Pi (z) = (Qi (z)Pn (z) − Pi (z)Qn (z))/Qn (z), (8.82)

240

8 Tensor Products and Forms

they satisfy the following recurrence: 

E−1 = 1, E0 = g0 = g, Ek+1 (z) = ak+1 (z)Ek (z) − βk Ek−1 (z).

3. We have

(8.83)

Ei (z) = γ0 (z) · · · γi (z).

(8.84)

4. In the expansion of Ei (z) in powers of z−1 , the leading terms is

β0 ···βi . zα1 +···+αi+1

5. With αi = deg ai , we have deg Qk = ∑ki=1 αi and deg Pk = ∑ki=2 αi . 6. The polynomials Ak (z), Bk (z) are coprime and solve the Bezout equation Pk (z)Ak (z) + Qk (z)Bk (z) = 1.

(8.85)

7. The rational function Pk (z)/Qk (z) is strictly proper, and the expansions of p(z)/q(z) and Pk (z)/Qk (z) in powers of z−1 agree up to order 2 ∑ki=1 deg ai + deg ak+1 = 2 ∑ki=1 αi + αi+1 .   8. For the unimodular matrices 

0 β0 −1 a1



0 βi −1 ai+1



, i = 0, . . . , n − 1, we have

0 βi−1 ··· −1 ai



 =

−Pi−1 Pi −Qi−1 Qi

 .

9.  Let Ti bethe fractional linear transformation corresponding to the matrix 0 βi . Then −1 ai+1

g(z) = (T0 ◦ · · · ◦ Ti−1 )(γi ) =

Pi − Pi−1 γi . Qi − Qi−1 γi

(8.86)

In particular, g(z) = (T0 ◦ · · · ◦ Tn−1)(γn ) = Pn (z)/Qn (z), i.e., we have Pn (z) = p(z), 10. The 11. The

Qn (z) = q(z).

atoms of Pk (z)/Qk (z) are {βi−1 , ai (z)}ki=1 . generating function of the Bezoutian of Qk (z) Qk (z)Ak (w) − Ak (z)Qk (w) z−w = ∑kj=1 (β0 · · · β j−1 )−1 Q j (z)

(8.87)

and Ak (z) satisfies

a j+1 (z) − a j+1(w) Q j (w). z−w

(8.88)

8.3 Some Classes of Forms

241

The formula for the expansion of the generating function is called the generalized Christoffel–Darboux formula . The regular Christoffel–Darboux formula is the generic case, in which all the atoms a j have degree one. 12. The Bezoutian matrix B(Qk , Ak ) can be written as B(Qk , Ak ) =

k−1

∑ (β0 · · · β j−1)−1 R˜ j , B(a j+1, 1)R j ,

j=1

where R j is the n j × n Toeplitz matrix ⎛

( j)

( j)

⎞

( j)

σ0 σ1 . . . σn j ⎜ . ... . . ⎜ ⎜ ... . . ⎜ Rj = ⎜ ⎜ .. . . ⎜ ⎝ . . . ( j) ( j) σ0 σ1

. .. ... ( j) . . . σn j

⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠

(8.89)

( j)

and Q j (z) = ∑ σν zν . 13. Assume the Hankel matrix Hn of (8.66) to be nonsingular. Let g = p/q be any minimal rational extension of the sequence g1 , . . . , g2n−1 . Then Hn−1 =

n

1

∑ β0 · · · β j−1 R˜ j B(1, a j+1)R j .

j=1

Proof. 1. Clearly, by the definition of the γi (z), we have, for i = 1, . . . , n − 1,

γi (z) =

βi , ai+1 (z) − γi+1 (z)

(8.90)

which can be rewritten as (8.81). Since γn−1 (z) = βn−1 /an (z) and γn (z) = 0, equation (8.81) holds also for i = n. 2. We compute first the initial conditions, using the initial conditions of the Lanczos polynomials. Thus E−1 (z) = Q−1 (z)Pn (z)/Qn (z) − P−1(z) = 1, E0 (z) = Q0 (z)Pn (z)/Qn (z) − P0 (z) = Pn (z)/Qn (z) = γ0 (z) = g(z). Next, we compute

242

8 Tensor Products and Forms

Ei+1 (z) = Qi+1 (z)Pn (z)/Qn (z) − Pi+1 (z) = (Qi+1 (z)Pn (z) − Pi+1 (z)Qn (z))/Qn (z) = [(ai+1 (z)Qi (z) − βi Qi−1 (z))Pn (z) − (ai+1 (z)Pi (z) − βi Pi−1 (z))Qn (z)] /Qn (z) = ai+1 (z)(Qi (z)Pn (z) − Pi (z)Qn (z))/Qn (z) − βi (Qi−1 (z)Pn (z) − Pi−1 (z)Qn (z))/Qn (z) = ai+1 (z)Ei (z) − βi Ei−1 (z). 3. We use the recurrence formula (8.83) and equation (8.81). The proof goes by induction. For i = 1 we have E1 = a1 E0 − β0 = a1 γ0 − β0 = γ0 γ1 . Assuming that Ei (z) = γ0 (z) · · · γi (z), we compute Ei+1 (z) = ai+1 Ei (z) − βi Ei−1 (z) = ai+1 (z)γ0 (z) · · · γi (z) − βi γ0 (z) · · · γi−1 (z) = γ0 (z) · · · γi−1 (z)(ai+1 (z)γi (z) − βi ) = γ0 (z) · · · γi−1 (z)(γi (z)γi+1 (z)). 4. From (8.90) we have βi /zαi+1 for the leading term of the expansion of γi . Since Ei (z) = γ0 (z) · · · γi (z), we have β0 · · · βi /zα1 +···+αi+1 = β0 · · · βi /zνi+1 for the leading term of the expansion of Ei (z). 5. By induction, using the recurrence formulas as well as the initial conditions. 6. We compute the expression Qk+1 (z)Pk (z) − Qk (z)Pk+1 (z) using the defining recurrence formulas. So Qk+1 (z)Pk (z) − Qk (z)Pk+1 (z) = (ak+1 (z)Qk (z) − βk Qk−1 (z))Pk (z) −(ak+1 (z)Pk (z) − βk Pk−1 (z))Qk (z) = βk (Qk (z)Pk−1 (z) − Qk−1 (z)Pk (z)), and by induction, this implies Qk+1 (z)Pk (z) − Qk (z)Pk+1 (z) = βk · · · β0 (Q0 (z)P−1 (z) − Q−1 (z)P0 (z)) = − βk · · · β0 . Therefore, for each k, Qk (z) and Pk (z) are coprime. Moreover, dividing by −βk · · · β0 , we see that the following Bezout identities hold: Pk (z)Ak (z) + Qk (z)Bk (z) = 1. 7. With the Ei (z) defined by (8.82), we compute Pn (z)/Qn (z) − Pi (z)/Qi (z) = (Pn (z)Qi (z) − Qn (z)Pi (z))/Qn (z)Qi (z) = Ei (z)/Qi (z).

8.3 Some Classes of Forms

243

Since deg Qi = ∑ij=1 α j and using part (4), it follows that the leading term of Ei (z)/Qi (z) is β0 · · · βi /z2 ∑ j=1 α j +αi+1 . This is in agreement with the statement of the theorem, since deg ai = ∑ij=1 α j . 8. We will prove it by induction. Indeed, for i = 0, we have i



0 β0 −1 a1



 =

−P0 P1 −Q0 Q1

 .

Next, using the induction hypothesis, we compute 

−Pi−1 Pi −Qi−1 Qi



0 βi −1 ai+1



 =

−Pi ai+1 Pi − βiPi−1 −Qi ai+1 Qi − βiQi−1



 =

−Pi Pi+1 −Qi Qi+1

 .

9. We clearly have g = γ0 = T0 (γ1 ) = T0 (T1 (γ2 )) = (T0 ◦ T1)(γ2 ) = · · · = (T0 ◦ · · · ◦ Ti−1 )(γi ). To prove (8.87), we use the fact that γn = 0. 10. From (8.76), and using (8.86), we get gk = (T0 ◦ · · · ◦ Tk−1 )(0) = Pk /Qk . 11. Obviously, the Bezout identity (8.85) implies also the coprimeness of the polynomials Ak (z) and Qk (z). Since the Bezoutian is linear in each of its arguments we can get a recursive formula for B(Ak , Qk ): B(Qk , Ak ) = B(Qk , (β0 · · · βk−1 )−1 Qk−1 ) = B(ak Qk−1 − βk−1 Qk−2 , (β0 · · · βk−1 )−1 Qk−1 ) = (β0 · · · βk−1 )−1 Qk−1 (z)B(ak , 1)Qk−1 (w) + B(Qk−1 , (β0 · · · βk−2 )−1 Qk−2 ) = · · · = ∑(β0 · · · βk−1 )−1 Q j (z)B(a j+1 , 1)Q j (w). 12. This is just the matrix representation of (8.88). 13. We use Theorem 8.39, the equalities Qn (z) = q(z), Pn (z) = p(z), and the Bezout equation (8.85) for k = n.  A continued fraction representation is an extremely convenient tool for the computation of the signature of the Hankel or Bezout quadratic forms. Theorem 8.52 (Frobenius). Let g(z) be rational having the continued fraction representation (8.75). Then   r i−1 1 + (−1)degai −1 . (8.91) σ (Hg ) = ∑ sign ∏ β j 2 j=0 i=1

244

8 Tensor Products and Forms

Proof. It suffices to compute the signature of the Bezoutian B(q, p). Using the equations of the Euclidean algorithm, we have  a1 q0 − q 1 , q0 β0 q0 (z)B(a1 , 1)q0 (w) + β1 B(q0 , q1 ) 

B(q, p) = B(q−1 , q0 ) = B =

1 β0

0

Here we used the fact that the Bezoutian is alternating. Since p(z) and q(z) are coprime, we have rank B(a1 , 1) + β1 rank B(q0 , q1 ) = deg a1 + degq0 = deg q = rank B(q, p). 0

The additivity of the ranks implies, by Theorem 8.15, the additivity of the signatures, and therefore     1 1 σ (B(q, p)) = σ B(a1 , 1) + σ B(q0 , q1 ) . β0 β0 We proceed by induction and readily derive r

σ (B(p, q)) = ∑ sign



i=1

1 β0 · · · βi



σ (B(ai+1 , 1)).

Now, given a polynomial c(z) = zk + ck−1 zk−1 + · · · + c0 , we have B(c, 1) =

k zi − wi c(z) − c(w) = ∑ ci , z−w z−w i=1

with ck = 1. Using the equality zi − wi = z−w we have

⎛

i

∑ z j wi− j−1 ,

j=1

c1 ⎜c ⎜ 2 ⎜ ⎜. B(c, 1) = ⎜ ⎜. ⎜ ⎝ ci−1 1

c2 . . . 1

. . ci−1 ... .. .

1

⎞ ⎟ ⎟ ⎟ ⎟ ⎟. ⎟ ⎟ ⎠

(8.92)

8.3 Some Classes of Forms

Hence

245



1

if deg c is odd,

0

if deg c is even. 

Equality (8.92) can now be rewritten as (8.91).

The Lanczos polynomials Qi (z) generated by the recurrence relation (8.78) satisfy certain orthogonality conditions. These orthogonality conditions depend of course on the rational function g(z) from which the polynomials were derived. These orthogonality relations are related to an inner product induced by the Hankel operator Hg . We will explain now this relation. Starting from the strictly proper rational function g(z), we have the bilinear form defined on the space of polynomials by {x, y} = [Hg x, y] = ∑ ∑ gi+ j ξi η j , i

(8.93)

j

where x(z) = ∑i ξi zi and y(z) = ∑ j η j z j . If g(z) has the coprime representation g(z) = p(z)/q(z), then KerHg = qF[z], and hence we may as well restrict ourselves to the study of Hg as a map from Xq to X q = Xq∗ . However, we can view all the action as taking place in Xq , using the pairing ·, · defined in (5.64). Thus, for x(z), y(z) ∈ Xq , we have [Hg x, y] = [π− q−1 px, y] = [q−1 qπ− q−1 px, y] = πq px, y = p(Sq )x, y. In fact, p(Sq ) is, as a result of Theorem 5.39, a self-adjoint operator in Xq . The pairing {x, y} = p(Sq )x, y can of course be evaluated by choosing an appropriate basis. In fact, we have st st co p(Sq )x, y = ([p(Sq )]co st [x] , [y] ). Since [p(Sq )]st = Hn , we recover (8.93). Proposition 8.53. Let Pi (z), Qi (z) be the Lanczos polynomials associated with the rational function g(z). Relative to the pairing {x, y} = p(Sq )x, y = [Hg x, y] = ∑ ∑ gi+ j ξi η j , i

(8.94)

j

the polynomial Qi (z) is orthogonal to all polynomials of lower degree. Specifically,  {Qi , z j } = [Hg Qi , z j ] =

0, j < α1 + · · · + αi , β 0 · · · β i , j = α1 + · · · + αi .

(8.95)

Proof. Recalling that the rational functions Ei (z) are defined in (8.82) by Ei (z) = g(z)Qi (z) − Pi (z), this means that

246

8 Tensor Products and Forms

π− Ei = π− gQi = Hg Qi . Since the leading term of Ei (z) is

β0 ···βi , zα1 +···+αi+1

this implies (8.95).



i−1 The set {z ji (Πk=0 Qk (z)) | i = 1, . . . , r − 1; ji = 0, . . . , ni − 1}, lexicographically ordered, is clearly a basis for Xq since it contains one polynomial for each degree between 0 and n − 1. We denote it by Bor and refer to it as the orthogonal basis because of its relation to orthogonal polynomials. The properties of the Lanczos polynomials can be used to compute a matrix representation of the shift Sq relative to this basis. p(z) Theorem 8.54. Let the strictly proper transfer function g(z) = q(z) have the r sequence of atoms {ai (z), βi−1 }i=1 , and assume that α1 , . . . , αr are the degrees of the atoms and

ak (z) = zαk +

αk −1

∑

(k)

ai zi .

i=0

Then the shift operator Sq has the following, block tridiagonal, matrix representation with respect to the orthogonal basis: ⎛

⎞ A11 A12 ⎜A A . ⎟ ⎜ 21 22 ⎟ ⎜ ⎟ or [Sq ]or = ⎜ . .. ⎟, ⎜ ⎟ ⎝ .. Ar−1 r ⎠ Ar r−1 Arr where

⎛

0.. . ⎜1 ⎜ ⎜ Aii = ⎜ . ⎜ . ⎝ 1 ⎛ 0..0 ⎜. ⎜ ⎜ Ai+1 i = ⎜ . ⎜ ⎝. 0.. .

⎞ (i) −a0 . ⎟ ⎟ ⎟ . ⎟ , i = 1, . . . r, ⎟ . ⎠ (i) −a ⎞αi−1 1 0⎟ ⎟ ⎟ . ⎟ , i = 1, . . . r − 1, ⎟ .⎠ 0

and Ai i+1 = βi−1 Ai+1 i . Proof. On the basis elements zi Q j−1 (z), i = 0, . . . , n j − 2, j = 1, . . . , r − 1, the map Sq indeed acts as the shift to the next basis element. For i = n j − 1 we compute

8.3 Some Classes of Forms

247

Sq zn j −1 Q j−1 = πq zn j Q j−1

 n j −1 ( j) k n j −1 ( j) k = πq zn j + ∑k=0 ak z − ∑k=0 ak z Q j−1 n −1 ( j)

j = a j Q j−1 − ∑k=0 ak zk · Q j−1 n j −1 ( j) k = Q j + β j−1Q j−2 − ∑k=0 ak z · Q j−1 .



In the generic case, in which all Hankel matrices Hi , i = 1, . . . , n, are nonsingular, all polynomials ai (z) are linear, and we write ai (z) = z − θi . The recurrence equation for the orthogonal polynomials becomes Qi+1 (z) = (z − θi+1 )Qi (z) − βi Qi−1 (z), and we obtain the following corollary. Corollary 8.55. Assuming all Hankel matrices Hi , i = 1, . . . , n, are nonsingular, then with respect to the orthogonal basis {Q0 (z), . . . , Qn−1 (z)} of Xq = XQn , the shift operator Sq has the tridiagonal matrix representation ⎛

⎞ θ1 β 1 ⎜1 θ . ⎟ 2 ⎜ ⎟ ⎜ ⎟ . .. ⎜ ⎟ or [Sq ]or = ⎜ ⎟. ⎜ ⎟ .. . ⎜ ⎟ ⎝ . . βn−1 ⎠ 1 θn The Hankel matrix Hn is symmetric, hence can be diagonalized by a congruence transformation. In fact, we can exhibit explicitly a diagonalizing congruence. Using the same method, we can clarify the connection between Cq , the companion matrix of q(z), and the tridiagonal representation. Proposition 8.56. Let g(z) = p(z)/q(z) be strictly proper, rational of McMillan degree n = degq. Assume all Hankel matrices Hi , i = 1, . . . , n, are nonsingular and let {Q0 (z), . . . , Qn−1 (z)} be the orthogonal basis of Xq . Then 1.

⎛

⎞⎛ q0,0 g1 ⎜ q ⎜ ⎟ ⎜ 1,0 q1,1 ⎟⎜. ⎜ ⎟⎜ . ⎜ . ⎟⎜. ⎜ ⎟⎜ ⎝ . ⎠⎝. . gn qn−1,0 . . .qn−1,n−1 ⎛ ⎜ ⎜ ⎜ =⎜ ⎜ ⎝

β0

⎞⎛ ⎞ . . . gn q0,0 q1,0 . . qn−1,0 ⎟⎜ ⎟ .... q1,1 . ⎟⎜ ⎟ ⎟⎜ ⎟ .... . . ⎟⎜ ⎟ ⎟⎜ ⎟ ⎠⎝ ⎠ .... . . . . . g2n−1 qn−1,n−1

⎞

β 0 β1 . .

β0 · · · βn−1

⎟ ⎟ ⎟ ⎟. ⎟ ⎠ (8.96)

248

8 Tensor Products and Forms

2. ⎛

⎞⎛ ⎞ q0,0 q1,0 . . qn−1,0 −q0 ⎜ ⎟ · ⎟ q1,1 . ⎟⎜ ⎟ ⎟⎜ ⎟ · ⎟⎜ . . ⎟ ⎟⎜ ⎟ ⎠ · ⎠⎝ . . 1 −qn−1 qn−1,n−1 ⎞⎛ ⎞ ⎛ q0,0 q1,0 . . qn−1,0 θ1 β1 ⎟⎜ 1 θ . ⎟ ⎜ q1,1 . ⎟⎜ ⎟ ⎜ 2 ⎟⎜ ⎟ ⎜ =⎜ . . . . ⎟⎜ ⎟. ⎟⎜ ⎟ ⎜ ⎠⎝ ⎝ . . . . βn−1 ⎠ 1 θn qn−1,n−1

0 ⎜1 ⎜ ⎜ ⎜ . ⎜ ⎝ .

(8.97)

Proof. 1. This is the matrix representation of the orthogonality relations given in (8.95). 2. We use the trivial operator identity Sq I = ISq and take the matrix representation [Sq ]stst [I]stor = [I]stor [Sq ]or  or . This is equivalent to (8.97). It is easy to read off from the diagonal representation (8.96) of the Hankel matrix the relevant signature information. A special case is the following. Corollary 8.57. Let g(z) be a rational function with the continued fraction representation (8.75). Then Hn is positive definite if and only if all ai (z) are linear and all the βi are positive.

8.3.8 The Cauchy Index Let g(z) be a rational transfer function with real coefficients having the coprime representation g(z) = p(z)/q(z). The Cauchy index of g(z), denoted by Ig , is defined as the number of jumps, on the real axis, of g(z) from −∞ to +∞ minus the number of jumps from +∞ to −∞. Thus, the Cauchy index is related to discontinuities of g(z) on the real axis. These discontinuities arise from the real zeros of q(z). However, the contributions to the Cauchy index come only from zeros of q(z) of odd order. In this section we will establish some connections between the Cauchy index, the signature of the Hankel map induced by g(z), and the existence of signaturesymmetric realizations of g(z). The central result is the classical Hermite–Hurwitz theorem. However, before proving it, we state and prove an auxiliary result that is of independent interest.

8.3 Some Classes of Forms

249

Proposition 8.58. Let g(z) be a real rational function. Then the following scaling operations (1) g(z) −→ mg(z), m > 0, (2) g(z) −→ g(z − a), a ∈ R, (3) g(z) −→ g(rz), r > 0, leave the rank and signature of the Hankel map as well as the Cauchy index invariant. Proof. (1) is obvious. To prove the rank invariance, let g(z) = p(z)/q(z) with p(z) and q(z) coprime. By the Euclidean algorithm, there exist polynomials a(z) and b(z) such that a(z)p(z) + b(z)q(z) = 1. This implies a(z − a)p(z − a) + b(z − a)q(z − a) = 1 as well as a(rz)p(rz) + b(rz)q(rz) = 1, i.e., p(z − a), q(z − a) are coprime and so are p(rz), q(rz). Now g(z − a) = p(z − a)/q(z − a) and g(rz) = p(rz)/q(rz), which proves the invariance of McMillan degree, which is equal to the rank of the Hankel map. Now it is easy to check that given any polynomial u(z), we have [Hg u, u] = [Hga ua , ua ], where ga (z) = g(z − a). If we define a map Ra : R[z] −→ R[z] by (Ra u)(z) = u(z − a) = ua (z), then Ra is invertible, R−1 a = R−a , and [Hg u, u] = [Hga ua , ua ] = [Hga Ra u, Ra u] = [Ra ∗ Hga Ra u, u], which shows that Hg = R∗a Hga Ra and hence that σ (Hg ) = σ (Hga ). This proves (2). To prove (3), define, for r > 0, a map Pr : R[z] −→ R[z] by (Pr u)(z) = u(rz). Clearly, Pr is invertible and Pr−1 = P1/r . Letting ur = Pr u, we have [Hgr u, u] = [π− g(rz)u, u] = [π− ∑(gk /rk+1 zk+1 )u, u] = ∑ ∑(gi+ j r−i− j ui )u j = ∑ ∑ gi+ j (ui r−i )(u j r− j ) = [Hg Pr u, Pr u] = [Pr∗ Hg Pr u, u],

250

8 Tensor Products and Forms

hence Hgr = Pr∗ Hg Pr , which implies σ (Hgr ) = σ (Hg ). The invariance of the Cauchy index under these scaling operations is obvious.  Theorem 8.59 (Hermite–Hurwitz). Let g(z) = p(z)/q(z) be a strictly proper, real rational function with p(z) and q(z) coprime. Then Ig = σ (Hg ) = σ (B(q, p)).

(8.98)

Proof. That σ (Hg ) = σ (B(q, p)) has been proved in Theorem 8.40. So it suffices to prove the equality Ig = σ (Hg ). Let us analyze first the case that q(z) is a polynomial with simple real zeros, i.e., q(z) = ∏nj=1 (z − a j ) and ai = a j for i = j. Let di (z) = q(z)/(z − ai). Given any polynomial u(z) ∈ Xq , it has a unique expansion u(z) = ∑ni=1 ui di (z). We compute [Hg u, u] = [π− gu, u] = [π− q−1 pu, u] = [q−1 qπ− q−1 pu, u] = πq pu, u = p(Sq )u, u = ∑ni=1 ∑nj=1 p(Sq )di , d j ui u j = ∑ni=1 ∑nj=1 p(ai )di , d j ui u j = ∑ni=1 p(ai )di (ai )u2i , since di (z) are eigenfunctions of Sq corresponding to the eigenvalues ai and since by Proposition 5.40, di , d j  = di (ai )δi j . From this computation it follows, since [p(Sq )]stco = B(q, p), that n

σ (Hg ) = σ (B(q, p)) = ∑ sign [p(ai )di (ai )]. i=1

On the other hand, we have the partial fraction decomposition g(z) = or

n

n ci p(z) =∑ , q(z) i=1 z − ai

p(z) = ∑ ci i=1

n q(z) = ∑ ci di (z), z − ai i=1

which implies p(ai ) = ci di (ai ), or equivalently, that ci = n

n

i=1

i=1

Ig = ∑ sign (ci ) = ∑ sign



p(ai ) di (ai ) .

Now obviously

 p(ai ) , di (ai )

and, since sign (p(ai )di (ai )) = sign (p(ai )/di (ai )), the equality (8.98) is proved in this case.

8.3 Some Classes of Forms

251

We pass now to the general case. Let q(z) = q1 (z) · · · qs (z) be the unique factorization of q(z) into powers of relatively prime irreducible monic polynomials. As before, we define polynomials di (z) by q(z) . qi (z)

di (z) =

Since we have the direct sum decomposition Xq = d1 Xq1 ⊕ · · ·⊕ ds Xqs , it follows that each f (z) ∈ Xq has a unique representation of the form f (z) = ∑si=1 di (z)ui (z) with ui (z) ∈ Xqi . Relative to the indefinite metric of Xq , introduced in (5.64), this is an orthogonal direct sum decomposition, i.e., di Xqi , d j Xq j  = 0

for i = j.

Indeed, if fi (z) ∈ Xqi and g j (z) ∈ Xq j , then di fi , d j f j  = [q−1 di fi , d j f j ] = [d j q−1 di fi , f j ] = 0, since di (z)d j (z) is divisible by q(z) and F[z]⊥ = F[z]. Let g(z) = ∑si=1 pi (z)/qi (z) be the partial fraction decomposition of g(z). Since the zeros of the qi (z) are distinct, it is clear that s

Ig = I p = ∑ I pi /qi . q

i=1

Also, as a consequence of Proposition 8.23, it is clear that for the McMillan degree δ (g) of g(z), we have δ (g) = ∑si=1 δ (pi /qi ), and hence, by Theorem 8.15, the signatures of the Hankel forms are additive, namely σ (Hg ) = ∑si=1 σ (H pi /qi ). Therefore, to prove the Hermite–Hurwitz theorem it suffices to prove it in the case of q(z) being the power of a monic prime. Since we are discussing the real case, the primes are of the form z − a and ((z − a)2 + b2 ), with a, b ∈ R. By applying the previous scaling result, the proof of the Hermite–Hurwitz theorem reduces to the two cases q(z) = zm or q(z) = (z2 + 1)m . Case 1: q(z) = zm . Assuming p(z) = p0 + p1 z + · · · + pm−1 zm−1 , then the coprimeness of p(z) and q(z) is equivalent to p0 = 0. Therefore we have g(z) = pm−1 z−1 + · · · + p0 z−m , which shows that  0 if m is even, Ig = I(p/zm) = sign (p0 ) if m is odd. On the other hand, Ker Hg = zm+1 R[z], and so

σ (Hg ) = σ (Hg |Xzm ).

252

8 Tensor Products and Forms

Relative to the standard basis, the truncated Hankel map has the matrix representation ⎛ ⎞ pm−1 . . . p1 p0 ⎜. ⎟ .... ⎜ ⎟ ⎜ ⎟ ... ⎜. ⎟ ⎜ ⎟. ⎜. ⎟ .. ⎜ ⎟ ⎝ p1 . ⎠ p0 Now, clearly, the previous matrix has the same signature as ⎛

0 ⎜. ⎜ ⎜ ⎜. ⎜ ⎜. ⎜ ⎝0 p0 and hence



σ (Hg ) =

⎞ . . . 0 p0 ⎟ .... ⎟ ⎟ ... ⎟ ⎟, ⎟ .. ⎟ ⎠ .

sign (p0 ) if m is odd, 0 if m is even.

Case 2: q(z) = (z2 + 1)m . Since q(z) has no real zeros, it follows that in this case Ig = I p = 0. So it suffices q

to prove that also σ (Hg ) = 0. Let g(z) = p(z)/(z2 + 1)m with deg p < 2m. Let us expand p(z) in the form p(z) =

m−1

∑ (pk + qkz)(z2 + 1)k,

k=0

with the pk and qk uniquely determined. The coprimeness condition is equivalent to p0 and q0 not being zero together. The transfer function g(z) has therefore the following representation: g(z) =

m−1

∑

k=0

pk + q k z . (z2 + 1)m−k

In much the same way, every polynomial u(z) can be written, in a unique way, as u(z) =

m−1

∑ (ui + vi z)(z2 + 1)i.

i=0

8.3 Some Classes of Forms

253

We compute now the matrix representation of the Hankel form with respect to the basis B = {1, z, (z2 + 1), z(z2 + 1), . . ., (z2 + 1)m−1 , z(z2 + 1)m−1 } of X(z2 +1)m . Thus, we need to compute [Hg zα (z2 + 1)λ , zβ (z2 + 1)μ ] = [Hg zα +β (z2 + 1)λ +μ , 1]. Now, with 0 ≤ γ ≤ 2 and 0 ≤ ν ≤ 2m − 2, we compute 

m−1 (pi + qi z)(z2 + 1)i γ 2 ∑i=0 z (z + 1)ν , 1 (z2 + 1)m ! γ 2 ν +i " m−1 (pi + qiz)z (z + 1) ,1 . = ∑i=0 (z2 + 1)m

[Hg zγ (z2 + 1)ν , 1] =

The only nonzero contributions come from the terms in which ν + i = m − 2, m − 1, or equivalently, when i = m − λ − μ − 2, m − λ − μ − 1. Now

!

"

⎧⎧ ⎪ ⎨ qi , γ = 0, ⎪ ⎪ ⎪ ⎪ p , γ = 1, ⎪ ⎪⎩ i ⎪ ⎪ ⎨ −qi , γ = 2,

(pi + qi z)zγ (z2 + 1)ν +i ,1 = ⎧ ⎪ (z2 + 1)m ⎪ ⎪ ⎨ 0, γ = 0, ⎪ ⎪ ⎪ ⎪ 0, γ = 1, ⎪ ⎪ ⎩⎩ qi , γ = 2,

i = m − ν − 1,

i = m − ν − 2.

Thus the matrix of the Hankel form in this basis has the following block triangular form: ⎞ ⎛ qm−1 pm−1 q1 p1 q0 p 0 ⎜ pm−1 qm−2 − qm−1 p1 q0 − q1 p0 −q0 ⎟ ⎟ ⎜ ⎟ ⎜ q0 p0 ⎟ ⎜ ⎟ ⎜ p0 −q0 ⎟ ⎜ M=⎜ ⎟. ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎠ ⎝ q0 p0 p0 −q0   q0 p0 By our coprimeness assumption, the matrix is nonsingular and its p0 −q0

determinant is negative. Hence it has signature equal to zero. This implies that also the signature of M is zero, and with this, the proof of the theorem is complete. 

254

8 Tensor Products and Forms

8.4 Tensor Products of Models The underlying idea in this book is that the study of linear algebra is simplified and clarified when functional, or module-theoretic, methods are used. This motivated us in the introduction of polynomial and rational models. Naturally, we expect that the study of tensor products and forms by the use of these models will lead to similar simplifications, and this we proceed to do. In our approach, we emphasize the concreteness of specific representations of tensor products of polynomial models. Thus, we expect that the tensor products of polynomial models, whether over the underlying field F or over the polynomial ring F[z], will also turn out to be (isomorphic to) polynomial models in one or two variables as the case may be. There are several different models we can use. Two of them arise from Kronecker products of multiplication maps, the difference being whether the product is taken over the field F or the ring F[z].

8.4.1 Bilinear Forms Let X , Y be F-vector spaces. We shall denote by L(X , Y ), as well as by Hom F (X , Y ), the space of F-linear maps from X to Y . We take now a closer look at the case of tensor products of two finite-dimensional F-vector spaces X , Y . In this situation, and in order to clearly distinguish it from the algebraic dual M  of a module M, we will write X ∗ = Hom F (X , F) for the vector space dual. Let BX = { fi }ni=1 , BY = {gi }m i=1 be bases of X and Y respectively. Let ∗ = {φ }n be the basis of X ∗ that is dual to B , i.e., it satisfies φ ( f ) = δ . BX i i=1 i j ij X Let F be a field and let X , Y , Z be F-vector spaces. A function φ : X × Y −→ Z is called a Z -valued bilinear form on X × Y if φ is linear in both variables. We will denote by Bilin (X , Y ; Z ) the space of all Z -valued bilinear functions. We will denote by Bilin (X , Y ; F) the set of all F valued bilinear functions on X × Y and refer to its elements as bilinear forms. There is a close connection between bilinear forms and linear transformations which we summarize in the following proposition. Proposition 8.60. Given F-linear vector spaces X , Y , a necessary and sufficient condition for a function φ : X × Y −→ F to be a bilinear form is the existence of a, uniquely defined, linear map T : X −→ Y ∗ for which

φ (x, y) = T x, y.

(8.99)

Proof. Assume T ∈ Hom F (X , Y ∗ ). Defining φ : X × Y −→ F by (8.99), it clearly follows that φ (x, y) is an F-valued bilinear form. Conversely, if φ (x, y) is an F-valued bilinear form on X × Y , then for each x ∈ X , φ (x, y) is a linear functional on Y . Thus there exists y∗x ∈ Y ∗ for which

8.4 Tensor Products of Models

255

φ (x, y) = y∗x , y. Define a map T : X −→ Y ∗ by T x = y∗x . Using the linearity of φ (x, y) in the x variable, we compute y∗α1 x1 +α2 x2 , y = φ (α1 x1 + α2 x2 , y) = α1 φ (x1 , y) + α2 φ (x2 , y) = α1 y∗x1 , y + α2 y∗x2 , y = α1 y∗x1 + α2 y∗x2 , y. This implies y∗α1 x1 +α2 x2 = α1 y∗x1 + α2 y∗x2 , which can be rewritten as T (α1 x1 + α2 x2 ) = α1 T x1 + α2 T x2 , i.e., T ∈ Hom F (X , Y ∗ ). Uniqueness of T follows by a standard argument. 

8.4.2 Tensor Products of Vector Spaces The availability of duality pairings allows us to take another look at matrix representations. Given F-vector spaces X , Y and elements f ∈ X ∗ and w ∈ Y , we use the notation v, f  = f (v). Next, we define a map w ⊗ f : X −→ Y by (w ⊗ f )v = v, f w,

v∈X.

(8.100)

It is easily checked that w⊗ f is the zero map if and only if f = 0 or w = 0; otherwise, it has rank one. Proposition 8.61. We have  Hom F (X , Y ) =

s

∑

# wi ⊗ v∗i

| wi ∈ Y

, v∗i

∗

∈ X ,s ∈ N .

(8.101)

i=1

Proof. Clearly, from (8.100) and the bilinearity of the pairing v, f , it follows that w ⊗ f is a linear transformation. Since Hom F (X , Y ) is closed under linear combinations, we have the inclusion Hom F (X , Y ) ⊃ {∑i wi ⊗ fi | wi ∈ Y , fi ∈ X ∗ }. Conversely, let T ∈ Hom F (X , Y ). We consider the bases BV = {e1 , . . . , en } of X and BW = { f1 , . . . , fm } of Y . A vector v ∈ X has a unique representation of the form v = ∑nj=1 α j e j . Similarly, w = T v ∈ Y has a unique representation of the form ∗ ∗ ∗ T v = ∑m i=1 βi f i . With BV ∗ = {e1 , . . . , en } the dual basis to BV , we have e j , ei  = δi j ∗ and BV ∗ = BV . Let Ei j ∈ Hom F (X , Y ) be the map defined by (4.12), i.e., by Ei j ek = δ jk fi . Clearly, we have Ei j v = v, e∗j  fi = ( fi ⊗ e∗j )v. The computation (4.14) shows that

(8.102)

256

8 Tensor Products and Forms m

n

T = ∑ ∑ ti j fi ⊗ e∗j ,

(8.103)

i=1 j=1

and hence the inclusion Hom F (X , Y ) ⊂ {∑i wi ⊗ fi |wi ∈ Y , fi ∈ X ∗ }. The two inclusions are equivalent to (8.101).  We proceed to define the tensor product of two vector spaces. Our preference is to do it in a concrete way, so that the tensor product is identified with a particular representation. Given F-vector spaces X , Y , let X ∗ be the vector space dual of X . For x∗ ∈ X ∗ and y ∈ Y , we define a map y ⊗ x∗ ∈ Hom F (X , Y ) by (y ⊗ x∗ )ξ = (x∗ ξ )y,

ξ ∈X.

(8.104)

Although we can give an abstract definition of a tensor product, we prefer to give a concrete one. Definition 8.62. Given F-vector spaces X , Y , we define Y ⊗F X ∗ = Hom F (X , Y ).

(8.105)

We call Y ⊗F X ∗ = Hom F (X , Y ) the tensor product of Y and X ∗ . The map φ : Y × X ∗ −→ Y ⊗F X ∗ defined by

φ (y, x∗ ) = y ⊗ x∗

(8.106)

is called the canonical map. Clearly, in this case, the canonical map φ is bilinear and bijective. Note that for finite-dimensional F-vector spaces X , Y , and using reflexivity, that is, (X ∗ )∗  X , we can also define Y ⊗F X = Hom F (X ∗ , Y ). Proposition 8.63. Let BX = {e1 , . . . , en } and BY = { f1 , . . . , fm } be bases of X ∗ = {e∗ , . . . , e∗ } be the dual basis to B . Then and Y respectively and let BX X n 1 1. The set

∗ BY ⊗ BX = { fi ⊗ e∗j | i = 1, . . . , m, j = 1, . . . , n}

(8.107)

∗ as the tensor is a basis for Y ⊗F X ∗ . We will refer to the basis BY ⊗ BX ∗ . product of the bases BY and BX 2. We have dim Y ⊗ X ∗ = dim Y · dim X . (8.108) B

3. For T ∈ Hom F (X , Y ), with the matrix representation [T ]BYX defined by (4.16), we have ∗ B [T ]BY ⊗BX = [T ]BYX . (8.109) Proof. 1. Note that we have the equality

8.4 Tensor Products of Models

257

fi ⊗ e∗j = Ei j ,

(8.110)

where the maps Ei j are defined by (4.7). Thus, the statement follows from Theorem 4.6. ∗ is mn; hence (8.108) follows. 2. The number of elements in the basis BY ⊗ BX n ∗ 3. Based on the identification (8.105), we have T = ∑m i=1 ∑ j=1 ti j f i ⊗ e j . Comparing this with (4.15), (8.109) follows.  n ∗ The representation T = ∑m i=1 ∑ j=1 ti j ei ⊗ f j of T ∈ Hom F (X , Y ) can generally be simplified. If rankT = dim Im T = k, then there exists a minimal-length representation k

T = ∑ ψ i ⊗ φi ,

(8.111)

i=1

where {φi }ki=1 is a basis of (Ker T )⊥ ⊂ X ∗ and {ψi }ki=1 is a basis for Im T ⊂ Y . The following proposition describes the underlying property of tensor products. Proposition 8.64. Let γ : Y × X ∗ −→ Z be a bilinear map. Then there exists a unique linear map γ∗ : Y ⊗F X ∗ −→ Z for which the following diagram commutes:

φ

Y ×X ∗

- Y ⊗X ∗

@

@

γ

@

@

γ∗

@

@ R @

? Z

Proof. Given the bilinear map γ ∈ Hom F (Y × X ∗ , Z ), we define γ∗ ∈ Hom F (Y ⊗ X ∗ , Z ) by

γ∗ (y ⊗ x∗ ) = γ (y, x∗ ).

(8.112)

It is easily checked that γ∗ is well defined and linear and that γ = γ∗ ◦ φ holds. Since any element of Y ⊗ X ∗ is a finite sum ∑ri=1 yi ⊗ x∗i , the result follows.  One of the fundamental properties of tensor products is the connection between maps defined on tensor products, bilinear forms (hence also quadratic forms), and module homomorphisms. In the special case of F-vector spaces X and Y , we have the following result.

258

8 Tensor Products and Forms

Proposition 8.65. 1. Let X , Y be finite-dimensional F-vector spaces. Then we have the following isomorphisms: (X ⊗F Y )∗  Bilin (X , Y ; F)  Hom F (X , Y ∗ ).

(8.113)

2. Let X , Y be finite-dimensional F-vector spaces. Then we have the following isomorphism (X ⊗F Y )∗  X ∗ ⊗F Y ∗  Y ∗ ⊗F X ∗ . (8.114) Proof. 1. We give only a sketch of the proof. Let φ (x, y) be a bilinear form on X × Y . For fixed x ∈ X , this defines a linear functional ηx ∈ Y ∗ by

ηx (y) = φ (x, y).

(8.115)

Clearly, ηx depends linearly on x; hence there exists a map Z : X −→ Y ∗ for which Zx = ηx . So φ (x, y) = (Zx)(y). Conversely, assume Z ∈ Hom F (X , Y ∗ ). Then for every x ∈ X , we have Zx ∈ Y ∗ . Defining φ (x, y) = (Zx)(y), we get a bilinear form defined on X × Y . The homomorphisms φ → Z and Z → φ are inverses to each other; hence the isomorphism Bilin (X , Y ; F)  Hom F (X , Y ∗ ) follows. Given a bilinear form γ : X × Y −→ F, by the definition of tensor products and with γ : X × Y −→ X ⊗F Y the canonical map, there exists a unique γ∗ ∈ (X ⊗F Y )∗ such that γ = γ∗ φ . The uniqueness implies that the map γ → γ∗ is injective. It is also surjective, since any γ∗ ∈ (X ⊗F Y )∗ induces a bilinear form γ on X ×Y by letting γ = γ∗ φ . Thus we have also the isomorphism (X ⊗F Y )∗  Bilin (X , Y ; F). 2. Assume ξi ∈ X ∗ , ηi ∈ Y ∗ . Then for every x ∈ X , we define s

s

i=1

i=1

∑ (ηi ⊗ ξi)x = ∑ ξi (x)ηi ,

(8.116)

which defines a map Y ∗ ⊗ X ∗ −→ Hom F (X , Y ∗ ). Conversely, if X is finite-dimensional, the image of any Z ∈ Hom (X , Y ∗ ) q is finite-dimensional, hence has a basis {ηi }i=1 , with ηi ∈ Y ∗ . Therefore we have, for suitable ξi ∈ X ∗ , the representation q

q

i=1

i=1

Zx = ∑ ξi (x)ηi = ∑ (ηi ⊗ ξi )x.

(8.117)

Thus Z → ∑qi=1 ξi ⊗ ηi defines an isomorphism from Hom(X , Y ∗ ) to Y ∗ ⊗X ∗ , and thus, by the isomorphism (8.113), also an isomorphism with X ∗ ⊗F Y ∗ . 

8.4 Tensor Products of Models

259

8.4.3 Tensor Products of Modules Our definition of tensor products of vector spaces was an ad hoc one. To get into line with standard algebraic terminology, we use the characterization of tensor products, given in Proposition 8.64, as the basis for the following definition. Definition 8.66. Let R be a commutative ring and X , Y R-modules. An R-module X ⊗R W is called a tensor product over R if there exists a bilinear map φ : X × Y −→ X ⊗R W such that for any bilinear map γ : X × Y −→ Z , there exists a unique R-linear map γ∗ : X ⊗R W −→ Z for which

γ = γ∗ φ .

(8.118)

The following theorem summarizes the main properties of tensor products. We will not give a proof and refer the reader to Hungerford (1974) and Lang (1965) for the details. Theorem 8.67. Let R be a commutative ring and V , W R-modules. Then 1. For any R-modules V , W , a tensor product V ⊗R W exists and is unique up to isomorphism. 2. We have the following isomorphisms: (⊕ki=1 Mi ) ⊗R N  ⊕ki=1 (Mi ⊗R N), M ⊗R (⊕lj=1 N j )  ⊕lj=1 (M ⊗R N j ), M ⊗R (N ⊗R P)  (M ⊗R N) ⊗R P, M ⊗R N  N ⊗R M.

(8.119)

3. If S ⊂ R is any subring and M, N are R-modules, then we have a well-defined surjective S-linear map M ⊗S N −→ M ⊗R N defined by m ⊗S n → m ⊗R n. 4. Let M1 , M2 be R-modules, with R a commutative ring. Let Ni ⊂ Mi be submodules. The quotient spaces Mi /Ni have a natural R-module structure. Let N be the submodule generated in M1 ⊗R M2 by N1 ⊗R M2 and M1 ⊗R N2 . Then we have the isomorphism M1 /N1 ⊗R M2 /N2  (M1 ⊗R M2 )/N.

(8.120)

5. Given R-modules M, N, L, we denote by Bilin R (M, N; L) the set of all R-bilinear maps φ : M × N −→ L. We have the following isomorphisms: Hom R (M ⊗R N, L)  Bilin R (M, N; L)  Hom R (M, Hom R (N, L)).

(8.121)

260

8 Tensor Products and Forms

8.4.4 Kronecker Product Models We define the F-Kronecker product of two scalar polynomials d1 (z), d2 (z) ∈ F[z] as the map d1 ⊗F d2 : F[z, w] −→ F[z, w] given by (d1 ⊗F d2 )q(z, w) = d1 (z)q(z, w)d2 (w). This map induces a projection πd1 ⊗F d2 in F[z, w] defined by

πd1 ⊗F d2 q(z, w) = (d1 ⊗F d2 )(π−z ⊗ π−w )(d1 ⊗F d2 )−1 q(z, w).

(8.122)

Thus, if q(z, w) is given as q(z, w) = ∑ri=1 ai (z)bi (w), then

πd1 ⊗F d2 q(z, w) = ∑ri=1 (πd1 ai )(z)(πd2 bi )(w) = ∑ri=1 d1 (z)π− (d1−1 ai )(z)π− (d2−1 bi )(w)d2 (w).

(8.123)

We obtain the F-Kronecker product model as Xd1 ⊗F d2 = Xd1(z)d2 (w) = Im πd1 ⊗F d2 . From now on, we will mainly use the more suggestive notation Xd1 (z)d2 (w) for the F-Kronecker product model. By inspection, one verifies that  Xd1 (z)d2 (w) =

deg d1 −1 deg d2 −1

∑

i=0

∑

# i

fi j z w

j

.

(8.124)

j=0

From this description, it follows that we have the dimension formula dimF (Xd1 (z)d2 (w) ) = deg d1 · degd2 .

(8.125)

Similarly, we define the F[z]-Kronecker product d1 ⊗F[z] d2 of two scalar polynomials d1 (z), d2 (z) as the map d1 ⊗F[z] d2 : F[z] −→ F[z] given by d1 ⊗F[z] d2 q(z) = d1 (z)q(z)d2 (z). This defines the projection map πd1 ⊗d2 : F[z] −→ F[z] given by πd1 ⊗F[z] d2 f = π(d1 d2 ) f . (8.126) In turn, this allows us to introduce the F[z]-Kronecker product model as Xd1 ⊗F[z] d2  F[z]/(d1 d2 )F[z]  Xd1 d2 .

(8.127)

Consequently, for the F[z]-Kronecker product model, we have the dimension formula dimF Xd1 ⊗F[z] d2 = deg(d1 d2 ) = deg d1 + degd2 = dimF Xd1 + dimF Xd2 .

(8.128)

8.4 Tensor Products of Models

261

Finally, from duality of polynomial and rational models, we achieve a representation of the dual to the Kronecker product model as (Xd1 ⊗F[z] d2 )∗ = (Xd1 d2 )∗ = X d1 d2 .

(8.129)

8.4.5 Tensor Products over a Field The tensor products were defined, in Definition 8.66, in a formal way. Our conviction is that a concrete representation for the tensor product is always preferable. Thus, we have the identifications, i.e., up to isomorphism, for tensor products of polynomial spaces, i.e., F[z] ⊗F F[z]  F[z, w], (8.130) as well as F[z] ⊗F[z] F[z] = F[z].

(8.131)

For our purposes, this is less interesting than the identification of the tensor product of two polynomial models. This will be achieved by the use of Theorem 8.67, and in particular, equation (8.120). Central to this is the isomorphism (5.6), namely Xq  F[z]/qF[z]. The availability of Kronecker product polynomial models allows us to give concrete representations for the tensor products. We show that the tensor product Xd1 ⊗F Xd2 of two polynomial models over the field F can be interpreted as a polynomial model for the two-variable polynomial d1 (z)d2 (w). Using (8.120) and the isomorphism (5.6), we have Xd1 ⊗F Xd2  (F[z]/d1 F[z]) ⊗F (F[z]/d2 F[z])  F[z, w]/(d1 (z)F[z, w] + d2 (w)F[z, w]). The submodule d1 (z)F[z, w] + d2 (w)F[z, w] can be seen as the kernel of the projection operator πd1 (z) ⊗ πd2 (w) acting in F[z, w] by (πd1 (z) ⊗ πd2(w) )( f (z)g(w)) = (πd1 (z) f (z))(πd2 (w) g(w)). From this, the above representation of Xd1 ⊗F Xd2 easily follows. Defining Xd1 (z)d2 (w) = Im (πd1 (z) ⊗ πd2(w) ) implies the isomorphism Xd1 ⊗F Xd2  Xd1 (z)d2 (w) ,

(8.132)

i.e., the F-tensor product of polynomial models is a polynomial model in two variables. Proposition 8.68. Given nontrivial polynomials d1 (z), d2 (z) ∈ F[z], we have the isomorphism Xd1 ⊗F Xd2  Xd1 ⊗F d2 = Xd1 (z)d2 (w) . (8.133)

262

8 Tensor Products and Forms

In particular, the F-tensor product of two polynomial models in one variable is the two-variable Kronecker product polynomial model, and the map γ∗ : Xd1 ⊗F Xd2 −→ Xd1 (z)d2 (w) induced by γ∗ ( f1 ⊗ f2 ) = f1 (z) f2 (w) (8.134) is an F-isomorphism. Proof. Let φ : Xd1 × Xd2 −→ Xd1 ⊗F Xd2 be the canonical map. Consider the Fbilinear map γ : Xd1 × Xd2 −→ Xd1 (z)d2 (w) defined by γ ( f1 , f2 ) = f1 (z) f2 (w). Thus γ∗ exists and is F-linear, leading to the following commutative diagram:

φ

Xd1 × Xd2

- Xd ⊗F Xd 1 2

@

@

γ

@

@

γ∗

@

@ R @

? Xd1 (z)d2 (w)

The description (8.124) shows that γ∗ is surjective. From the dimension formula (8.125) we conclude that Xd1 ⊗F Xd2 and Xd1 (z)d2 (w) have the same F-dimension, and the result follows.  Being the tensor product of two vector spaces over the field F, the tensor product model Xd1 ⊗F Xd2 is clearly an F-vector space. We will see later that it carries also a natural F[z, w]-module structure. We now connect the space of F-linear maps between polynomial models Xd1 , Xd2 , namely Hom F (Xd1 , Xd2 ), with the tensor product Xd2 ⊗F Xd1 . Using the residue form on Xd1 , i.e.,  f , g = (d1−1 f g)−1 , where h−1 denotes the coefficient of z−1 in the Laurent series expansion of h(z), we can get the concrete form of the isomorphism as follows. Theorem 8.69. We define a map Ψ : Xd2 (z)d1 (w) −→ Hom F (Xd1 , Xd2 ) as follows. For q(z, w) ∈ Xd2 ⊗F Xd1 , the map is q(z, w) → Ψ (q), where Ψ (q) : Xd1 −→ Xd2 is defined, for f ∈ Xd1 , by

Ψ (q) f =  f , q(z, ·) = (q(z, ·)d1 (·)−1 f (·))−1 .

(8.135)

The map q(z, w) → Ψ (q) defines a vector space isomorphism, i.e., we have the following isomorphism: Xd2 ⊗F Xd1  Hom F (Xd1 , Xd2 ).

(8.136)

8.4 Tensor Products of Models

263

Proof. Let q1 (z), . . . , qs (z) denote a basis of Xd1 (z)d2 (w) . Then it is easily seen that Ψ (q1 ), . . . , Ψ (qn ) are linearly independent in Hom F (Xd1 , Xd2 ). The result follows because both spaces have the same dimension.  Using the isomorphism Xd2 ⊗F Xd1  Xd2 (z)d1 (w) ,

(8.137)

we obtain the explicit isomorphism Ψ : Xd2 ⊗F Xd1 −→ Hom F (Xd1 , Xd2 ) given, for g ∈ Xd1 , by Ψ ( f2 ⊗ f1 )g = [d1−1 g, f1 ] f2 = g, f1  f2 , (8.138) where the pairing g, f  is defined by (5.64). We consider next the F-vector space dual of the tensor product. Proposition 8.70. Given nontrivial polynomials d1 (z), d2 (z) ∈ F[z], we have the following identification of the vector space dual of Xd1 ⊗F Xd2 : (Xd1 ⊗F Xd2 )∗  X d1 ⊗F X d2  X d1 (z) ∩ X d2 (w) .

(8.139)

Proof. We compute, using vector space annihilators, (Xd1 ⊗F Xd2 )∗  (F[z]/d1 F[z] ⊗F F[z]/d2 F[z])∗  (d1 (z)F[z, w] + d2 (w)F[z, w])⊥  (d1 (z)F[z, w])⊥ ∩ (d2 (w)F[z, w])⊥ d1 ⊗1 ∩ X 1⊗d2 =X  =

h(z, w) ∈ z−1 F[[z−1 , w−1 ]]w−1 |h(z, w) =

$ p(z, w) . d1 (z)d2 (w) 

8.4.6 Tensor Products over the Ring of Polynomials From Proposition 8.68, we have seen that the F-tensor product of polynomial models, namely Xd1 ⊗F Xd2 , takes us out of the realm of single-variable polynomial spaces. The situation changes when the underlying ring is F[z]. Since polynomial models Xd1 , Xd2 are F[z]-modules, their tensor product, namely Xd1 ⊗F[z] Xd2 , is an F[z]-module too. The tensor product is, by Definition 8.66, an abstract object. Again, for clarity, as well as computational purposes, we would like to have a concrete representation for it, and this is done next. In analogy with (8.133), we have the F[z]-module isomorphism

264

8 Tensor Products and Forms

Xd1 ⊗F[z] Xd2  F[z]/d1 F[z] ⊗F[z] F[z]/d2 F[z]  F[z]/(d1 (z)F[z] + d2 (z)F[z])  F[z]/((d1 ∧ d2 )F[z])  Xd1 ∧d2 .

(8.140)

Proposition 8.71. Given nontrivial polynomials d1 (z), d2 (z) ∈ F[z], let φ : Xd1 × Xd2 −→ Xd1 ⊗F[z] Xd2 be the canonical map and let γ : Xd1 × Xd2 −→ Xd1 ∧d2 be defined by γ ( f1 , f2 ) = πd1 ∧d2 ( f1 f2 ). (8.141) Then the map γ∗ : Xd1 ⊗F[z] Xd2 −→ Xd1 ∧d2 , defined by

γ∗ ( f1 ⊗ f2 ) = πd1 ∧d2 ( f1 f2 ),

(8.142)

is an F[z]-module isomorphism that implies the isomorphism Xd1 ⊗F[z] Xd2  Xd1 ∧d2 .

(8.143)

The commutativity of the following diagram holds:

φ

Xd1 × Xd2

- Xd ⊗F[z] Xd 1 2

@

@

γ

@

γ∗

@

@

@ R @

? Xd1 ∧d2

Proof. We check first that γ∗ is well defined on the tensor product space. This follows immediately from the identity

γ∗ (p · f1 ⊗ f2 ) = πd1 ∧d2 (p f1 f2 ) = πd1 ∧d2 pπd1 ∧d2 ( f1 f2 ) = p · γ∗ ( f1 ⊗ f2 ), which shows that γ∗ is a well defined F[z]-homomorphism. Clearly, the map γ is surjective, since for any f (z) ∈ Xd1 ∧d2 , we have γ ( f , 1) = f . This implies also the surjectivity of γ∗ . Next, we note that dimF (Xd1 ⊗F[z] Xd2 ) = deg(d1 ∧ d2 ) = dimF Xd1 ∧d2 , which shows that γ∗ is bijective. This proves the isomorphism (8.143).

(8.144) 

8.4 Tensor Products of Models

265

Clearly, we have the dimension formula dimF Xd1 ⊗F[z] Xd2 = dimF X(d1 ∧d2 ) = deg(d1 ∧ d2 ) ≤ max(deg d1 , deg d2 ) (8.145) ≤ deg d1 · degd2 = dimF (Xd1 ⊗F Xd2 ). This computation shows that in the case of tensor products over rings, a lot of dimension collapsing can occur. In fact, in the special case that d1 (z), d2 (z) are coprime, then Xd1 ⊗F[z] Xd2 = 0. The following theorem is the analogue of the isomorphism (8.136) for the case of F[z]-tensor products. We give two versions of it, using both Xd1 ⊗F[z] Xd2 and Xd1 ∧d2 . Theorem 8.72. Given nontrivial polynomials d1 (z), d2 (z) ∈ F[z], then 1. We have the F[z]-isomorphism of F[z]-modules Xd1 ⊗F[z] Xd2  Hom F[z] (Xd1 , Xd2 ).

(8.146)

The isomorphism is given by the map ψ  : Xd1 ⊗F[z] Xd2 −→ Hom F[z] (Xd1 , Xd2 ) defined by f1 ⊗ f2 → ψ f1 ⊗ f2 :

ψ f1 ⊗ f2 (g) = d2 π− ((d1 ∧ d2 )−1 f1 f2 g) =



d2 d1 ∧ d 2



πd1 ∧d2 ( f1 f2 g),

g ∈ Xd1 . (8.147)

2. We have the F[z]-isomorphism of F[z]-modules Xd1 ∧d2  Hom F[z] (Xd1 , Xd2 ).

(8.148)

The isomorphism is given by the map ψ : Xd1 ∧d2 −→ Hom F[z](Xd1 , Xd2 ) defined by n → ψn defined, for n ∈ Xd1 ∧d2 , by

ψn (g) = πd2 (n2 g),

g ∈ Xd1 .

(8.149)

Proof. 1. We use the isomorphism (8.143) to identify Xd1 ⊗F[z] Xd2 with Xd1 ∧d2 . Choosing any element f (z) ∈ Xd1 ∧d2 , by construction ψ f is F[z]-linear. To show injectivity of ψ  , assume ψ f = 0, i.e., for all g(z) ∈ Xd1 we have 0 = ψ f g =   d2 d1 ∧d2 πd1 ∧d2 ( f g). This holds in particular for g(z) = 1. Thus πd1 ∧d2 f = 0 and necessarily f (z) ∈ (d1 ∧ d2 )F[z]. However, we assumed f (z) ∈ Xd1 ∧d2 . Thus the direct sum decomposition F[z] = Xd1 ∧d2 ⊕ (d1 ∧ d2 )F[z] implies f (z) = 0, and the injectivity of ψ  follows. Using the F-linear isomorphism (Xd1 ⊗F[z] X d2 )∗  Hom F[z] (Xd1 , Xd2 ), it follows that dimF Hom F[z] (Xd1 , Xd2 ) = dimF Xd1 ⊗F[z] Xd2 = dimF Xd1 ⊗F[z] Xd2 . Hence injectivity implies surjectivity, and we are done. n(z) 2. Note that n(z) ∈ Xd1 ∧d2 if and only if h(z) = d (z)∧d ∈ X d1 ∧d2 = X d1 ∩ X d2 . 1 2 (z) In turn, h(z) is rational and strictly proper and has a unique representation of

266

8 Tensor Products and Forms

the form h(z) = d (z)∧d (z) . Writing di (z) = (d1 (z) ∧ d2 (z))di (z), we have, with 1 2 ni (z) = n(z)di (z), n(z)

h(z) =

n(z) n1 (z) n2 (z) = = , d1 (z) ∧ d2 (z) d1 (z) d2 (z)

(8.150)

which also implies the intertwining relation n2 (z)d1 (z) = d2 (z)n1 (z).

(8.151)

We compute now, with n(z) = πd1 (z)∧d2 (z) ( f1 (z) f2 (z)),

ψ f1 ⊗ f2 g = d2 π− (d1 ∧ d2 )−1 f1 f2 g = d2 π− (d1 ∧ d2 )−1 (πd1 ∧d2 ( f1 f2 ))g = d2 π− (d1 ∧ d2 )−1 ng = d2 π− d2−1 n2 g = πd2 n2 g = ψn (g).  The most important aspect of Theorem 8.72 is the establishment of a concrete connection between the space of maps Z intertwining the shifts Sd1 and Sd2 , i.e., satisfying ZSd1 = Sd2 Z on the one hand, and the F[z]-tensor product of the polynomial models Xd1 and Xd2 on the other. Note that (8.149) is the more familiar representation of an intertwining map; see Theorem 5.17. Next, we turn to the identification of the dual to the tensor product. Proposition 8.73. Given nontrivial polynomials d1 (z), d2 (z) ∈ F[z], then we have the series of F[z]-isomorphisms (Xd1 ⊗F[z] Xd2 )∗  (Xd1 ∧d2 )∗  X d1 ∧d2  X d1 ∩ X d2  Xd1 ⊗F[z] Xd2  Hom F[z] (Xd1 , Xd2 ).

(8.152)

Proof. We use the duality results on polynomial models, as well as the isomorphisms (8.143) and (8.148). 

8.4.7 The Polynomial Sylvester Equation Note that the polynomial models Xd1 and Xd2 not only have a vector space structure but are actually, by (5.8), F[z]-modules. This implies that Xd2 (z)d1 (w) , and hence also, using the isomorphism (8.132), Xd2 ⊗F Xd1 , have natural F[z, w]-module structures. This is defined by p(z, w) · q(z, w) = πd2 (z)⊗d1 (w) (p(z, w)q(z, w)), where p(z, w) ∈ F[z, w].

q(z, w) ∈ Xd2 (z)⊗d1 (w) , (8.153)

8.4 Tensor Products of Models

267

Similarly, we define an F[z, w]-module structure on the tensored rational model X d2 (z)⊗d1 (w) by letting, for p(z, w) = ∑ki=1 ∑lj=1 pi j zi−1 w j−1 ∈ F[z, w] and h(z, w) ∈ X d2 (z)⊗d1 (w) , p(z, w) · h(z, w) = π d2 (z)⊗d1 (w) [∑ki=1 ∑lj=1 pi j zi−1 h(z, w)w j−1 ].

(8.154)

Proposition 8.74. Let d2 (z) ∈ F[z] and d1 (w) ∈ F[w] be nonsingular polynomials. Then 1. The F[z, w]-module structure on X d2 (z)⊗d1 (w) defined by (8.154) can be rewritten as k

l

p(z, w) · h(z, w) = (π−z ⊗ π−w) ∑ ∑ pi j zi−1 h(z, w)w j−1 .

(8.155)

i=1 j=1

2. With the F[z, w]-module structure on Xd2 (z)⊗d1 (w) and X d2 (z)⊗d1 (w) , given by (8.155) and (8.154) respectively, the multiplication map d2 (z) ⊗ d1(w) : X d2 (z)⊗d1 (w) −→ Xd2 (z)⊗F d1 (w) is an F[z, w]-module isomorphism, i.e., we have Xd2 (z)⊗d1 (w)  X d2 (z)⊗d1 (w) .

(8.156)

Proof. 1. Follows from (8.154). 2. Follows, using Proposition 8.74, from the fact that h(z, w) ∈ X d2 (z)⊗d1 (w) if and only if h(z, w) = π d2 (z)⊗d1 (w) h(z, w), equivalently, if and only if

πd2 (z)⊗d1 (w) d2 (z)h(z, w)d1 (w) = d2 (z)h(z, w)d1 (w), i.e., d2 (z)h(z, w)d1 (w) ∈ Xd2 (z)⊗d1 (w) .



Special cases that are of interest are the single-variable shift operators Sz , Sw : Xd2 (z)⊗d1 (w) −→ Xd2 (z)⊗d1 (w) , defined by Sz q(z, w) = πd2 (z)⊗d1 (w) zq(z, w) = πd2 (z) zq(z, w), Sw q(z, w) = πd2 (z)⊗d1 (w) q(z, w)w = πI⊗d1 (w) q(z, w)w.

(8.157)

If we specialize definition (8.153) to the polynomial p(z, w) = z − w, we get, with q(z, w) ∈ Xd1 (z)⊗F d2 (w) , S q(z, w) = (z − w) · q(z, w) = πd2 (z)⊗F d1 (w) (zq(z, w) − q(z, w)w).

(8.158)

In fact, with A2 ∈ F p×p and A1 ∈ Fm×m , if D2 (z) = zI − A2 and D1 (w) = wI − A1 , then Q(z, w) ∈ XD2 (z)⊗D˜ 1 (w) if and only if Q(z, w) ∈ F p×m, i.e., Q(z, w)

268

8 Tensor Products and Forms

is a constant matrix. In that case we have XD2 (z)⊗F D˜ 1 (w) = F p×m and (z − w) · Q = π(zI−A2 )⊗(wI−A˜ 1 ) (z − w)Q = A2 Q − QA1 ,

(8.159)

which is the standard Sylvester operator. The equation (z − w) · Q = R reduces in this case to the Sylvester equation A2 Q − QA1 = R.

(8.160)

We proceed now to a more detailed study of the Sylvester equation in the tensored polynomial model framework. We saw above that the classical Sylvester equation AX − XB = C

(8.161)

S Q(z, w) = π(zI−A)⊗(wI−B) ˜ (z − w)Q = C,

(8.162)

corresponds to the equation

with Q,C ∈ X(zI−A)⊗F(wI−B) ˜ necessarily constant matrices. Note that every t(z, w) ∈ Xd2 (z)⊗d1 (w) has a full row rank factorization of the form 1 (w), t(z, w) = R2 (z)R

(8.163)

with R2 (z) ∈ XD2 and R1 (w) ∈ XD1 both of full row rank. The following theorem reduces the analysis of the general Sylvester equation to a polynomial equation of Bezout type. A special case is of course the homogeneous Sylvester equation which has a direct connection to the theory of Bezoutians. Theorem 8.75. Let d2 (z) ∈ F[z] and d1 (w) ∈ F[w] be nonsingular. Defining the Sylvester operator S : Xd2 (z)⊗d1 (w) −→ Xd2 (z)⊗d1 (w) by S q(z, w) = πd2 (z)⊗d1 (w) (z − w)q(z, w) = R2 (z)R˜ 1 (w)

(8.164)

(8.164), then for R2 (z) ∈ Xd2 ⊗I and R˜ 1 (w) ∈ XI⊗d1 we have 1. The Sylvester equation Sd2 q − qSd1 = t(z, w) = R2 (z)R˜ 1 (w)

(8.165)

S q = R2 (z)R˜ 1 (w),

(8.166)

or equivalently is solvable if and only if there exist polynomials n2 (z) ∈ Xd2 (z) and n1 (z) ∈ Xd1 (z) for which d2 (z)n1 (z) − n2 (z)d1 (z) + R2 (z)R˜ 1 (z) = 0. (8.167)

8.4 Tensor Products of Models

269

We will refer to (8.167) as the polynomial Sylvester equation, or PSE. In that case, the solution is given by q(z, w) =

d2 (z)n1 (w) − n2 (z)d1 (w) + R2 (z)R˜ 1 (w) . z−w

(8.168)

2. Q(z, w) ∈ XD2 (z)⊗D˜ 1 (w) solves the homogeneous polynomial Sylvester equation, or HPSE, if and only if there exist polynomials n2 (z) ∈ Xd2 and n1 (z) ∈ Xd1 that satisfy d2 (z)n1 (z) − n2 (z)d1 (z) = 0, (8.169) in terms of which q(z, w) =

d2 (z)n1 (w) − n2 (z)d1 (w) , z−w

(8.170)

i.e., q(z, w) is a Bezoutian. Proof. 1. Assume there exist polynomials n2 (z) ∈ Xd2 ⊗I and n1 (z) ∈ Xg1 solving equation (8.167), and for which q(z, w) is defined by (8.168). We note first that under our assumptions on R2 (z), R1 (z), d2 (z)−1 q(z, w)d1 (w)−1 n1 (w)d1 (w)−1 − d2 (z)−1 n2 (z) + d2 (z)−1 R1 (z)R2 (w)d1 (w)−1 = z−w is strictly proper in both variables, i.e., q(z, w) is in Xd2 (z)⊗d1 (w) . We compute S q(z, w) = πd2 (z)⊗d1 (w) (z − w)q(z, w) = πd2 (z)⊗d1 (w) (d2 (z)n1 (w) − n2 (z)d1 (w) + R2 (z)R1 (w)) = R2 (z)R1 (w), i.e., q(z, w) is indeed a solution. To prove the converse, we note that for the single-variable case, given a nonzero polynomial d2 (z) ∈ F[z], we have, for f (z) ∈ Xd1 , Sd1 f = z f (z) − d1 (z)ξ f , where ξ f = (d1−1 f )−1 . This implies that for q(z, w) ∈ Xd2 (z)⊗d1 (w) , we have Sz⊗1 q(z, w) = zq(z, w) − d2 (z)n1 (w) and S1⊗w q(z, w) = q(z, w)w − n2 (z)d1 (w) with n1 d1−1 , d2−1 n2 strictly proper. Thus, assuming q(z, w) is a solution of the PSE, we have Sz−w q(z, w) = [zq(z, w) − d2 (z)n1 (w)] − [q(z, w)w − n2 (z)d1 (w)] = [zq(z, w) − q(z, w)w] − [d2 (z)n1 (w) − n2 (z)d1 (w)] = R2 (z)R1 (w). Equation (8.162) reduces to

270

8 Tensor Products and Forms

[zQ(z, w) − Q(z, w)w] − [d2 (z)n1 (w) − n2 (z)d1 (w)] = R2 (z)R1 (w), or q(z, w) =

d2 (z)n1 (w) − n2 (z)d1 (w) + R2 (z)R1 (w) . z−w

(8.171)

(8.172)

However, since q(z, w) ∈ Xd2 (z)⊗d1 (w) is a polynomial, we must have that (8.167) holds. 2. Follows from the previous part. 

8.4.8 Reproducing Kernels In Theorem 8.69, we proved the isomorphism (8.136), which shows that any linear transformation K : Xd1 −→ Xd2 can be represented by a unique polynomial k(z, w) ∈ Xd2 (z)d1 (w) , with the representation given, for f (z) ∈ Xd1 , by (K f )(z) =  f (·), k(z, ·).

(8.173)

With the identification Xq∗ = Xq , given in terms of the pairing (5.64), we can introduce reproducing kernels for polynomial models. To this end we consider the ring F[z, w] of polynomials in the two variables z, w. Let now q(z) be a monic polynomial of degree n. Given a polynomial k(z, w) ∈ F[z, w] of degree less than n in each variable, it can be written as k(z, w) = ∑ni=1 ∑nj=1 ki j zi−1 w j−1 . Such a polynomial k(z, w) induces a linear transformation K : Xd −→ Xd , defined by n

n

(K f )(z) =  f (·), k(z, ·) = ∑ ∑ ki j zi−1 w j−1 , f .

(8.174)

i=1 j=1

We will say that the polynomial k(z, w) is the kernel of the map K defined in (8.174). Now, any f (w) ∈ Xd has an expansion f (w) = ∑nk=1 fk ek (w), with {e1 (z), . . . , en (z)} the control basis in Xq . Thus, we compute (K f )(z) = ∑ni=1 ∑nj=1 ki j zi−1 w j−1 , f  = ∑ni=1 ∑nj=1 ∑nk=1 ki j zi−1 fk w j−1 , ek  = ∑ni=1 ∑nj=1 ∑nk=1 ki j fk zi−1 δ jk = ∑ni=1 ∑nj=1 ki j f j zi−1 . This implies that [K]stco = (ki j ). Given a polynomial q(z) of degree n, we consider now the special kernel, defined by q(z) − q(w) k(z, w) = . (8.175) z−w

8.4 Tensor Products of Models

271

Proposition 8.76. Given a monic polynomial q(z) = zn + qn−1 zn−1 + · · · + q0 of degree n, then, with k(z, w) defined by (8.175): 1. We have k(z, w) =

n

n

j=1

j=1

∑ w j−1 e j (z) = ∑ z j−1 e j (w).

(8.176)

 f (·), k(z, ·) = f (z).

(8.177)

2. For any f (z) ∈ Xd , we have

3. We have the following matrix representations: st [K]co co = [K]st = I.

(8.178)

j−1 k j−k−1 Proof. 1. We use the fact that z j − w j = (z − w) ∑k=1 zw to get j j q(z) − q(w) = zn − wn + ∑n−1 j=1 q j (z − w ) = (z − w) ∑k=1 zk wn−k−1 + (z − w) ∑ j=1 q j ∑k=1 zk w j−k−1 = (z − w) ∑ j=1 w j−1 e j (z)

= ∑nj=1 w j−1 e j (z). The equality and hence q(z)−q(w) z−w follows by symmetry. 2. Let f (z) ∈ Xq and f (z) = ∑nj=1 f j e j (z). Then

q(z)−q(w) z−w

= ∑nj=1 z j−1 e j (w)

k(z, ·), f  = ∑nj=1 w j−1 e j (z), ∑nk=1 fk ek (w) = ∑nj=1 ∑nk=1 fk e j (z)w j−1 , ek (w) = ∑nj=1 ∑nk=1 fk e j (z)δ jk = ∑nk=1 fk ek (z) = f (z). 3. Obviously, by the previous part, K acts as the identity; hence its matrix representation in any basis is the identity matrix. The matrix representations in (8.178) can also be verified by direct computations.  A kernel with the properties described in Proposition 8.76 is called a reproducing kernel for the space Xq . We note that the map K : Xq −→ Xq , defined by Proposition 8.76, is the identity map. As such, it clearly satisfies ISq = Sq I, i.e., it is an intertwining map. Of course, not every map in Hom F (Xq , Xq ) is intertwining. What makes K an intertwining map is the special form of the kernel given in (8.175), namely the fact that the kernel is one of the simplest Bezoutians, a topic we will pick up in the sequel. Given a polynomial M(z, w) = ∑ni=1 ∑nj=1 Mi j zi−1 w j−1 in the variables z, w, we have an induced bilinear form in Xq × Xq defined by

φ ( f , g) = M f , g = M(z, ·), f w gz .

(8.179)

272

8 Tensor Products and Forms

In turn, this induces a quadratic form given by φ ( f , f ) = M f , f  = M(z, ·), f w f z . Proposition 8.77. Let q(z) ∈ F[z] and let Bst and Bco be the standard and control bases respectively of Xq , defined in Chapter 5. 1. The matrix representation of the bilinear form φ ( f , g), given by (8.179), with respect to the control and standard bases of Xq is (Mi j ). 2. φ is symmetric if and only if (Mi j ) is symmetric. 3. The form φ is positive definite if and only if (Mi j ) is a positive definite matrix. Proof. 1. Mi j , the i j entry of [M]stco , is given by Me j , ei  = ∑nk=1 ∑nl=1 Mi j zk−1 wl−1 , e j w , ei z = ∑nk=1 ∑nl=1 Mi j zk−1 wl−1 , e j w , ei z = ∑nk=1 Mk j zk−1 , ei z = ∑nk=1 Mk j δik = Mi j . 2. Follows from (8.179) and the previous computation. 3. Follows from the equality φ ( f , f ) = ([M]stco [ f ]co , [ f ]co ).



In the next subsection, we will return to the study of representations of general linear transformations in terms of kernels. This will be done using the polynomial representations of the tensor products of polynomial models, both over the underlying field F and over the polynomial ring F[z].

8.4.9 The Bezout Map Since a map intertwining the polynomial models Xq1 and Xq2 is automatically linear, there is a natural embedding of Hom F[z] (Xd1 , Xd2 ) in Hom F (Xd1 , Xd2 ). This, taken together with the isomorphisms (8.137) and (8.146), shows that there exists a natural embedding of Xd1 ⊗F[z] Xd2 in Xd1 ⊗F Xd2 . In order to make this embedding specific, we use concrete representations for the two tensor products, namely the isomorphisms (8.136) and (8.146). One result of using concrete representations for the tensor product is that they reveal, in the clearest possible conceptual way, the role of the Bezoutians in the analysis of maps intertwining two shifts. Theorem 8.78. Given d1 (z), d2 (z) ∈ F[z] − {0}, let Ψ : Xd1 (z)d2 (w) −→ Hom F (Xd1 , Xd2 ) be given by (8.135) and let ψ : Xd1 ⊗F[z] Xd2 −→ Hom F[z](Xd1 , Xd2 ) be given by (8.147). Let i : Hom F[z] (Xd1 , Xd2 ) −→ Hom F (Xd1 , Xd2 ) be the natural embedding. For n(z) ∈ Xd1 ∧d2 having the representation (8.150), we define the Bezout map β : Xd1 ∧d2 −→ Xd1 (z)d2 (w) by

β (n) = q(z, w) =

d2 (z)n1 (w) − n2 (z)d1 (w) , z−w

(8.180)

8.4 Tensor Products of Models

273

i.e., q(z, w), denotes the Bezoutian based on the intertwining relation (8.151). Then β is injective, and we have the equality

Ψ ◦ β = i ◦ ψ,

(8.181)

i.e., the Bezout map is the concretization of the embedding of Xd1 ⊗F[z] Xd2 in Xd1 ⊗F Xd2 . Proof. Note, as above, that any element n(z) ∈ Xd1 ∧d2 yields a unique strictly n(z) proper function h(z) = d (z)∧d = nd1 (z) = nd2 (z) . Thus n(z) determines unique 1 2 (z) 1 (z) 2 (z) polynomials n1 (z), n2 (z) satisfying the intertwining relation (8.151). By inspection, d1 (z)−1 q(z, w)d2 (w)−1 ∈ z−1 Fsep [[z−1 , w−1 ]]w−1 and thus q(z, w) ∈ Xd1 (z)d2 (w) . This shows that β is well defined and F-linear. The map β is injective, since q(z, w) = 0 n2 (w) implies, by (8.150), that nd1 (z) (z) = d (w) is constant. In turn, we conclude that h(z) = 1

2

n(z) d1 (z)∧d2 (z)

is a constant which, by strict properness, is necessarily zero. To prove the equality (8.181), we compute, with β (n) = q(z, w) given by (8.180), & % d2 (z)n1 (w) − n2 (z)d1 (w) g, q(z, ·) = g, z−w ! " d2 (z)n1 (w) − n2 (z)d1 (w) −1 = d1 (w) g(w), z−w  w d2 (z)n1 (w) − n2 (z)d1 (w) −1 d1 (w) g(w) = z−w w−1  d2 (z)n1 (w)d1 (w)−1 g(w) n2 (z)g(w) − = z−w z−w −1 = −d2 π+ n1 d1−1 g + n2g = n2 g − d2π+ d2−1 n2 g = πd2 n2 g.

(8.182)

Here (F(z, w))w−1 denotes the coefficient of w−1 in the Laurent expansion of F(z, w) with respect to the variable w.  The availability of the Bezout map and the representation (8.182) allow us to extend Theorem 5.17, giving a characterization of elements of Hom F[z] (Xd1 , Xd2 ). Theorem 8.79. Given nonzero polynomials d1 (z), d2 (z) ∈ F[z], then Z ∈ Hom (Xd1 , Xd2 ), i.e., it satisfies ZSd1 = Sd2 Z, if and only if there exist polynomials n1 (z), n2 (z) satisfying n2 (z)d1 (z) = d2 (z)n1 (z), (8.183) and for which Z has the representation Zg = πd2 n2 g, Proof. Follows from Theorem 8.78.

g(z) ∈ Xd1 .

(8.184) 

274

8 Tensor Products and Forms

8.5 Exercises 1. Jacobi’s signature rule. Let A(x, x) be a Hermitian form and let Δ i be the determinants of the principal minors. If the rank of A is r and Δ1 , . . . , Δ r are nonzero then

π = P(1, Δ1 , . . . , Δr ),

ν = V (1, Δ1 , . . . , Δr ),

where P and V denote, respectively, the number of sign permanences and the number of sign changes in the sequence 1, Δ 1 , . . . , Δ r . m n m i i 2. Let p(z) = ∑m i=0 pi z = pm ∏i=1 (z − αi ) and q(z) = ∑i=0 qi z = qn ∏i=1 (z − βi ). n m Show that detRes (p, q) = pm qn ∏i, j (βi − α j ). 3. Show that a real Hankel form n

n

Hn (x, x) = ∑ ∑ si+ j−1 ξi ξ j i=1 j=1

is positive if and only if the parameters sk allow a representation of the form sk =

n

∑ ρ j θ jk ,

j=1

with ρ j > 0 and θ j distinct real numbers. 4. Given g1 , . . . , g2n−1 , consider the Hankel matrix Hn given by (8.66). Let a(z) = n−1 i i ∑n−1 i=0 ai z and x(z) = ∑i=0 xi z be the polynomials arising out of the solutions of the system of linear equations ⎛

g1 ⎜. ⎜ ⎜ ⎜. ⎜ ⎝. gn

⎞⎛ ⎞ ⎛ ⎞ . . . gn x0 r1 ⎟⎜. ⎟ ⎜. ⎟ .... ⎟⎜ ⎟ ⎜ ⎟ ⎟⎜ ⎟ ⎜ ⎟ .... ⎟⎜. ⎟ = ⎜. ⎟ ⎟⎜ ⎟ ⎜ ⎟ ⎠⎝. ⎠ ⎝. ⎠ .... . . . g2n−1 xn−1 rn

with the right-hand side being given respectively by  ri = and

0, i = 1, . . . , n − 1, 1, i = n, 

ri =

1, i = 1, 0, i = 2, . . . , n.

8.5 Exercises

275

Show that if a0 = 0, we have xn−1 = a0 , the Hankel matrix Hn is invertible, and its inverse is given by Hn−1 = B(y, a), where the polynomial y is defined by y(z) = a(0)−1 zx(z) = q(z) − a(0)−1q(0)a(z). 5. Show that a Hermitian Toeplitz form n

Tn (x, x) = ∑

n

∑ ci− j ξi ξ j

i=1 j=1

with the matrix

⎛

c0 ⎜ . ⎜ ⎜ Tn = ⎜ . ⎜ ⎝ . cn

. . c−n+1 .. . .. . .. . .. .

⎞ c−n . ⎟ ⎟ ⎟ . ⎟ ⎟ . ⎠ c0

is positive definite if and only if the parameters ck = c−k allow a representation of the form ck =

n

∑ ρ j θ jk ,

j=1

with ρ j > 0, |θ j | = 1, and the θ j distinct. 6. We say a sequence c0 , c1 , . . . of complex numbers is a positive sequence if for all n ≥ 0, the corresponding Toeplitz form is positive definite. An inner product on C[z] is defined by ⎛

c0 ⎜ . ⎜  ⎜ a, bC = a0 . . . an ⎜ . ⎜ ⎝ . cn

⎞⎛ ⎞ . . . cn b0 ⎜ ⎟ ... . ⎟ ⎟⎜ . ⎟ n n ⎟⎜ ⎟ . . . . ⎟ ⎜ . ⎟ = ∑ ∑ c(i− j) ai b j . ⎟⎜ ⎟ . . . . ⎠ ⎝ . ⎠ i=0 j=0 . . . c0 bn

Under our assumption of positivity, this is a definite inner product on C[z]. a. Show that the inner product defined above has the following properties: zi , z j C = c(i− j) and za, zbC = a, bC . b. Let φn be the sequence of polynomials, obtained from the polynomials 1, z, z2 , . . . by an application of the Gram-Schmidt orthogonalization procedure. Assume all the φn are monic. Show that

276

8 Tensor Products and Forms

φ0 (z) = 1,   c  0  .  1  φn (z) =  . det Tn−1   cn−1   1 c. Show that

φn , zn−i C = 0,

d. Show that

.. . .. . .. . . . c0 . . zn−1

 c−n  .   . .  c−1  zn 

i = 0, . . . , n − 1. n−1

φn (z) = 1 − z ∑ γi+1 φi (z), i=0

where the γi are defined by

γi+1 =

1, zφi  .  φi  2

The γi are called the Schur parameters of the sequence c0 , c1 , . . . . e. Show that the orthogonal polynomials satisfy the following recurrences  φn (z) = zφn−1 (z) − γn φn−1 (z),  φn (z) = φn−1 (z) − zγn φn−1 (z),

with the initial conditions for the recurrences given by φ0 (z) = φ0 (z) = 1. Equivalently, 

f. Show that g. Show that

φn (z) φn (z)



 =

z − γn −zγn 1



φn−1 (z)  φn−1 (z)

  ,

φ0 (0) φ0 (0)

 =

  1 . 1

γn = −φn (0). φn 2 = (1 − γn2 )φn−1 2 /

h. Show that the Schur parameters γn satisfy |γn | < 1. i. The Levinson algorithm. Let {φn (z)} be the monic orthogonal polynomials i associated with the positive sequence {cn }. Assume φn (z) = zn + ∑n−1 i=0 φn,i z .

8.6 Notes and Remarks

277

Show that they can be computed recursively by rn+1 = (1 − γn2 )rn , 1 n γn+1 = ∑ ci+1 φn,i , rn i=0 φn+1 (z) = zφn (z) − γn+1 φn (z),

r0 = c0 ,

φ0 (z) = 1.

j. Show that the shift operator Sφn acting in Xφn satisfies i

Sφn φi = φi+1 − γi+1 ∑ γ j Πνi = j+1 (1 − γν2)φ j + γi+1 Πνi =1 (1 − γν2 )φ0 . j=1

k. Let c0 , c1 , . . . be a positive sequence. Define a new sequence cˆ0 , cˆ1 , . . . through the infinite system of equations given by ⎞ ⎛ ⎞ 1 cˆ0 /2 c0 /2 0 . . . ⎜ ⎟ ⎜0⎟ c ˆ ⎜ ⎟ 1 ⎟ ⎜ c1 c0 /2 0 . . ⎟ ⎜ ⎟ ⎜ ⎟ ⎟⎜ ⎜ = . ⎜ . ⎟. ⎟ ⎜ ⎝ c2 c1 . . . ⎠ ⎜ ⎟ ⎜ ⎟ ⎝ . ⎠ ⎝.⎠ . . . .. . . ⎛

⎞

⎛

Show that cˆ0 , cˆ1 , . . . is also a positive sequence. l. Show that the Schur parameters for the orthogonal polynomials ψn , corresponding to the new sequence, are given by −γn . m. Show that ψn satisfy the recurrence relation  ψn = zψn−1 + γn ψn−1 ,  ψn = ψn−1 + zγn ψn−1 .

The initial conditions for the recurrences are ψ0 (z) = ψ0 (z) = 1.

8.6 Notes and Remarks The study of quadratic forms owes much to the